You might be thinking: "But, if I see an result/error value that I didn't expect while running my program, the stack trace will help me track down the issue!" Yeah, no kidding. So, let's also start adding stack traces to our successful values, too! After, all, if I call my division function and get back a `Result::Ok` with a weird number that I didn't expect, I might want to trace that back, too, right? (This suggestion is sarcastic to prove a point. It should, hopefully, sound ridiculous to add stack traces to every return value from every function.)
I don't think I disagree with the ends you're proposing (don't add stack traces to every value, don't add stack traces specifically to Result::Err(E) variants); however, this is a bad way to justify it. Tools like dtrace / bpftrace do exactly this kind of stack tracing for both success and error cases across entire systems. This is a good thing™, and is actually very useful for both debugging, performance profiling, and understanding what your code is really doing on the hardware.
So I guess I disagree with how you're framing it. I would argue that adding stack traces to every value in Rust would be bad because it is a lot of overhead for something your kernel can and will do better.
The issue is that Rust's Result (and Java's checked exceptions) require a different paradigm. A Result is in the type signature because it's part of your domain's API design. It's just values. It's not for* debugging. You use a debugger for that or programmatically panic when something is truly unexpected and get the stack trace from that.*
This really is the gist of it. However, I will say that in my experience the reason that Result types are nice (over e.g. exceptions) is that putting the error cases in the type contract means that you can have the compiler check when someone hasn't handled an error case (? and unwrap are "handling" it even if they may not always be appropriate), as well as statically verify which variants may be unused. One very frustrating thing I've had to encounter in C++ is finding a whole list of different errors that have been duplicated as multiple different opaque (e.g. behind a unique_ptr<std::exception> or some such) exceptions across the codebase.
Being able to know what variants of error can come out of an API is great! It just happens that working with a rich type system like Rust makes it possible to do all manner of things that languages-with-only-exceptions cannot.
Quick nit/question. For the fold method, I don't think I've seen it called sentinel value. Usually it is seed or initial?
I think in Scheme it is common to call it knil, mirroring how lists use nil as the "sentinel" value which marks the end of a proper list. I opted to name it sentinel in that article (and in the docs) for two reasons:
2. Transducers necessarily abstract across a lot more than loops / lists. Lisps do a lot of really cool (and optimized) stuff with lists alone and Scheme is no different in this regard. However, because of how Scheme primarily exports list operations in (scheme base) is really easy to run into a situation where lists are used in an ad-hoc way where another data structure is more appropriate. This includes vectors, sets, mappings, hashmaps, etc. Transducers-the-library is meant to be general across operations that work on all of these types, so I chose language that intentionally departs from thinking in a list-focused way.
Now, my main question. Transducer? I'm curious on the etymology of that word. By itself, I don't think I could ever guess what it was referencing. :(
It's not a reducer, because they serve as higher-order functions that operate on reducers. Instead, the values they accept are transient through the function(s), so they transduce. You should watch the video, I think Rich explains the origins of his language very well.
Author of the post here, hi! Funny to see this resurface again, I have made a number of changes to the transducers library since this blog post (see: https://wiki.call-cc.org/eggref/5/transducers).
A vector-reduce form would be trivial but icky, and I chose not to do it to not have to have the continuation safety discussion.
I am not sure what "continuation safety" refers to in this context but I wanted a library that would give me a good out-of-the-box experience and have support for commonly used Scheme types. I have not yet added any folders/collectors/transducers specific to some types (like anything related to streams or SRFI-69), but I think a broad swath of types and patterns are currently covered.
I think in particular my griped regarding vectors were that collectors such as `collect-vector`, `collect-u8vector`, etc. were not implemented. There is a chance to break out of these collectors using continuations but that's not really a good argument to not have them (I hope this is not what you're referring to!).
Anyway, if I read things correctly the complaint that srfi-171 has delete dupes and delete neighbor dupes forgets that transducers are not always used to or from a data structure. They are oblivious to context. That is why both are necessary.
I think this is exactly my argument: they are oblivious to context and actually do the wrong thing by default. I've seen this happen in Rust with users preferring `dedup` or `dedup_by` (from the Itertools crate) rather than just constructing a HashSet or BTreeSet. It almost always is used as a shortcut to save on a data structure, and time and again I've seen it break workflows because it requires that the chain of items is first sorted.
I think this is is particularly damning for a library that means to be general purpose. If users want to implement this themselves and maintain it within their own code-bases, they're certainly welcome to; however, I don't personally think making this kind of deduping "easy" helps folks in the general sense. You'd be better off collecting into a set or bag of some kind, and then transducing a second time.
From what I can see the only differences are ordering of clauses to make the transduce form generic and naming conventions. His library shadows a bunch of bindings in a non-compatible way. The transduce form is still not generic but moves the list-, vector-, generator- part of transduce into a "folder". Which is fine. But a generic dispatch would be nicer.
Shadowing bindings in a "non-compatible" way can be bad, but it also helps to make programs more clean. If you're using transducers across your codebase, you almost certainly aren't also using e.g. SRFI-1's filter.
As for generic dispatch: I agree wholeheartedly. I wish we had something like Clojure protocols that didn't suck. I've looked into ways to (ab)use variant records for this sort of thing, but you run into an open/closed problem on extending the API. This is really something that needs to be solved at the language level and something like COOPS / GOOPS incurs a degree of both conceptual and actual performance overhead that makes them somewhat unsatisfying :(
And also: thank you for SRFI-171. I disagree with some of the design decisions but had it not been written I probably wouldn't have even considered transducers as something worth having.
Of course, this requires buying into a set of tooling and learning a lot of specific idioms. I can't say I've used it, but from reading the docs it seems sound enough.
I think in terms of use-cases there is bound to be a lot of overlap. Of course, in terms of comparable products, I'd have to mention that Luxonis sells a good number of different products, so understand that there's a necessary disclaimer here that HiFi isn't going to replace every possible option that Luxonis offers (especially their modules / full robot offerings, HiFi is just the sensor!).
In terms of how we differentiate ourselves, I think the main focus is going to be primarily in terms of depth quality and software. Our expertise in providing robust calibrations, combined with the improved optics on the HiFi allow us to produce much higher quality depth frames, more consistently, than what we see on the market today.
In fact, the whole purpose of building this sensor was to bring to market a 3D depth camera that provides good quality data, always, to better enable long-term autonomy. AI tools and capabilities enhance that data in a way that customers have told us existing market offerings are currently lacking.
As for software: We're a software company at heart, which helps, and HiFi is meant to be a flagship representation of what our software can power. We've used a lot of sensors, sensor APIs, SDKs, etc. and the number one thing we find is that these systems are complex, opaque, and difficult to debug or understand. A big part of designing software for me personally is producing software that is legible; not in the sense that one can literally read it (because reading code isn't easy), but in the sense that the software itself can be understood in the abstract. We're hoping that when we ship HiFi and the corresponding SDK that folks will appreciate the steps we've taken to make working with it understandable and obvious.
It would perhaps be more accurate to say that this is a big leap forward compared to most existing off-the-shelf depth cameras for robotics. To address the iPhone specifically: you probably aren't going to mount iPhones on a bunch of production robots in the field.
Comparing to other alternatives in the robotics space (I've listed RealSense and Structure above, but there are others), there is somewhat of a laundry list of potential pitfalls and issues that we've seen folks trip over again and again.
Calibration is a big one, and a large part of what we're doing with HiFi is launching it with it's own automatic, self-calibration process (no fiducials). There are some device failures that a process like this wouldn't be able to handle, but the vast majority of calibration problems in the field result from difficult tooling or requirements, a need to supply one's own calibration software, or a combination of hardware and software that make the process difficult. A nickel for every time someone has to train a part-time operator to fix calibration in the field, and I'd own Amazon.
Depth quality and precision is another big pitfall — there are folks out there today using RealSense for their robot, but we've talked to a number of folks who just don't rely on the on-board depth. It's too noisy, it warps flat surfaces, etc. Lots of little details that on the surface you might not think about when just looking at a list of cameras! Putting our edge AI capabilities aside, the improved optics and compute available on the HiFi allow us to build a sensor that always provides good depth. That sounds like a baseline for this kind of tech, but there's plenty of examples otherwise on the market today!
Software is probably the other last big thing that we really want to leap forward on. We don't have too much to say about our SDK today, but when we launch it we hope to make working with these sensors a lot easier. I work with RealSense quite a bit (I am the maintainer of realsense-rust), and quite honestly what has been a solid overall hardware package for many years (until HiFi, I hope) is let down by how confusing it is to use librealsense2 in any meaningful project.
Needless to say, I think HiFi stands on some solid merits and I'm not sure it can be directly compared to other 3D sensors in e.g. iPhones, mostly because the expected use-case is so utterly different.
Hey all, senior engineer on the Tangram team here — we're really excited to be launching HiFi today!
Through past work with similar sensors [0][1], we've heard a lot of feedback about what people want from a depth camera! With our expertise in calibration & shipping sensor SDKs in the past, we saw this as an amazing opportunity to ship what we think is a big leap forward in sensing: 1.6MP cameras, AI capability (up to 8 TOPS / 8GiB onboard memory driven via TIDL), and what is most relevant to me: self-calibration and rock-solid software support.
I've spent the better part of my career building and helping develop multi-sensor systems! We hope you'll pick HiFi for your project (and even if you don't, none of our software is locked to any specific hardware vendor). HiFi is a great chance for us to flagship our software on a first-class piece of hardware, and we want to share that superpower with all of you!
I think the core difference is that as a teen we see people reject tech and assume that it's the tech itself being rejected. That there is some underlying progress being shunned.
As an adult I realize that the tech is a layer of gloss and glamour on actively making the world worse than I knew it could be. I didn't have that perspective as a teen because as a teen I hadn't known the world.
> but I have thought about generalising things like numerical ranges by having something like unfold-transduce.
This is more or less what I was wondering about. Numerics, ports, SRFI-41 streams, etc. There's a lot of stuff that isn't in e.g. r7rs-small but is more or less expected in most Scheme implementations.
> That's also an outlier, but less than 30k isn't which makes this post somewhat misleading (at best).
I don't disagree that this _isn't_ misleading in a way but it's also important to consider that most EVs are subject to what you said right after:
> Ford's cars are probably too expensive because they ignored the EV market for a decade and it takes time to scale up production and reduce costs.
Notably that Tesla still isn't at a scale of production where walking off the lot with a < $30k vehicle is possible! There usually isn't a lot to walk off of, but its rare that the base model will be available at all!
Most of the cars they are producing are not the base package, and likely will never be. The thing about supply chains here is that the "base model" is a much smaller percentage of actual manufacturing. Most of the time if a manufacturer were to produce 1M units a year, less than 30% will be the very base model. I'm not sure there are clear numbers published anywhere, but you can bet that while the "base model" might seem affordable you're unlikely to find it due to the same supply chain constraints — Tesla would much rather up-sell you to some package above the base model and availability is restricted as-is, so which customers do you think they'll prioritize? Tesla isn't even producing millions of units yet (I think last year was just over 600k?), and this is just a drop in the bucket in the number of cars purchased each year.
> I'd guess this is because Ford can't profitably make an affordable sedan (they can't do so without losing a lot of money) so they need to make a more expensive vehicle with better margins, Tesla did this too of course - they just had a decade head start while the legacy manufacturers did nothing.
They don't "need" to make larger vehicles with better margins, they just prefer to have better margins. A lot of this is a downstream effect of CAFE exemptions on "light trucks," which applies to both SUVs and modern pick-up trucks. I guess my disagreement here isn't that they "can't" make sedans unless you're defining "can't" in terms of "what the shareholders said." We can absolutely regulate these things and it would probably be beneficial to do so.
As for why Tesla isn't making massive trucks? EV physics (weight of batteries vs. amount of energy to move said weight) somewhat preclude this, but counterpoint there is that Tesla is making a truck and wants to be on top of that market. Sedans are in a weird space with EVs since the added weight kind of puts you in a range<->weight arms race. While most people probably don't need as much range as they think they do, you end up picking between the atrocious Hummer EV or a Model 3 (which is still very large for most of the trips you'd take with a sedan!!!).
Look I'm not exactly engaged enough to dismantle this piece by piece so this will probably be my last comment but:
> Ambivalence is the human default, and logic requires it must be so.
You'd do well to do more than assert it. This is ideology.
> You loudly proclaim your aversion to all human suffering, past and present, and claim to know how to fix it.
I said no such thing, and the remainder of your prior statements are also asserting I made any such claim. Making efforts to fix wrongs is not itself a moral failure, nor is it some kind of foolish pride.
> We can't address ALL suffering. That doesn't mean that we can't address ANY suffering.
What is odd to me is that this is exactly my point. If you somehow think that racism isn't still "in front of us" as you so boldly claim, I encourage you to prove that substantially and convince the people who to this day still feel victimized by it.
> But the solution to the KKK (the original recipe anti-black version) is not to invent a ~KKK (the crispy anti-white version) and tell whites that if they don't join ~KKK then they are in the KKK.
I haven't claimed this at all. For what its worth though — you are in some form invoking the paradox of intolerance here. I'm not sure why you felt the need to write this screed, it is entirely separate from anything I've said and completely off-the-rails.
I think it is probably unwise to pre-suppose an extreme here (that society will never "progress").
The default action today is "do nothing and don't acknowledge the problem." Suggesting any action be taken against that status quo does not in any way suggest that it is a permanent inviolable law that society must continuously optimize for nor does it suggest that it can't be done in tandem with other "progress" society may achieve.
> Yeah, every time the discussion comes up about, for example, reparations for the descendants of slaves, I start out thinking: it's been 150 years!
Well, that might be a bit disingenuous. The "last chattel slave" was only freed around September 1942. I've seen this reference in several places, but the most direct one is a footnote on a wikipedia page [0].
Regardless, it is probably not worth putting a time limit on suffering. The children and grandchildren of enslaved black people are still alive today! Waving it away with "time has passed" seems more an attempt to bury the issue than to approach it with some semblance of acknowledging the wrong done.
The incentive structures of the organizations producing the software.
If your software is slow, it's probably because nobody was incentivized to make it not-slow. There's plenty of software out there that's fast, but if perceived slowness isn't costing someone money and making a company's life harder, it's unlikely to get solved.
Two examples:
1. My gmail can be horrifically slow some days. It's not because Google is unable to make IMAP sync fast, or that there's some inherent protocol limitation for what I perceive as 'being slow.' I run my own mail on DigitalOcean and I consistently am faster to sync large quantities of mail relative to a gmail address. Google isn't incentivized to make their free service faster, since they aren't losing money when I complain that gmail is 5 seconds slower to sync similar or lower volumes of mail compared to my own personal server. Note: I understand why there's good reasons for this.
2. Netflix and Youtube have a ton of backend servers and services that can be selected to immediately stream to any given computer on the planet, should someone click on a video. These are extremely highly performant and allow basic compute-heavy loads to be spun up at literally the click of a button on a webpage. It is specifically some engineer's job to get delivery windows on videos down as much as humanly possible because the organization makes more money from users (people tend to avoid services with huge delays, thus decreasing ad / subscription revenue) and spends less money if they aren't wasting compute.
Software speed is entirely dictated by the incentive structure around it. If you want to complain about why software is slow, it is useful to understand who's problem that is, and whether perceived performance is enough of a blip on the radar to be worth investing someone's time in it.
> If the solution is to ban cars because you don't like the noise, it sounds like you just don't like cars.
Guilty as charged. That said: the noise itself isn't entirely just the problem so much as how prolonged it is. Having buses and readily-available public transport that are only loud occasionally and on a fixed schedule isn't the same as a constant stream of traffic. Even if the bus or train is louder (and light rail is usually much, much quieter), it isn't all-day, only at fixed intervals.
There are marginal gains to be had. Reducing the amount of driving and traffic in our cities is good for a lot of reasons, but particularly with noise it's a combination of speed, vehicle size, and overall frequency of vehicles that makes a killer combination.
The solution is to place noise restrictions on vehicles (this goes against manufacturers putting off any kind of noise reduction), reducing speeds (this involves walkable / bikeable infrastructure), and expanding public transit like bus and light rail access to reduce the total number of personal vehicles on the road.
Yes, this will make things "worse" for drivers. But it's not the end of the world. If you ever visit Zurich or Geneva, you'll be able to distinctly tell the difference between an American city and a European city in terms of noise alone :) Zurich's tram system is honestly something that every city should dream of having, and it is as quiet as one could hope for.
Some of these are things we can control with infrastructure and regulation (e.g. why is your airport so close to housing?), and others aren't really what I would consider problematic noise.
Sirens are an unfortunate reality, but the reason they're so loud is because most cars have a ton of sound-proofing. I'm sure you know this, but you can hear sirens blocks away when they're trying to alert a driver... maybe 20 metres in front of them to move out of the way?
As for excitable drunks, I think there are places where that's a constant problem, but again, this isn't constant and permanent. Traffic noise and airplanes are pretty constant though - I live on what should be a pretty quiet stretch of land but the local stroad always has at least a few really loud cars in the wee hours of the morning. The rest of the day it's just lots of regular cars making a lot of noise as cars are want to do. At least with the excitable drunks they don't make noise all hours of the day, since they have to sleep or be drinking at some point!
_The_Brownsville_Herald.jpg){kind=link}
I don't think I disagree with the ends you're proposing (don't add stack traces to every value, don't add stack traces specifically to Result::Err(E) variants); however, this is a bad way to justify it. Tools like dtrace / bpftrace do exactly this kind of stack tracing for both success and error cases across entire systems. This is a good thing™, and is actually very useful for both debugging, performance profiling, and understanding what your code is really doing on the hardware.
So I guess I disagree with how you're framing it. I would argue that adding stack traces to every value in Rust would be bad because it is a lot of overhead for something your kernel can and will do better.
The issue is that Rust's Result (and Java's checked exceptions) require a different paradigm. A Result is in the type signature because it's part of your domain's API design. It's just values. It's not for* debugging. You use a debugger for that or programmatically panic when something is truly unexpected and get the stack trace from that.*
This really is the gist of it. However, I will say that in my experience the reason that Result types are nice (over e.g. exceptions) is that putting the error cases in the type contract means that you can have the compiler check when someone hasn't handled an error case (? and unwrap are "handling" it even if they may not always be appropriate), as well as statically verify which variants may be unused. One very frustrating thing I've had to encounter in C++ is finding a whole list of different errors that have been duplicated as multiple different opaque (e.g. behind a unique_ptr<std::exception> or some such) exceptions across the codebase.
Being able to know what variants of error can come out of an API is great! It just happens that working with a rich type system like Rust makes it possible to do all manner of things that languages-with-only-exceptions cannot.