This is generally true. What I find fascinating is the application of information that is not present in the data but that can be confirmed by the data. I’m thinking specifically of deblurring algorithms people use to recover text. If you can guess what font is being used in a screengrab with, say, someone’s bank account blurred out, you can take the i formation contained in the downloaded font and try combinations until you match the blur algorithm that may also be information not contained in the image.
What’s also crazy, is that even if all you do is black out the text, kerning alone can still be used to guess the correct text with some degree of accuracy. I worked on redaction software at one point, and these were very much the types of things we thought about when dealing with text.
There are people working on this problem honestly. The general solution 10 years ago was a micro kernel. Today, I’m not sure. The linux model is starting to look dated, with similar problems elsewhere. Modern hardware design looks less and less like classic textbook design, with all kinds of random chips having direct memory access to memory the cpu uses on some shared bus. Where even things like on board blue tooth chips can become attack vectors on the system.
There was a good keynote on the topic 5 years ago By Timothy Roscoe
Does anyone know if Ben Jordan's AI generated music algorithm would work here to spot fake voices? He's looking for compression artifacts introduced from the training data in the output music. I'd expect similar compression was used for training general voice data. Only issue I can think of is the compression of the audio link most of these scams are going to be using might mask the training data compression.
He's an odd guy who likes to investigate some really dodgy stuff. And I suspect many here on HN would appreciate him.
That led to 7-OH showing up in my feed at some point. In a nutshell, it's the strongest of the active compounds in the kratom plant, that binds to opioid receptors. 7-OH is the marketing name for the alkaloid 7-hydroxymitragynine that is being refined from kratom. It's not clear to me if it's simply extracted, or as some quick research suggests, it might also be a chemical alteration of the more commonly occurring compound in kratom mitragynine.
Anyway, this stuff looks super habit forming, and, of course, the refined version is that much more addictive.
I still remember the days of servers as pets, rather than cattle, and I was harping about server uptime. A wizened server admin piped in and said he rebooted his servers once a week. Said, if you do it any less frequently, then the odds of catching an error causing change while the person who made said change (possibly himself) is still around and can remember what they did go down precipitously. So, to avoid headaches and potential downtime when it mattered, he would just take servers out of rotation and reboot them, and make sure they came back online.
You don’t need a deep dive to see supermarket consolidation that keeps happening year after year. When there is less competition to drive down prices, it is very safe to say to assume that consumers will get less and less surplus for any change a grocery makes.
That sort of begs the question about elevators and escalators. I’ve never been charged riding those, and I can’t imagine fares tacked on in Switzerland. Have they been ruled on? An elevator in a public building is very much public transport.
I know it’s stupid, but I’m genuinely curious now.
If you think your data isn’t being hoovered up I’d like to point out that every model is possible due to federal crimes committed to obtain the information they were trained on. Regardless of how much you are paying, your data is worth another petty civil infraction.
> the queue’s maximum size just needs to be greater than the spikes you expect
There is one truth I have come to know, said by someone far wiser tha me: A queue is either empty or full. Which is to say a queue can either handle all the data coming in, or it can’t. When it can’t it will fill to capacity. This is a probabilistic thing, and you can only decide how many nines to plan for. And it’s worse than it looks at first, because queuing theory is very non intuitive with non linearities that make it very hard to reason about wothout having your nose rubbed in it.
So that means, that yes, you can keep doubling the size of your queue. And no, you can’t ever make it big enough to deal with a poisson distribution. And while you’re at it you will likely need to add workers. And you’re still back to capacity planning and deciding how much money to throw at the problem.
What you may be getting at, and what the article sort of failed at, is that queues are still super valuable for smoothing small spikes, or even large predictable ones. But a queue alone, without backpressure, or overflow will likely cause systems to fall over. Sometimes in ways that are hard to recover from, especially if you have some kind of microservices inspired architecture, where one thing going offline causes another queue elsewhere to fill. Or worse, bringing a failed service back online stresses another system causing it to fall offline. (not meant to be a dig on microservices by the way)
They study aspects of this phenomenon in UI design. There is a constant struggle between power users and novices, and a good UI absolutely cannot cater to both. Which means that over the life of a product, its UIs will tend to get more complicated and inscrutable as your user base levels up. Adobe products are often cited as prime examples.
This, of course, makes on boarding and new user acquisition harder, and can severely limit product growth. And this also leaves space for simpler products to come along and cater to the novice market. Or, companies can fight this tendency, and remove features, or make them harder to use, in order to cater to less demanding demographics.
What I'd be curious about is what spotify looks like as their market share levels off. Do they keep catering to the automatic playlist crowd, or does their average user get more sophisticated over time?
Sorry, I was using imprecise (and possibly incorrect language). There are intentional radiators, and unintentional radiators. Using a given chip/SOC/module can greatly reduce the burden of dealing with the FCC for the purposes of intentional radiators. It's why you see products using Espressif's WROVER module have markings similar to
I'm far from an expert, but from what I understand the FCC cares most about consumer electronics is devices that stomp on the spectrum. And so frequency, antenna power, and signal band matter a lot. So you need to make sure that your antenna is only ever emitting in the band it's allowed, and that the total power never exceeds some amount, where the allowed amount is a function of the area under the curve of bandwidth vs antenna strength.
So when I say "there is no way", what I'm referring to, are the functions that configure drivers don't accept out of bounds values. And functions that ultimately drive the antenna can't drive them hard enough to be in violation. The main reason I know any of this, was that I found a function when working on firmware for the ESP32 on a commercial device, and I thought I could set the power to a level that I thought was too high. Well, that's when I learned what the binary blob that Espressif supplies was for. The guardrails are baked into the API for that blob.
So, does that mean you can't go out of your way to subvert those guardrails? No, but you would be incredibly foolish to knowingly create a device that will get the attention of the FCC. Similarly, there's nothing stopping you from building a circuit that amplifies the signal the device sends to the antenna. But when you're potentially talking about fines per event, and fines per device, it's wise to make sure you play nice.
If the wi-fi chip you're using has free firmware, where none of it is obfuscated, it's very likely that the limitations are baked directly into the chip, such that there is no register combination that would allow it to be out of compliance. Also, I'm not sure that all chips have transitive FCC licensing, so it might be wise to look into that before releasing the device commercially.
And keep in mind, I'm not even talking about creating accidental radios from poorly designed analog circuits, or unshielded high frequency digital circuits. That's a whole other can of worms.
I agree there is not much of a clear call to action. As a firmware engineer who has worked with bluetooth amd wifi, this is a key phrase. It’s also a big fantasy. FCC compliance is a big headache, and part of why people buy a given chip is the FCC certification comes with it. For instance, if I throw an ESP32 into a product and use wifi, I don’t need further certification. That can only happen if “there is no way” you can make the radio do what the FCC doesn’t allow. A general stategy for this is for the company to give a binary blob for radio related functions that limits the radio capabilities that you need to link to in your final build.
So that means there is almost zero chance the chip makers will ever publicly move away from binary blobs. At best they might quietly support reverse engineering efforts by open source driver projects.
That said, I would love it if all the chips I worked with had a battle hardened non vendor alternative. One major downside to these binary blobs is that they can be buggy. We were recently able ro rewrite our Bluetooth firmware to use an opensource version which greatly sped up the data throughput since it didn’t have a bug that killed byte transfer. But we don’t use this code lightly. FCC violations are crazy expensive and not something you take lightly.
I was thinking the same thing, though I couldn't remember the timeline. Makes me wonder if there was something already in the zeitgeist, or if it was fueled by the obsession with purity in the series. I could totally see Breaking Bad causing chemists to want to up their game, or causing chemists to get clowned for having low purity.
Reading this piece, I'm reminded of a podcast I heard some years ago where they were interviewing an early google marketing employee who was talking about the economics of google search. They said they'd done some surveys and concluded that they determined that the average user would get something like $20/year of value, and so that was the most they could realistically charge for search. Meanwhile, they could make something like $500/user in Q4 alone for advertising. So, of course, advertising.
I just don't think that LLM business models can survive the allure of advertising dollars, any more than Search could, or TV, or Radio, or Movies. Ignoring the talk of copilot putting ads into pull requests, there is just no way that publicly hosted LLMs will not end up inserting ads into the output.
Not sure what algorithm Deezer is using, but Benn Jordan is a fairly tech savvy musician who talks about ways to id AI generated music by looking for compression artifacts used by the training data.
It’s worse than that. Eventually everybody calls into code that hits hardware. That is the level that the compiler (ironically?) can no longer make guarantees. Registers change outside the scope of the currently running program all the time. Reading a register can cause other registers on a chip to change. Random chips with access to a shared memory bus can modify the memory that the comipler deduced was static. There be dragons everywhere at the hardware layer and no compiler can ever reason correctly about all of them, because, guess what, rev2 of the hardware could swap a footprint compatible chip clone that has undocumented behavior that. So even if you gave all you board information to the compiler, the program could only be verifiably correct for one potential state of one potential hardware rev.
I’m strongly reminded of early google every time I use AI for research. I used to be able to know little about a topic, try to search on it and get shit results. But, google would give me pages of results. So I could skim a lot and eventually on page 10, I stumble across some term of art, and that term would greatly improve my search. Rinse and repeat, and I’d have a good sense about the topic I was interested in.
You can’t really do that with google anymore, and I can’t remember the last time I bothered to actually learn something that wasn’t trivial from google. ChatGPT, however, has been a game changer. I can ask a really dumb question and get some basic info about the thing I’m asking about, and while it’s often not quite what I’m looking for, it gives me clues to follow, and I can quickly zero in on what I’m looking for, often in new contexts.
As an autodidact who’s main motivation to go to college was to get access to the stacks and direct internet access, I can’t even begin to tell you how game changing LLMs seem to be for learning.
To your point though, my concern is we don’t know how to teach how to learn, and LLMs will likely seduce many into bad behavior and poor research hygiene. I treat my research the same way I attack the stacks, but take someone who’s never been to a research library and ask them to create a report on some topic, and just why? That is the basic resistance, why?, why do what an LLM is almost literally built to do. Yet that is also highly related to individual learning, to take a bunch of disperate sources and synthesize output related to the input.
I suspect we’ll learn how to use LLMs in the same way we learned how to use calculators. But I have no doubt that on average (or maybe median or mode?) calculators have made us less capable to do basic arithmetic, and I suspect LLMs will also cause a great percentage of the population to be worse at sythesizing information. I’d hope that it’s only the same people who would have otherwise only gotten their information from TV, but I do have a slight fear it will creep past that subsection of the population.
You joke, but this is the very problem I always run into vibe coding anything more complex than basically mashing multiple example tutorials together. I always try to shorthand things, and end up going around in circles until I specify what I want very cleanly, in basically what amounts to psuedocode. Which means I've basically written what I want in python.
This can still be a really big win, because of other things that tend to be boiler around the core logic, but it's certainly not the panacea that everyone who is largely incapable of being precise with language thinks it is.
This is generally true. What I find fascinating is the application of information that is not present in the data but that can be confirmed by the data. I’m thinking specifically of deblurring algorithms people use to recover text. If you can guess what font is being used in a screengrab with, say, someone’s bank account blurred out, you can take the i formation contained in the downloaded font and try combinations until you match the blur algorithm that may also be information not contained in the image.
What’s also crazy, is that even if all you do is black out the text, kerning alone can still be used to guess the correct text with some degree of accuracy. I worked on redaction software at one point, and these were very much the types of things we thought about when dealing with text.