With purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K.
There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically.
The topic of LLM jailbreaking is an important one. As machines increase in power their ability to benefit society increases, as does their ability to harm society.
This paper intends to address the question: "Is it possible to create a perfect universal guardian against LLM jailbreaks?". This means:
Universal: applies to all prompts and input combinations
Perfect: is able to successfully distinguish harmful versus not harmful
Computable: is feasible to compute
We show that this combination does not exist and Request for Comment.
A more feasible approach involves post-hoc filtering of results, which involves an easier problem of decideability. In order to implement such approaches, both reasoning traces an intermediate code changes would likely need to be hidden from the user until the results are determined.
Note: this paper was written in conjunction with ChatGPT 5.5 Pro. The topic of the reducibility of such problems to The Halting Problem has been top of mind for me for some time. In one sense, it is a trivial conclusion. Yet it is one of high importance.
Markdown has emerged as the "lingua franca" of LLMs. It is structured enough to convey meaning, and yet readable everywhere.
However, as document lengths increase:
1. It is wasteful to put the entire context in memory
2. It can reduce accuracy by providing less relevant information
3. Increase the possibility of errors during edits
Inspired by Cloudflare's Markdown Content Type and Code Mode, we propose a lightweight document object model for Markdown to:
1. Reduce costs
2. Increase the accuracy of locating key sections
3. Increase speed
4. Increase the accuracy of editing documents
On timing – I just had to start planning a bit better. In worst case I'd start the meeting without video and turn on a minute or two in. Well worth it for me.
I use Camo (lifetime $79 license) and an iPhone SE I picked up on eBay for $200.
I've had dozens of people comment how clear my camera is. Many now purchased a similar setup. I used OBS for a long time since it was free, but the quality wasn't close.
If you're in a line of work that requires you to stream it's a worthwhile investment by a great company that just keeps getting better.
You're absolutely right regarding from market highs.
To do this correct you'd need purchases and sales versus assets.
All that said, it's why I put a rough estimate of 30% (instead of 60%) and hadn't included any losses in broader commercial paper (which has dropped). So the overall estimate is defensible.
Also agree on Basel 2-like risk. The primary issue with Tether is the marketing around stability and the transparency. If it was treated like any other "money market" with better NAV reporting, position reporting and rules around withdrawals then we'd mitigate some of the risks.
Investments were this way in the 1800s and early 1900s.
A lot of protections have been put into place since then. There's a reason we prosecute Ponzi schemes, offer FDIC and SIPC insurance, stress test banks, etc.
It's fantastic to see great research combined with great tutorials – bravo. You all have bitten off an incredibly hard problem with pragmatism and tenacity. Keep up the great work.
To the authors: did you any of your own recordings? I've used my own and clips online, in WAV and other formats, at various sampling rates.
All of the results come back gibberish. The results in the training data seem just fine. Curious if you've tested the above to ensure it didn't overfit.
As I understand they do have a working prototype. I believe their chip is 3mm and works in pockets, etc. There's been so much hype, so little delivered in this area that I'm finding it hard to sift the wheat from the chaff.
Via the Lodash CLI you can "lodash underscore". This results in 6.7kb gzipped. In my experience, jdalton is a very solid coder and the lodash package is excellently maintained.
There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically.
This becomes an optimization problem for token t:
min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + token cost[t]