I'm surprised the author didn't mention the other argument for paraxanthine being less addictive than caffeine: it has a less acute/peaky curve of adenosine blocking.
I'm not a biologist, but I'm under the impression that your body uses the heuristic of "the more acutely a neurotransmitter is suddenly flooded into our system, the more of a homeostatic counter-response we're going to launch in the form of things like dopamine downregulation (etc, depending upon what neurutransmitter we're talking about)".
I'm not entirely sure this is true, but it seems to be corroborated by other researchers (e.g. Anna Lembke in her book Dopamine Nation, which isn't about caffeine though).
This is why substances like theacrine claim to offer even less tolerance than paraxanthine: it has a super gradual adenosine-blocking curve, with super long half-life (like 12-16 hours, IIRC). So when you take one theacrine, you won't notice it for hours, but its effects will last longer than one day (though I forget what its interaction with sleep is supposed to be?).
This is so cool. Real-time accent feedback is something language learners have never had throughout all of human history, until now.
Along similar lines, it would be useful to map a speaker's vowels in vowel-space (and likewise for consonants?) to compare native to non-native speakers.
I can't wait until something like this is available for Japanese.
I think reading this makes it even clearer that the 70.3% score should just be discarded from the benchmarks. "I got a 7%-8% higher SWE benchmark score by doing a bunch of extra work and sampling a ton of answers" is not something a typical user is going to have already set up when logging onto Claude and asking it a SWE style question.
Personally, it seems like an illegitimate way to juice the numbers to me (though Claude was transparent with what they did so it's all good, and it's not uninteresting to know you can boost your score by 8% with the right tooling).
Yes, however Claude advertised 70.3%[1] on SWE bench verified when using the following scaffolding:
> For Claude 3.7 Sonnet and Claude 3.5 Sonnet (new), we use a much simpler approach with minimal scaffolding, where the model decides which commands to run and files to edit in a single session. Our main “no extended thinking” pass@1 result simply equips the model with the two tools described here—a bash tool, and a file editing tool that operates via string replacements—as well as the “planning tool” mentioned above in our TAU-bench results.
Very impressive! But under arguably the most important benchmark -- SWE-bench verified for real-world coding tasks -- Claude 3.7 still remains the champion.[1]
Incredible how resilient Claude models have been for best-in-coding class.
[1] But by only about 1%, and inclusive of Claude's "custom scaffold" augmentation (which in practice I assume almost no one uses?). The new OpenAI models might still be effectively best in class now (or likely beating Claude with similar augmentation?).
SimulaVR[1][2] is releasing our standalone (belt-strappable) compute packs this year, which will (ii) come pre-installed with our FOSS Linux VR Desktop compositor and (ii) work with AR headsets like the Rokid Max series (and potentially the XReal headsets). So basically: you'll get full Linux Desktop apps in AR (not just Android ones) with actual VR window management (not just "dumb monitor mode").
[1] I know we're taking forever D: But we intend for this to be a way to release an intermediate product (which we've been making anyway for our full headsets).
Did anyone else notice that o3-mini's SWE bench dropped from 61% in the leaked System Card earlier today to 49.3% in this blog post, which puts o3-mini back in line with Claude on real-world coding tasks?
1. Suppose we have a `./nixos_binary_program_with_glibc-newer` compiled on a NixOS machine against bleeding edge `glibc-newer`.
2. `./nixos_binary_program_with_glibc-newer` will have `/nix/store/glibc-newer/linux-ld.so` path hardcoded into its ELF header which will be used when the program launches to find all of the program's shared libraries, and so forth. (And this is a fact that `ldd` will obfuscate!).
3. When `./nixos_binary_program_with_glibc-newer` is distributed to machines which use `glibc-older` instead of `glibc-newer`, the hardcoded `linux-ld.so` from (2) will fail to be found, leading to a launch error.
4. (3) will also happen on machines which don't use nix in the first place.
=======Will's Solution========
1. Use `patchelf` to hardcode a standard FHS `ld-linux.so` location into `nixos_binary_program_with_glibc-newer`'s ELF header (using e.g. `/lib64/ld-linux-x86-64.so.2` as the path)
2. Use a metaloader to launch `nixos_binary_program_with_glibc-newer` with an augmented `RPATH` which has a bunch of different `/nix/store/glibc-newer` paths, so that nix machines can find a suitable `ld-linux.so` to launch the program with.
This will make `nixos_binary_program_with_glibc-newer` work on any machine, including both non-nix machines and nix machines (which might be running older versions of glibc by default)!
I'm not a biologist, but I'm under the impression that your body uses the heuristic of "the more acutely a neurotransmitter is suddenly flooded into our system, the more of a homeostatic counter-response we're going to launch in the form of things like dopamine downregulation (etc, depending upon what neurutransmitter we're talking about)".
I'm not entirely sure this is true, but it seems to be corroborated by other researchers (e.g. Anna Lembke in her book Dopamine Nation, which isn't about caffeine though).
This is why substances like theacrine claim to offer even less tolerance than paraxanthine: it has a super gradual adenosine-blocking curve, with super long half-life (like 12-16 hours, IIRC). So when you take one theacrine, you won't notice it for hours, but its effects will last longer than one day (though I forget what its interaction with sleep is supposed to be?).