For some definitions of better, yes. Chinese is more token efficient for representing fixed text, for example, although this does not always lead to better performance on downstream tasks.
As the author - this was adapted from a thread posted on X in March (linked in article). AI did the adaptation, I wrote the original article. It seems like it inserted grammatically correct hyphens, otherwise the copy is mine.
Hello, I am the author - this is not an LLM-generated article, I wrote this by hand and had an LLM adapt it from a thread on X. You can see the original thread here: https://x.com/mathemagic1an/status/2035850046735098065
> the fact that language models have human-interpretable representations and neurons has been known since BERT... Circuits research also does not come from Anthropic...
The article does not claim Anthropic invented the field, rather that they have had important contributions to it. This is intended as an overview into a specific set of ideas that are working for mechanistic interpretability. Not a formal literature review.
Would love to understand how you compare to other providers like Modal, Daytona, Blaxel, E2B and Vercel. I think most other agent builders will have the same question. Can you provide a feature/performance comparison matrix to make this easier?
> static analysis tools that produce flowcharts and diagrams like this have existed since antiquity, and I'm not seeing any new real innovation other than "letting the LLM produce it".
Inherent limitation of static analysis-only visualization tools is lack of flexibility/judgement on what should and should not be surfaced in the final visualization.
The produced visualizations look like machine code themselves. Advantage of having LLMs produce code visualizations is the judgement/common sense on the resolution things should be presented at, so they are intuitive and useful.
Vector embedding is not an invention of the last decade. Featurization in ML goes back to the 60s - even deep learning-based featurization is decades old at a minimum. Like everything else in ML this became much more useful with data and compute scale
Gary Marcus has been taking victory laps on this since mid-2023, nothing to see here. Patently obvious to all that there will be additional innovations on top of LLMs such as test-time compute, which nonetheless are structured around LLMs and complementary
Very cool and interesting project. Ideas like this are a threat to traditionally-conceived project management platforms like Linear; that being said, Linear and others (Monday, ClickUp, etc.) are pushing aggressively into UX built for human/AI collaboration. I guess the question is how quickly they can execute and how many novel features are required to properly bring AI into the human project workspace
This does not take into account the fact that experienced developers working with AI have shifted into roles of management and triage, working on several tasks simultaneously.
Would be interesting (and in fact necessary to derive conclusions from this study) to see aggregate number of tasks completed per developer with AI augmentation. That is, if time per task has gone up by 20% but we clear 2x as many tasks, that is a pretty important caveat to the results published here
Used in multiple similar publications, including "Guiding Language Models of Code with Global Context using Monitors" (https://arxiv.org/abs/2306.10763), which uses static analysis beyond the type system to filter out e.g. invalid variable names, invalid control flow etc.
If you are looking for an alternative that can also chat with you in Slack, create PRs, edit/create/search tickets and Linear, search the web and more, check out codegen.com
A friend asked me to do diligence on this company circa 2021 given my personal background in ML. The founder was adamant they had a "100% checkout success rate" based on AI, which was clearly false. He also had 2 other startups he was running concurrently (?)
This is true and is essentially a form of arbitrage. Anthropic is eating the cost of your elevated queries with their $20 flat fee subscription.
The "famously huge API token costs" you are referring to is Cline passing the Anthropic API cost through to you with no markup. You even input your own API token.
Codemodder has extensive Java support, which Codegen does not support at the moment. Otherwise, my understanding of Codemodder is that it is focused on AST-level syntactical modifications. Codegen computes a richer graph datastructure, and this can be used for sophisticated modifications that depend on inheritance hierarchies, function usages, cross-file references and more.
Codemodder is written in Java, whereas you can write Codegen in a jupyter notebook or anywhere you can run Python.