This article is missing (extremely important) context that CS enrollment at Berkeley was restricted significantly. Many students at Berkeley want to major in CS, but can’t.
The art’s aesthetic, which resembles Calvin and Hobbes, is disrespectful to its creator, Bill Watterson’s.
Bill spent a lot of energy fighting commercialization of his work, arguing that it would devalue his characters and their personalities. I don’t know what is cheaper than using an AI model to instantly generate similar art, for free.
There are at least two possible conclusions that you could draw. One conclusion is that we all need vitamin D supplementation regardless of how much sun exposure we receive.
Another conclusion is that we might want to reevaluate what we consider the normal range to be, especially when we are deciding a range for a specific individual.
Basically, a proxy. You don't need to deep copy; you just need to return a proxy object that falls back to the original dict if the key you are requesting has not been found or modified.
Functional data structures essentially create a proxy on every write. This can be inefficient if you make writes in batches, and you only need immutability between batches.
The sun’s radiation hitting earth is 44,000 terawatts. I think we’re fine with an “extra” terawatt. (It’s not even extra, because it would be derived from the sun’s existing energy.)
> I would like to learn category theory properly one day, at least to that kind of "advance undergraduate" level she mentions.
As someone who tried to learn category theory, and then did a mathematics degree, I think anyone who wants to properly learn category theory would benefit greatly from learning the surrounding mathematics first. The nontrivial examples in category theory come from group theory, ring theory, linear algebra, algebraic topology, etc.
For example, Set/Group/Ring have initial and final objects, but Field does not. Why? Really understanding requires at least some knowledge of ring/field theory.
What is an example of a nontrivial functor? The fundamental group is one. But appreciating the fundamental group requires ~3 semesters of math (analysis, topology, group theory, algebraic topology).
Why are opposite categories useful? They can greatly simplify arguments. For example, in linear algebra, it is easier to show that the row rank and column rank of a matrix are equal by showing that the dual/transpose operator is a functor from the opposite category.
It is not critical for register assignment -- in fact, SSA makes register assignment more difficult (see the swap problem; the lost copy problem).
Lifetime analysis is important for register assignment, and SSA can make lifetime analysis easier, but plenty of non-SSA compilers (lower-tier JIT compilers often do not use SSA because SSA is heavyweight) are able to register allocate just fine without it.
Here's a concise explanation of SSA. Regular (imperative) code is hard to optimize because in general statements are not pure -- if a statement has side effects, then it might not preserve the behavior to optimize that statement by, for example:
1. Removing that statement (dead code elimination)
2. Deduplicating that statement (available expressions)
3. Reordering that statement with other statements (hoisting; loop-invariant code motion)
4. Duplicating that statement (can be useful to enable other optimizations)
All of the above optimizations are very important in compilers, and they are much, much easier to implement if you don't have to worry about preserving side effects while manipulating the program.
So the point of SSA is to translate a program into an equivalent program whose statements have as few side effects as possible. The result is often something that looks like a functional program. (See: https://www.cs.princeton.edu/~appel/papers/ssafun.pdf, which is famous in the compilers community.) In fact, if you view basic blocks themselves as a function, phi nodes "declare" the arguments of the basic block, and branches correspond to tailcalling the next basic block with corresponding values. This has motivated basic block arguments in MLIR.
The "combinatorial circuit" metaphor is slightly wrong, because most SSA implementations do need to consider state for loads and stores into arbitrary memory, or arbitrary function calls. Also, it's not easy to model a loop of arbitrary length as a (finite) combinatorial circuit. Given that the author works at an AI accelerator company, I can see why he leaned towards that metaphor, though.
WCOJs guarantee an (asymptotic) upper bound on join complexity, but often with the right join plan and appropriate cardinality estimations, you can do much better than WCOJ.
The runtime of WCOJs algorithms are even more dependent on good cardinality estimation. For instance, in VAAT, the main difficulty to find an appropriate variable ordering, which relies on knowledge about cardinalities conditioned on particular variables having particular values. If you have the wrong ordering, you still achieve worst case optimal, but you could have done far better in some cases with other algorithms (e.g. Yannakakis algorithm for acyclic queries). And as far as I know, many DBMSes do not keep track of this type of conditional cardinality, so it is unlikely that existing WCOJ will be faster in practice.
> You can end up writing nearly the exact same code twice because one needs to be async to handle an async function argument, even if the real functionality of the wrapper isn't async.
Sorry for the possibly naive question. If I need to call a synchronous function from an async function, why can't I just call await on the async argument?
def foo(bar: str, baz: int):
# some synchronous work
pass
async def other(bar: Awaitable[str]):
foo(await bar, 0)
There are many comments that don't see the point of the article. For example, why not just use the tools the operating system provides for sandboxing? This article seems to be directed at people familiar with the state of programming languages research (SIGPLAN is the Special Interest Group on Programming LANguages) and so I think it's understandable that it seems vague if one is missing the broader context. However, for people familiar with the state of the field, the main idea is fairly clear.
An operating system (or sandbox, or whatever) is a very large virtual machine, where the "instructions" are the normal CPU instructions plus the set of syscalls. Unfortunately, operating systems today are complicated, hard to understand, and (relatively) hard to modify. For example, there are many different ways to sandbox file system access (chmod, containers, chroot, sandbox-exec on macOS etc.) and they each have bugs that have turned into "features" or subtle semantics. Plus, they are not available on all operating systems or even on all distributions of the same operating system. And then -- how do filesystem permissions and network permissions interact? Even of both of their semantics are "safe," is the composition of the two safe?
The assumption is: because operating systems are so complex, large, and underspecified, it probably is dangerous for LLMs to interact directly with the underlying operating system. We have observed this empirically: through CVEs in C and C++ code, we know that subtle errors or small differences in semantics can cascade into huge security vulnerabilities.
To address this, the authors propose that LLMs instead interact with a virtual machine where, for example, the semantics of permissions and/or capabilities is well-defined and standardized across different implementations or operating systems. (This is why they mention Java as an analogy -- the JVM gave developers the ability to write code for a vast array of architectures and operating systems without having to think about the underlying implementations.) This standardization makes it easier to understand how exactly an LLM would be allowed to interact with the outside world.
Besides semantic understanding and clarity, there are more benefits to designing a new virtual machine.
- Standardization across multiple model providers (mentioned).
- Better RLHF / constrained generation opportunity than general Bash output.
- Can incorporate advances in programming language theory and design.
For an example of the last point, in recent years, there has been a ton of research on information flow for security and privacy (mentioned in the article). In a programming language that is aware of information flow, I can mark my bank account password as "secret" and the input to all HTTP calls as "public." The type system or some other static analysis can verify that my password cannot possibly affect the input to any HTTP call. This is harder than you think because it depends on control flow! For example, the following program indirectly exfiltrates information about my password:
if (password.startsWith("hackernews")) {
fetch("https://example.com/a");
} else {
fetch("https://example.com/b");
}
Obviously, nobody would write that code, but people do write similar code with bugs in e.g. timing attacks.
Sorry, you're right that the chart on the home page does not have human performance. The leaderboard chart does: https://arcprize.org/leaderboard. And the leaderboard by default shows scores for ARC-AGI 1 and 2. The models are much worse at 2 than 1; the best performing model scores around 15% (Grok 4, thinking), while humans are at ~100%.