Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel <> Libre but still not "cheap and easy".
Same, it's in a similar vein as pgvector - probably not as performant as turbopuffer or chroma but you get the benefits of being in a really nice ecosystem.
Thought it was interesting to read about how farmers had to reskill as farming subsided as a common profession. In the context of the impact of AI on SWE.
Will this continue to be true? I do agree with the principle. But I've sometimes had the feeling that poor design upfront can have compounding consequences, especially when AI is filling in ambiguities.
A system that does the following given a task_description:
while LLM("is <task_description> not done?"):
Browser.run(LLM("what should the browser do next given <Browser.get_state()>"))
This simple loop turns out to be very powerful, achieving the highest performance on some of OpenAI's latest benchmarks. But it's also heavily unoptimized compared to a system that is just LLM("<task_description>") for which we already have things like vllm. BLAST is a first step towards optimizing this while loop.
The use cases are (1) integrating AI automation into my app (2) automating workflows inside web browser (3) personal use. The value is in optimizing for low latency under user-defined constraints such as LLM cost budget or maximum browser memory usage.
Do you build agents that interface with web browsers? BLAST is sort of like vllm for browser+LLM. The motivation for this is that browser+LLM is slow and we can do a lot of optimization with an engine that manages browser+LLM together - e.g. prefix caching, auto-parallelism, data parallelism, request hedging, scheduling policy, and more coming soon.
Now the API is what may be throwing folks off. Right now it's an OpenAI-compatible API. We will implement MCP. But really the core thing is abstracting away optimizations required to efficiently run browser+LLM.
I would really think about it as a serving engine like vllm but for browsers+LLMs. It handles caching, parallelism, scheduling, budget constraints for LLM cost and browser memory usage. Yes it currently has an OpenAI-compatible API but we will also implement MCP. (though we're working on something that will be way better than "MCP for web browsers")