Peter Fenton invested in the company when it was called Infra.App originally (I think it's still online: https://infra.app/). It was an access management product, then it became desktop Kubernetes product, both they pivoted out of. I imagine he invested based on the technical acumen of the founders and then worked with them to pivot when Ollama started working out. Almost all of the other investors invested in the founders based on that original idea space (which wasn't a great one but was quite adjacent.) I think that includes 8VC, YC, Pace Capital, GTMFund, who all followed on here.
Since it pivoted, I imagine the story they are telling is the usage. But you can add to that Peter Fenton has had incredible success in commercial open source (Docker-ish, Elastic, JBoss, CockroachDB, TimeScaleDB, and if you include Benchmark there is Elastic, Confluent, etc.) so a bunch of these people are probably trying to ride the Peter Fenton/Benchmark train. My guess is that's what Theory is doing.
Investors are not free but they are very hype driven. There are < 10 investors who are truly experts in open source in my opinion in the market (and maybe < 5.) It's a really wacky cadence and I would argue there are different types of OSS businesses, which makes it even more difficult to be an expert. It's something I struggle with personally investing in OSS -- I think I understand the likely motion for an OSS startup and then > 50% of the time I am wrong.
The story of Reflection AI is supposedly that the company was faffing and failing at winning in the coding agent space, but was introduced to Jenson, who suggested they build an open-weight model and said he would fund it. That turned into a $2 billion financing with NVIDIA doing roughly $500 million and was a complete pivot.
I think the bet would have to be that a US Open Weight company either: 1. Gets a lot of money from Jenson who views them as a counterbalance to the big labs in his ecosystem and a way to generate leverage (the same way he is positioning neoclouds-- it also could be synergistic with neoclouds who could offer the model serving endpoints) 2. Can fast follow the same way Mistral does (which, honestly, seems like just distilling the Chinese model, which distills the US lab but is pretty innovative on a whole lot of architecture both in training and serving land.) 3. AND figure out some (maybe not super lucrative but lucrative enough) sort of business model, as well. There are lots of possible business models, so I will be curious how this whole space evolves.
If they have a really seamless fine-tuning experience and maybe can help you extract the data you need to FT (which is one of the big challenges in actually getting fine-tuning democratized), maybe you would use it because "Tinker" defaults to it.
The model could also be more flexible for non-coding use-cases (they show the results for reasoning being strong) so maybe the argument is to use it for non-coding use-cases to drive relatively deterministic conclusions for non-coding agents (they have also done some determinism work on kernels, which could be useful in pulling on that thread of deterministic models that are fine-tuned for everything that is not writing code.)
That said, I'm not sure how much all the work they have done actually synergizes or if the market size (at least in the short to medium term) is big enough for a huge outcome from the company's current valuation with those bets as the enterprise agent estate is taking a while to evolve. Hence companies like Anthropic and OpenAI are throwing tons of consulting money at the problem.
It could be but there are a host of companies going after open weights models: Arcee, Reflection, Llama (TBD on Meta's focus on closed-source versus open-source), etc.
That said, the fine-tuning API + open weight model at least is a semblance of a viable business that could work so I will be curious about it. I'm not sure the synergy is fully there (why is someone with an open weights model privelaged to fine-tune it better if it's just QLora or Lora) but let's see!
I'm writing a short series of tutorials on FlashAttention: from theory to efficient CUDA kernels.
Part 1 is the theoretical foundation. It walks through a modern algebraic formalism showing that FlashAttention is an associative operation, which lets you treat it as a regular reduction on the GPU and apply all the same scheduling optimizations. Some recent MLSys and CVPR (ELSA) papers lean on this framing, and I find it much more powerful than the original.
This framework is particularly useful in ML compilers, where you should implement general optimizations applicable to many operations rather than writing specialized kernels. This article shows that attention actually belongs to a large family of "secretly-associative" operations, walks through a handful of examples, and links a few concepts from abstract algebra that let you identify whether an operation is secretly associative.
Overview:
- Safe softmax, Welford's variance, and FlashAttention are the same secretly-associative operation
- The twisted monoid (transport of structure), why the max-rescale coupling doesn't break associativity
- The qk_scale = log2(e)/√D like in FA-2 derived from scratch
- Numerical analysis: overflow bounds, error limits, and why tiling never amplifies error
- Bird's 3rd Homomorphism Theorem as a test for whether any loop is secretly associativ
Adding to the prior comments as my intuition matched yours, there’s a nice Reddit thread that gives some context into how it can be faster even if you require exact matches: https://www.reddit.com/r/LocalLLaMA/s/ARxHLqRjdM
The TLDR/key (from my understanding) is that verifying N tokens can be faster than generating N tokens.
Congratulations to their team. SQLGlot is a really powerful tool that a lot of companies use so a huge contribution to the OSS community so hopefully it continues to be supported and gets better and better!
Feels like two types of companies raised money: - Companies trying to couple the cloud with a programming language. - More recently, companies trying to couple GPUs with a programming language/alternative to CUDA.
If you are running things locally (I would think especially on the edge, whether on not the LLM is local or in the cloud) this would matter. Or if you are running some sort of agent orchestration where the output of LLMs is streaming it could possibly matter?
Today there are so many other solutions: Stytch, Descope, PropelAuth (For B2B companies), and others.
VCs went a bit ham on this category when Auth0 got bought. I sense that the general thought process was: Auth0 multi-billion dollar company -> Auth0 will become crappy under Okta > replace Auth0 and build a bit company.
Basically people are constantly calculating metrics based on existing tables. Think something as simple as a moving average or the sum of two separate columns in a table. Once upon a time you would set up a cronjob and populate these every day as a SQL query in some python or Perl script.
Dbt introduced a language for managing these “metrics” at scale including the ability to use variables and more complex templates (Jinja.)
More broadly dbt did two other things: 1. It pushed the paradigm from ETL to ELT (so stick all the data in your warehouse and then transform it rather than transform it at extraction time.) 2. It created the concept of an “analytics engineer” (previously know as guy who knows SQL or business analyst.)
What’s interesting is how much it contrasts with TechCrunch’s story: ‘Most of Command AI’s 30-person, San Francisco-based team will be joining Amplitude. Command AI’s co-founder and CEO James Evans wouldn’t reveal the terms of the deal, but said candidly that an acquisition wasn’t something he’d been planning on.
“Our growth was great and we had plenty of runway,” Evans told TechCrunch. “We weren’t out shopping ourselves or anything. But when Amplitude reached out a little while ago — this summer — we got really excited about the combination and became convinced that we could grow faster and reach more users together.”’
Interestingly, according to Axios, the price was pretty limited: "Amplitude (Nasdaq: AMPL) acquired CommandAI, an SF-based software user experience startup, for $20m (net of cash). CommandAI (fka CommandBar) had raised around $23m from Insight Partners, Itai Tsiddon, Thrive Capital, and BoxGroup."
I would be curious to learn more about the rationale to sell the business as I understand the strategic value to Amplitude. Interestingly, these next-generation digital adoption platforms have generally been pretty challenged.
Been watching this episode unfold on Twitter and has read about Matt’s domain hijacking of thesis, etc.
It seems to me like Matt is the type of person who likes to hide behind character and other ad hominem attacks rather than addressing actual issues at hand. Perhaps this is because of a psychological issue but I can’t really know. Normally I would think the community would be repulsed by this and would find an alternative.
What is remarkable is that there has not been enough community animus to fork relative to Terraform and OpenTofu. It shows the power of the underlying GPL license and “plugin” approach. Something to think about for other companies that may be thinking about relicensing versus just building a durable ecosystem around their proprietary brand and assets.
This whole thing comes off as tone-deaf and deceptive even to me (who is all for COSS monetizing.) Warpstream was sponsoring Benthos, it sounds like they didn't get a great heads up of this happening, which makes the project owner sound self-serving. Then you renamed the repo and relicensed some connectors all in one go without giving anyone from the community a chance to opine or think about how this affects them.
Finally, Redpanda did some partnerships with vendors nobody cares about whose businesses are at risk to show how you are opening up the ecosystem.
It actually comes off as somewhat malicious and Ashley's note where he notes he didn't read the article also comes off as not caring about developers (even insofar as he has facts wrong -- if the plug APIs remain compatible this creates more choice for users.)