There's a bunch of other fully open models, including the [Marin](https://marin.community/) series of models out of Stanford and Nvidia regularly releases fully open models.
MoEs have a lot of technical complexity and aren't well supported in the open source world. We plan to release a MoE soon(ish).
I do think that MoEs are clearly the future. I think we will release more MoEs moving forward once we have the tech in place to do so efficiently. For all use cases except local usage, I think that MoEs are clearly superior to dense models.
(I’m a researcher on the post-training team at Ai2.)
7B models are mostly useful for local use on consumer GPUs. 32B could be used for a lot of applications. There’s a lot of companies using fine tuned Qwen 3 models that might want to switch to Olmo now that we have released a 32B base model.
This is why I will never work somewhere with a short post termination exercise period (PTEP). If it’s not at least 5 years, ideally 10, they don’t seriously consider equity something that employees are owed.
Can you explain? In most cases, preferences won’t come into play, assuming you raise at a standard 1x preference and sell for more than you have raised. In that case, owning 0.5% should roughly translate into $5M (modulo dilution).
I hate how every company that I place an order with treats that as permission to send a constant drip of marketing emails. I send them straight to spam.
The rule of thumb is roughly 44gb, as most models are trained in bf16, and require 16 bits per parameter, so 2 bytes. You need a bit more for activations, so maybe 50GB?
you need enough RAM and HBM (GPU RAM) so it’s a constraint on both.
These prices are insane- you can do much, much better than those prices if you negotiate. If you're paying $7.20 per GPU-hour for an H100, you should find another supplier.
There's a bunch of other fully open models, including the [Marin](https://marin.community/) series of models out of Stanford and Nvidia regularly releases fully open models.