The pattern I've noticed with a lot of open source LLMs is that they generally tend to underperform the level that their benchmarks say they should be at.
I haven't tried this model yet and am not in a position to for a couple days, and am wondering if anyone feels that with these.
They're not that good at arb. Real arb doesn't even really exist anymore. Even when it does, it's not JS that closes it. They market make, which is different.
I wonder if Sam did something in the name of his own philosophy, but was financially suicide. Like vastly underestimating the costs of training/inferencing to the board, but justifying it to himself because it's all going towards building AGI and that's what matters.
But them firing him also means that OpenAI's heavy hitters weren't that devoted to him either. Obviously otherwise they would all leave after him. Probably internal conflict, maybe between Ilya and Sam, with everyone else predictably being on Ilya's side.
It's true at last anecdotally. Related, there's a running joke in some subset of the industry about the types of exotic derivatives that are so complex and esoteric "only french banks" trade them.
> He was so good at finance that at his previous job at a hedge fund, he supervised its biggest day of losses in company history.
Small irrelevant detail, but nags me nonetheless. Jane Street is not a hedge fund. They don't remotely operate like one. They're a prop market making firm.
Honestly an idea I've had with the recent surge of interest in LLMs is some type of robotic pet. I feel like LLMs can understand human language and images(hopefully soon) to the extent that a pet should be able to. Some kind of robotic dog that can not only express emotions, but understand what you say to it, seems like it would be pretty successful if done correctly.
The nice thing is that it doesn't even have to understand perfectly, a GPT-4 level of vision/language understanding would be perfectly fine. It doesn't have to hold a conversation, just be able to be happy or sad when it should.
Most derivatives traders I know in the industry do some version of buy-and-hold for their personal portfolios, but one of the best I know does something completely different. He sticks to a philosophy of scanning multiple "small" cap companies(<50-100mm mktcap) until he finds one he generally likes, then figures out absolutely everything he can about them. Every piece of information available, down to calling whoever he can in management. Then once he decides he likes it, he commits 20-30% of his portfolio into them, often becoming a small, but notable investor in the company itself.
He's made massive amounts of money from this. He admits that it's basically a second job in terms of time and effort spent, but believes that it's replicable because no institutional investor is actually looking at these stocks, leading to hypothetical mispricings.
I think the next big innovation in LLMs (sort of like the attention mechanism) will be some method of distributing work to much smaller, specialized, and capable units, rather than having one giant network.
We already see hints of this with MoE, but something entirely new wouldn't surprise me.
When Meta released a highly censored LLaMa, I think it was pretty clear that over time, the market would tighten up and release progressively more powerful uncensored models until demand was met. I'm glad to see this happen in real time.
I think(hope?) Llama 3 will be a MoE architecture that shows >GPT-3.5 level performance. Interesting to think Meta will probably continue to spearhead the open source AI movement.
> The new voice capability is powered by a new text-to-speech model, capable of generating human-like audio from just text and a few seconds of sample speech.
I'm more interested in this. I wonder how it performs compared to other competitor models or even open source ones?
Honestly, the current general state of software may be a good proxy for what AI will settle to. I can imagine major open-source models, trained and generated by nonprofit efforts in a roughly similar fashion to Linux. Entire businesses might be built on top of "servicing" this model, such as enterprise-grade finetuning, serving, etc. Like the author mentioned, no business wants their core functionality to be dependent on external factors. As I understand it, this is also the case for some Linux-based corporations that focus on building a business around open source software. Of course, there will be proprietary models. Will the average home user cook their own custom distro of LLaMA 10 to be their home assistant? No. They'll probably use Alexa or whatever proprietary solution is out there.
Uncertain, but I'd be willing to be that open source AI will win the way Linux won. In the ways that it matters.
Most people confuse market making/risk holding with high frequency statistical arbitrage strategies. I'm not totally sure exactly what Knight Capital was running, but generally the only "little" mistakes that would cause HFT market takers such as Jump(for the most part) would blow up is some type of egregious technical error like this, or some type of assumption violations outside of market conditions(legal, structural, etc.). Compare this to market makers like Jane Street who hold market risk in exchange for EV, and thus could lose money just based off of market swings (not to blowup levels if they know what they're doing), and you can see the difference between the styles.
I'm a proponent of both. But generally I hold more respect for actual market makers who hold positions and can warehouse risk.
Jane Street's version of this was absolutely intentional.
[1] https://www.reuters.com/article/business/high-frequency-trad...