Our Small ML Team Beat OpenAI and Anthropic in a Specialized Domain [pdf](digits.com)
digits.com
Our Small ML Team Beat OpenAI and Anthropic in a Specialized Domain [pdf]
https://digits.com/_assets/downloads/beyond-the-hype-evaluating-llms-vs-digits-agl.pdf
3 comments
Huge congrats - and when you look at the latency graphs as well it really shows the value of these specialised systems!
[deleted]
As an ML engineer at Digits’ AI team, I spent the last 2 years anxious that our verticalized approach would be obsolete once OpenAI or Anthropic released their next API. How could our specialized accounting system compete when these giants had billions in funding?
But today, our results speak volumes: we’ve built an ML system that outperforms GPT-4o by 54% in transaction categorization, achieving 93.5% accuracy through domain-specific optimization rather than raw compute. After watching our system reach 100% accuracy for some businesses through specialized learning loops, I’m convinced that small teams focusing on specific problems have a future.
The lesson for other ML engineers feeling the existential dread of competition with foundation models: domain expertise and specialization can beat generalized approaches in vertical applications. You don’t need to match their resources—you just need to solve one problem exceptionally well.
But today, our results speak volumes: we’ve built an ML system that outperforms GPT-4o by 54% in transaction categorization, achieving 93.5% accuracy through domain-specific optimization rather than raw compute. After watching our system reach 100% accuracy for some businesses through specialized learning loops, I’m convinced that small teams focusing on specific problems have a future.
The lesson for other ML engineers feeling the existential dread of competition with foundation models: domain expertise and specialization can beat generalized approaches in vertical applications. You don’t need to match their resources—you just need to solve one problem exceptionally well.