LLM App Stack – a.k.a. Emerging Architectures for LLM Applications(github.com)
github.com
LLM App Stack – a.k.a. Emerging Architectures for LLM Applications
https://github.com/a16z-infra/llm-app-stack
5 comments
> endless demand for AI projects
If you need any help, let me know!
I'm a freelancer and have been doing various LLM projects too, using RAGs (Zep, Bedrock KB), promptfoo (prompt testing).
A lot of it is deployed via AWS CDK, using serverless components, such as Step Functions - great for orchestration.
Using TypeScript end to end.
If you need any help, let me know!
I'm a freelancer and have been doing various LLM projects too, using RAGs (Zep, Bedrock KB), promptfoo (prompt testing).
A lot of it is deployed via AWS CDK, using serverless components, such as Step Functions - great for orchestration.
Using TypeScript end to end.
grabbed your email, I'll try and get in touch next week
Could you describe the types of projects you worked on?
I have a close friend who is constantly hustling but has little technical expertise, so he just says "yes" and hopes I'll swoop in and figure out execution.
We built a lead qualification and sales coaching plug-in for a CRM, the thing has a lot of personality. IMO it was a project conceived to automate sales that ended up automating sales coaching.
From there I've done a fair bit of red teaming customer-facing chatbots, then helped save some stuck projects because these approaches to problem solving are so new.
Something that was really satisfying was a project to introduce natural language queries to a ridiculous array of internal corporate databases without sending private data out of the network. That thing is like an extremely boring corporate Jarvis.
I have absolutely zero traditional "AI" skill set, but I guess there's a lot of room downstream for implementation engineering.
We built a lead qualification and sales coaching plug-in for a CRM, the thing has a lot of personality. IMO it was a project conceived to automate sales that ended up automating sales coaching.
From there I've done a fair bit of red teaming customer-facing chatbots, then helped save some stuck projects because these approaches to problem solving are so new.
Something that was really satisfying was a project to introduce natural language queries to a ridiculous array of internal corporate databases without sending private data out of the network. That thing is like an extremely boring corporate Jarvis.
I have absolutely zero traditional "AI" skill set, but I guess there's a lot of room downstream for implementation engineering.
You'd be amazed at the number of YC companies building that boring version of SQL Jarvis!
That's cool, I'd love to see it become ubiquitous.
It's actually a really fun problem so long as you're burning up tokens on someone else's api key
It's actually a really fun problem so long as you're burning up tokens on someone else's api key
Big +1, it seems like half of the startups world is working on this
More like a16z talking their own book. It’s too early to be defining a stack so concretely.
This doesn't mention text-generation-inference, vllm, or Mistral which is raises eyebrows. I would like to see more importance given to privately deployed LLMs that are not consumed through third party APIs
[deleted]
I accidentally stepped into seemingly endless demand for AI projects this year, and it has been a god-send for my crushing burnout. Truly interesting stuff that makes me feel like I felt when I was a kid learning unix.
I can accomplish such objectively cool things that add measurable value with these new tools, but the hype makes it all feel mildly gross.
I am both excited and feel the need to wash my hands.