We work with smaller data centers that have single node GPU capacity. So while there's a shortage for clouds that have capacity for clusters, we're focusing on aggregating single nodes behind a unified API. It works for our users too-- a single A100 is more than enough to fine-tune llama2 or Mistral7b on your own data.
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.
Hey! Co-founder of Brev.dev here and was in that video. We're actually a small team of 3-- makes it easy to maneuver when marketing and product are the same :)
We were building cloud dev environments in 2021, which I now feel strongly do not have PMF. Writing a fun post on that...
We were trying to make coding in the cloud as seamless as coding locally. Users started using that for GPUs ~1 year ago. A solid dev env experience makes sense for AI/ML developers. The infrastructure problems are more complicated & more expensive.
We've since reworked our product and aim to build AWS SageMaker but without the confines of any particular cloud. We're starting by sourcing cheap GPUs & having 1st class notebook experience for fine-tuning and training. Inference coming soon!
Hey, you could use a template on brev.dev to spin up a gpu box with the model and Jupyter notebook. Alternatively, the falcon 7b model should be small enough for colab
Finetuning a smaller model leading to better performance seems like a significant finding that'll lead to a lot of companies fine-tuning their own internal "ChatGPT"s
That makes sense, Brev.dev is a really simple way to run your code on a configured GPU without having to change your code. It'll also optimize your GPU to save money when possible.
Hey, founder of Brev.dev here. Brev lets you suspend the instances when not in use, and also auto-stops it after 3 hours of inactivity to avoid expensive surprises. Would love for you to give it a shot
That would be a great follow on. I posted the repo if you want to leave any feedback or help me continue building it out: https://github.com/brevdev/dev-browser
100%, also just looking up dev docs for frameworks. Something like "write a post API in Golang's Gin framework" instead of looking up what that syntax was again
Totally! I found ChatGPT to be more helpful in this use case since it outputted full code snippets instead of generating it line by line. The context was also extremely valuable in making iterations (like "make it work for mac")
This is a very clever approach. After reading through the comments, I really appreciate your focus on ecosystem impact. This seems to be one of the most direct approaches to solving the climate crisis. I'm rootin for y'all!
We felt the same way. This Paul Graham quote is funny “If Zoom were the default and someone just invented face to face, it would seem like an amazing step forward.”
We’re growing our team and are in person in SF if you’re interested: nader at brev.dev
This is a huge pain, mostly stemming from using the same computer for different projects, because dependencies start to conflict. Big companies (fb, google, workday) solve this with cloud-hosted dev environments. Cloud environments let me contribute to our code base as a technical founder without having to have the technical expertise of a CTO. I'd rather not go back because while I know how to code, I'd rather not spend time configuring dev environments if someone can just share theirs with me. (disclaimer: I work at https://brev.dev)
If you've scratched your own itch, find more people with your own itch. Try to onboard every user: this let's you show them how to use it, and you'll learn more watching them use it than any second-degree data you pick up.
Also, as you can imagine, the smaller data centers have a pretty rough experience for provisioning. Our API handles difficulties from those smaller data centers to make the experience consistent with provisioning from a major cloud service provider.