I've been neck-deep in AI and LLMs for a while now, and I kept running into the same problem: every learning resource out there for LLMs either tries to teach you all of deep learning from scratch, or throws you into a sea of random “awesome-LLM” repos, hoping you can connect the dots yourself.
So, I wrote up the roadmap I wish existed a year ago: how to actually learn LLMs and build real things, with none of the bloat. It's geared towards folks with a CS (or practical programming) background who want to skip the endless ML prerequisites and get their hands dirty. The approach:
- Concepts first, then phases, then resources/tools
- Each phase has concrete projects (build an autograd engine, write a mini-GPT, fine-tune with LoRA, etc)
- The goal is to get you actually building and shipping, not just watching lectures
What's your workflow like? I use AI Chat. I load Qwen2.5-1.5B-Instruct with llama.cpp server, fully offloaded to the CPU, and then I config AI Chat to connect to the llama.cpp endpoint.
I think I know what he means. I use AI Chat. I load Qwen2.5-1.5B-Instruct with llama.cpp server, fully offloaded to the CPU, and then I config AI Chat to connect to the llama.cpp endpoint.
This is my second blogpost, and in this one: SWE Agentic Framework – think of it as the puppet master for coders plus Replit's next nemesis. MoEs – imagine a team of AI experts, each shouting answers when it's their topic. Quantizations & Mixed Precision – turning AI from gourmet to fast food without losing the flavor. Batch Inference – AKA AI's quiz night, answering all questions at once. LLM Architectures – blueprints for our chatty AI friends. vLLM and Tensor Parallelism – or the thing that makes big AI models run lean. DeepSeek v2.5 – our open weights savior. Embedding Models – translating human words into AI-understandable numbers. Speculative Decoding – or AI's attempt at mind-reading, guessing your sentences before you finish them.
In the next blogpost I plan on addressing the main pain points of the hardware build and following up on the most-asked questions I received on the first one. I apologize for taking so long to get that out there, but it is taking me more than I expected to properly cover everything I want.
Please let me know if you have any comments or questions, and always feel free to reach out either here or via the social links on my website.
Yeah, you get it. I have two PDUs with NEMA L6-30P plugin, and two NEMA L6-30R outlet. Each outlet is connected to it's dedicated 30A 240v circuit (which gives you a total of 60A, or 6000w).
Cooling is its own story... Man, figuring out this process was hella of a journey
What kind of a dryer? Because that cannot be right. Dryers require a 30amp 240volt dedicated breakers by code in most counties in most states nowadays.
Actually, putting each PSU on its own circuit is crazy dangerous. In the scenario of your suggestion, if one goes out, you are in for a fire. Highly recommend against that.
I agree with you on all of that. I went down the rabbit hole to understand what's up, but I also hired someone and told them exactly what I wanted: breakers amps and volts, outlets type, surge protector over the entire breaker box up to 120k, etc (I am going to be writing about power and electricity in part 3 of this blogpost series). Electricity was on top of the things I was not going to cheap out on because the risk vs reward made no sense to me.
Re: cooling; I have an AC vent directed on the setup, plus planned out in-out in the most optimal way possible to maximize cooling. I have installed like 20 more fans since taking these pictures :D
Yeah, you get it. I am using 3x 1600w 240V Platinum PSUs from Superflower (They're the manufacturer of EVGAs PSUs. Top notch stuff, did my homework) I however decided against server PSUs like the one you suggested because the extra setup overhead + noise.
As I mentioned in my reply to OP, very doable as long as you do your research. The only thing I did not do was not doing the installation itself because I was not comfortable with it, but I pretty much had everything named to the contractor, and even how I would have gone about the installation process was exactly how he did it.
Hit me up on Twitter or Email, we can chat ideas about this venture
Hey man, I think I came across your blog at some point before while trying to figure out my own power plan for this beast (check my comment to OP for more context), so kudos to you for that.
I would say that power limiting is a potential work around, and it should work perfectly fine for inference, but when it comes to trainning you will want to squeeze every ounce of power. So, depends on your goal.
What CPU/Mobo/Storage are you running with those two 3090s for a 700w to work? I am gonna say, if at any point you're pushing more than 500w out of that PSU, you might be risking the 80% safety rule. I would have at least used a 850w just to be safe with two 3090s + rest of hardware.
Oh yeah, my original setup was an RTX 4090 + an RTX 3090, and I swear one night I had the circuit breaker trip more than 15 times before I gave up. I have a UPS so I would run to the box before my system shuts down. Most houses are equipped with 15amp 120v breakers, these should never exceed 1500w, and their max is 1800w but then you're really risking it.
So, as mentioned on the article, I actually have installed (2) 30amp 240v breakers dedicated entirely for this setup (and the next one in case I decide to expand to 16x GPUs over 2 nodes lol). Each breaker is supposed to power up to 6000w at ease. I also installed a specific kind of power outlet that can handle that kind of current, and I have done some extreme research into PDUs. I plan on posting about all of that in this series (part 3 according to my current tentative plans) so stay tuned and maybe bookmark the website/add the RSS feed to your digest/or follow me on any of the socials if this is something that you wanna nail down without spending a month on research like me :'D
Hey guys, this is something I have been intending to share here for a while. This setup took me some time to plan and put together, and then some more time to explore the software part of things and the possibilities that came with it.
Part of the main reason I built this was data privacy, I do not want to hand over my private data to any company to further train their closed weight models; and given the recent drop in output quality on different platforms (ChatGPT, Claude, etc), I don't regret spending the money on this setup.
I was also able to do a lot of cool things using this server by leveraging tensor parallelism and batch inference, generating synthetic data, and experimenting with finetuning models using my private data. I am currently building a model from scratch, mainly as a learning project, but I am also finding some cool things while doing so and if I can get around ironing out the kinks, I might release it and write a tutorial from my notes.
So I finally had the time this weekend to get my blog up and running, and I am planning on following up this blog post with a series of posts on my learnings and findings. I am also open to topics and ideas to experiment with on this server and write about, so feel free to shoot your shot if you have ideas you want to experiment with and don't have the hardware, I am more than willing to do that on your behalf and sharing the findings
Please let me know if you have any questions, my PMs are open, and you can also reach me on any of the socials I have posted on my website.
I've been neck-deep in AI and LLMs for a while now, and I kept running into the same problem: every learning resource out there for LLMs either tries to teach you all of deep learning from scratch, or throws you into a sea of random “awesome-LLM” repos, hoping you can connect the dots yourself.
So, I wrote up the roadmap I wish existed a year ago: how to actually learn LLMs and build real things, with none of the bloat. It's geared towards folks with a CS (or practical programming) background who want to skip the endless ML prerequisites and get their hands dirty. The approach:
- Concepts first, then phases, then resources/tools - Each phase has concrete projects (build an autograd engine, write a mini-GPT, fine-tune with LoRA, etc) - The goal is to get you actually building and shipping, not just watching lectures
Hope you guys find it useful.
-Ahmad