I have been doing this with claude code and openai codex and/or cline. One of the three takes the first pass (usually claude code, sometimes codex), then I will have cline / gemini 2.5 do a "code review" and offer suggestions for fixes before it applies them.
Yes, offloading some layers to the GPU and VRAM should still help. And 11gb isn't bad.
If you're on linux or wsl2, I would run oobabooga with --verbose. Load a GGUF, start with a small number of GPU layers and creep up, keeping an eye on VRAM usage.
If you're on windows, you can try out LM Studio and fiddle with layers while you monitor VRAM usage, though windows may be doing some weird stuff sharing ram.
Would be curious to see the diffs. Specifically if there's a complexity tax in offloading that makes the CPU-alone faster but in my experience with a 3060 and a mobile 3080, offloading what I can makes a big diff.
That is what the RAG system does. The PDF is chunked and thrown into a vector store. And then when prompted, only the relevant bits are retrieved and stuffed into the context and sent to the LLM.
So yeah it's kinda smoke and mirrors. In some cases, for some long PDFs, it works really well. If it's a 500 page PDF with many disparate topics, it may do fine.
I know how that works. And my point was not that they should or will be replaced, but rather that they are no less expendable than developers (not very much).
But the decisions they make are one of the things that can be automated. I do not know if you have been inside one of these places but the executives are not doing a great job deciding (at mine they decided opensearch was a better bet than elastic and switched existing installations).
A new regime came in and then bad decision after bad decision drove our best talent away. Consultants, everywhere.
Also, that number is much lower. Full time devs are down, contractors and consultants are up. As a full time dev at one of these places, it felt like the number of executives was growing as everything else shrank.
Perhaps you are right about the highest levels, but think about all of the middlemen executives and what they do.
And even that -- I think an AI could choose to not spend millions on Deloitte or Accenture on software that inevitably failed.
Formerly engineer @ a large pharma focused on data science projects, primarily JS and Python, also interested in copyright / open source licensing.
hitesh at gmail.