I love the style of this blog-post, you can really tell that Luke has been deep down in the rabbit hole, encountered the Balrog and lived to tell the tale.
I use a keyboard shortcut to start and stop recording and it will put the transcription into the clipboard so I can paste into any app.
It's a huge productivity boost - OP is correct about not overthinking trying to be that coherent - the models are very good at knowing what you mean (Opus 4.5 with Claude Code in my case)
I agree with this - I hear a lot of hate towards vibe coding but my experience with voice dictation and using 20 years experience in the trenches and so being very specific telling the model what to do has been, well, refreshing to say the least.
I used to pride myself of knowing all the little ins and outs of the tech stack, especially when it comes to ops type stuff. This is still required, the difference is you don't need to spend 4 hours writing code - you can use the experience to get to the same result in 4 minutes.
I can see how "ask it for what you want and hope for the best" might not end well but personally - I am very much enjoying the process of distilling what I know we need to do next into a voice dictated prompt and then watching the AI just solve it based on what I said.
I would pay hundreds of dollars per month for the combination of cursor and claude - I could not get my head around it when my beginner lever colleague said "I just coded this whole thing using cursor".
It was an entire web app, with search filters, tree based drag and drop GUIs, the backend api server, database migrations, auth and everything else.
Not once did he need to ask me a question. When I asked him "how long did this take" and expected him to say "a few weeks" (it would have taken me - a far more experienced engineer - 2 months minimum).
His answer was "a few days".
What I'm not saying is "AGI is close" but I've seen tangible evidence (only in the last 2 months), that my 20 year software engineering career is about to change and massively for the upside. Everyone is going to be so much more productive using these tools is how I see this.
We used https://github.com/pgvector/pgvector under the hood and found it extremely easy to integrate with our database schema - being able to just specify the structure of a table and have metadata fields alongside the embeddings made the code very easy to reason about.
Great question! scheduling workloads onto GPUs in a way where VRAM is being utilised efficiently was quite the challenge.
What we found was the IO latency for loading model weights into VRAM will kill responsiveness if you don't "re-use" sessions (i.e. where the model weights remain loaded and you run multiple inference sessions over the same loaded weights).
Obviously projects like https://github.com/vllm-project/vllm exist but we needed to build out a scheduler that can run a fleet of GPUs for a matrix of text/image vs inference/finetune sessions.
We are finding that fine tuning is very good at setting the style and tone of responses. A potential use case we are thinking about is what if your star sales person leaves the company? Could you fine tune an LLM on their conversations with customers and then do inference where it would write text in the style of your star sales person.
We are also adding function calling so the model would know to reach out to an external API to fetch some data before generating a response.
Thanks for the feedback - I agree that fine tuning a) has potential and b) is not easy :-)
> Also, I don’t get the comparison of rag vs finetuning in articles like this - why not do both
It's interesting you say this because we are very close to adding RAG support to Helix sessions and it will be "both at the same time" not an "either or" setup.
You can choose to do either or but we are interested in seeing if doing both at the same time yields better results than either or - watch this space!
This is spot on, the thing we've not yet done is make it easy to import a repo(s) code and the associated metadata into a fine tuning session easily.
> I often wonder how you'd go about organizing training data for a full historic github repo in a way that makes sense for training (or RAG)?
This is the hard part :-) But you are right - it would be intriguing to see what the output of a fune-tuned & RAG model would look like for this use-case. We are currently experimenting with adding RAG alongside the fine tuned model (so it's both, not either or) to see if it produces better results.
I will make sure we take a look at the gihub repo use case because it feels like that would be an interesting experiment to do!
Fine tuning on your documents will really help to answer questions in the style and tone of those documents, so in that way, yes it helps.
It would be possible to include some parts of the new documents in the prompt so you can answer questions about new facts in the style and tone of your old documents, which we feel is useful. We are also experimenting with adding Retrieval Augmented Generation alongside fine tuning to see if the results are better than either or.
This is a really cool blogpost - that you can influence a model with guidance towards a desired outcome feels like a really valuable feature. I'm currently working on a project that is generating stable diffusion images and this kind of technique would be so useful ;-)
We are thinking for people writing documentation, blogs or Intranets where there are many different pages and there is a more polished sense of branding.
You are quite right that if you want to quickly publish a single page - using Google docs directly would be faster :-)
Yes - dealing with access to a users entire drive is something we are looking at to try and get working for exactly that reason - the words `this app can delete all your files` is scary :-)
There isn't an option to remove the footer but should be +1 added to backlog - thanks again!
We've been working hard on https://nocode.works - which turns Google docs into nice looking websites - it's not pluggable yet but it doesn't require any setup on behalf of the user.
We do have a template system though so it would be possible for a frontend dev to design their own wrapper for the content.
Perhaps part of that was that IE was really bad at it's job and as users were told about Chrome and Firefox and they tried it and it was much faster - they switched because IE was not nearly as good. Facebook is pretty good at it's job - despite questionable practices. If Facebook started to suck from a user perspective (a bit like MySpace did) - users would move to another thing in a shot.