I wanted to share what I've been working on - it's a AI site builder that runs in the browser powered by WebGPU and OnnxRuntime-Web.
I have got the following all working to varying degrees:
- text to code generation
- image to code generation
- microphone to text to code generation
If you are on Mac for instance, it will interface directly with your GPU to power the LLM interface. It only requires downloading the models, and then everything after that is offline.
It's not even close to as powerful as Claude or ChatGPT, but I like the idea of having the LLM run directly on your machine.
More technical insight:
- I also got the Typescript / React app to compile itself in the browser via a service worker https://github.com/pdufour/llm-coder/blob/main/src/sw.js but took it offline due to some oddities with service workers.
- A lot of the new speech models are a lot better than anything built into your phone / computer. I wonder when more computers will have them built in.
- I added a CSP to the iframe only because I was worried about spamming sites since I update the iframe anytime a new token comes in. So if you have an image on the page it will get reloaded every time the iframe is updated. Otherwise there would be no reason for it.
I have got the following all working to varying degrees:
- text to code generation
- image to code generation
- microphone to text to code generation
If you are on Mac for instance, it will interface directly with your GPU to power the LLM interface. It only requires downloading the models, and then everything after that is offline.
It's not even close to as powerful as Claude or ChatGPT, but I like the idea of having the LLM run directly on your machine.
I just did this for fun, but I am looking for a new role if anyone's hiring - https://www.linkedin.com/in/pauldufour/!
More technical insight: - I also got the Typescript / React app to compile itself in the browser via a service worker https://github.com/pdufour/llm-coder/blob/main/src/sw.js but took it offline due to some oddities with service workers.
- A lot of the new speech models are a lot better than anything built into your phone / computer. I wonder when more computers will have them built in.
- I added a CSP to the iframe only because I was worried about spamming sites since I update the iframe anytime a new token comes in. So if you have an image on the page it will get reloaded every time the iframe is updated. Otherwise there would be no reason for it.