Hi Peter! Heard your name from a couple of friends who have gotten their visas from you! Nice to see you here.
What kind of travel restrictions apply for someone on an O-1 visa (potentially looking to file for EB-1A)? Is there a recommended minimum period of stay in the US?
The main issue is that its very slow and expensive to browse the internet like this. The LLM will only perform well if you have it do chain of thought reasoning, and that has a latency hit because of a longer generation.
Gooey.AI | Full Time | REMOTE / ONSITE | Bengaluru | http://gooey.ai/
We are building a GenAI platform that abstracts all the latest LLMs, animation tools, speech reco, etc from OpenAI, Stability, Google, Meta, etc.
One of our projects - https://farmer.chat - was demoed at the UN General Assembly in April 2023.
Don't use pickle. Use JSON for stuff you can't represent in HTML forms.
For database connections, large datasets, etc (stuff you can't send off to the browser) use redis.
This is a trade-off between DX and UX. Holding session state means you need sticky session routing, and restarting servers kills your user's sessions. Plus imposing a websocket on your users is a cardinal sin, makes scaling incredibly hard, makes page load times abysmal, makes disconnects a nightmare for both you and your user.
Its fine if you're creating something like stable diffusion web ui though, which is meant to be a single user app.
The way to solve session management is to just store the state in html forms and send it back to a stateless http endpoint. That way you get a website that behaves the way all websites work, and can scale easily.
I've also built a similar service[1] that does this with inpainting instead of textual inversion, so it preserves the face exactly and returns in seconds, not hours.