don't you think it's highly unlikely that someone will stumble over the power cable in a hosted datacenter like hetzner?
and even if, you could just run a provisioned secondary server that jumps in if the first becomes unavailable and still be much cheaper.
yea right.
Privacy is a fundamental right in the EU (GDPR, Charter of Fundamental Rights), while the U.S. legal system offers almost no general privacy protection. On top of that, the NSA has a long history of warrantless surveillance and backdoors (Snowden, PRISM), with very limited oversight. In practice, it’s far costlier to push mass privacy infringements in Europe than in the U.S.
esp. for image data libraries, why not provide the images as a dump instead? No need to crawl 3mil images if the download button is right there. Now put the file on a cdn or Google and you're golden
we use it and are fairly happy. but provider latency is insanely high (500ms+) for embedding models. best to host on-cluster.
hybrid quality is good but modification options are extremely limited and the score very obscure for anything but ranking within the set.
> For each predicted output token, I want to know exactly which source document(s) were utilized including indices from those documents and relevant statistics.
You don't do this for any other kind of decision or tool, why do you need it for LLMs?
i think in truth you need a source that convinces you (or the regulator) that your choice is acceptable, so that you can pass off the responsibility.
If the LLM were to give you an answer backed by relevant (cited) source documents of regulations and a good explanation it would make no difference compared to a human worker doing the same. -> this is already possible
SMART goals and KPIs are meant to improve work focus, not work ethos. Those two are not in opposition to each other. IME engineers who don't want to do measurably good work are inexperienced, disgruntled or unmotivated.
Forcing goals onto a disgruntled/unmotivated engineer will end badly.
Giving no goals to motivated engineers will also end badly.
Giving bad, unrelated or unachievable goals to motivated engineers will end in disgruntled or unmotivated engineers.
Thanks for your input!
server-side rendering should fix most of the first issue. You can get all the relevant data pre-rendered and it's pretty easy to do with the right framework. I've used quasar.dev for this in the past. It should also solve the initial load times (to an extent).
The chaining can be fixed with lazy loading/code splitting.
The only thing i haven't fixed yet is hiding the cookie banner for bots :/
This sounds like a perfect job for jina.ai -> sharding, redudancy, automatic up- and down scaling, security features and most importantly: flexibility to use and switch whatever (vector) database
https://jina.ai/