Jason, here is a story about how much your work impacts us.
We run a decently sized company that processes hundreds of millions of images/videos per day. When we first started about 5 years ago, we spent countless hours debugging issues related to memory fragmentation.
One fine day, we discovered Jemalloc and put it in our service, which was causing a lot of memory fragmentation. We did not think that those 2 lines of changes in Dockerfile were going to fix all of our woes, but we were pleasantly surprised. Every single issue went away.
Today, our multi-million dollar revenue company is using your memory allocator on every single service and on every single Dockerfile.
For website hosting, it's okay but not great. We encountered issues when we tried to cache a lot of images. Their CDN storage seems really low compared to Cloudflare and Cloudfront. It results in a really bad hit ratio the moment we try to deliver a lot of images.
These servers are indeed job processing servers. They are critical but not milliseconds critical. Cooling, security, monitoring, backup generators, and backup of data all are taken care of.
Nice ideas, but we have chosen a really simple Kubernetes deployment. We only install the host OS (ubuntu server) and then join the self-hosted GPUs as workers in a Kubernetes cluster.
No other task is needed and our Grafana monitors if the server (and its containers) are up and running.
Interestingly, we tried that RTX4000 before we decided to buy our own. Yes, ours will break even in 14 months. ($2300 cost + $40 per month datacenter cost)
Streaming video is still hard to do for a developer today and we are solving that with scalable and cheap infra for streaming.