As you might have guessed, connection pools don't work well with Lambda. For database connection pool, they came up with another chargeable service, RDS Proxy, to fix the problem created by Lambda :)
I am not familiar with Replicate, but based on their website, they charge per GPU type. I didn't see the GPU type set in the example. Is it baked in as part of the "a16z-infra/llama13b-v2-chat" model?
As someone who straddles both Python and Java world, I don't seem hear this kind of malicious packages problem with Maven (central) as I do with PyPI. What could be the difference? Perhaps Java world is full of old farts like me doing mundane thing with a handful of well established libraries.
Cloud is a service that moves CapEx to OpEx. What ever cost savings it may provide is a side effect of this move. If you moved to cloud and do not see any savings, that means you have sized planned your capacity well. But there are a lot of businesses that cannot plan ahead for various reasons, and cloud provides value to them.
Perhaps we should look at how CAD impacted draughtsmen. Their job scope basically got folded into engineers'. There are probably a lot of us who are software draughtsmen.
First sentence of chapter 1: "Artificial Intelligence (AI) is that branch of computer science dealing with symbolic, nonalgorithmic methods of problem solving. "
Interesting how it's almost opposite of what the field is now.