IBM 3380 Model K introduced in 1987, has 7.5 GB of storage and costed about $160000 to $170000, or adjusted for inflation it is $455000 in 2025 US dollars, that's $60666/GB. A Solidigm D5-P5336 drive that can store 128 TB costs about $16500 in 2025 US dollars, that is $0.129/GB. That's a 470279x price reduction in slightly less than 40 years. So what is likely going to happen to LLM pricing? No one knows and both your example as well as mine doesn't mean anything.
I think it really depends on the task at hand, and not that intuitive. At some point accessing the memory might be slower than repeating the computation, especially when the storage is slow.
have you considered using meshlets technique like Unreal Engine Nanite or Assassin's Creed? It could potentially open the door for better culling and more effective depth prepass.
It is important to not leave out the context that Google needed TPUs for their AI development. Now the question is what does Zoho email and office software gain from fabricating custom chips?
If the goal is to make a CPU that is "fast enough to host H100 competitor AI hardware", then why bother with x64? Huawei could have just produced a powerful ARM chip to go with their new AI processor. After all, Nvidia GB200 also uses Arm-based CPUs (the Grace in GB).
DuckDB has certain optimizations which allows it to read only parts of a parquet file. It can also read remote file in streaming fashion so it does not have to wait for the entire file to be downloaded or to store a large amount of data in memory.
Relevant documentation: https://duckdb.org/2021/06/25/querying-parquet.html