I have been thinking along these lines myself. Most of the time, if we need to calculate things, we'd use a calculator or some code. We wouldn't do it in our head, unless it's rough or small enough. But that's what we ask LLMs to do!
I believe we juggle 7 (plus or minus 2) things in our short term memory. Maybe short term memory could be a tool!
We also don't have the knowledge of the entire internet in our heads, but meanwhile we can still be more effective at strategy/reasoning/planning. Maybe a much smaller model could be used if the only thing it had to do is use tools and have a basic grasp on a language.
> The cost of computing decreases over time. No one will pay $100 Billion to train GPT-6. That is absurd. The current top supercomputer in the world (Frontier) cost $600M.
Just wanted to ask the question - do you think frontier has provided more or less value to the world than gpt-4?
I agree 100% and I'd add, static typing helps when you inherit a poorly written codebase. This unfortunately is probably 99% of codebases in the wild. It has happened so many times when i've seen a lack of test coverage, a lack of understanding of the codebase but the business requirement to make changes.
Having a type system in place makes minor refactoring possible in this nightmare scenario.
I have been in the situation of poorly written ruby codebases and I can tell you 100% that I would prefer to have a poorly written java codebase with its static types. I prefer ruby as a language but man when it's bad, it's terrible. Just trying to work out the intent of a function when multiple types are passed in as the same argument over the codebase is pure hell.
2022 m1 macbook pro here - and i have one from work too. never had either crash, haven't plugged this one in for literally days, and right now i have an ide, vscode, ableton, slack, chrome with like 400 tabs in multiple windows and it's cool to the touch.
i'd love to know the operations they were doing and what they used the sorted sets for. I find for this type of thing, if the ids of the users are integers you can often use a bitset and optimise the crap out of it via SIMD. If memory is a huge concern, or if the data is super sparse you can use a roaring bitset.
I have solved a similar problem, in a similar way and i've found polars <https://www.pola.rs/> to solve this quite well without needing clickhouse. It has a python library but does most processing in rust, across multiple cores. I've used it for data sets up to about 20GB no worries, but my computer's ram became the issue, not polars itself.