Author of collabmem here! Let me add some detail on how the memory stays manageable as it grows.
Two mechanisms keep it sustainable, neither deletes anything, and both are discussed with you before being applied:
- Upward consolidation — when the episodic index grows large, mature stable knowledge from old episodes is extracted into the world model. Consolidated index entries move to a searchable archive; the original notes stay put. The active index keeps focused on recent work while the world model absorbs what's been learned.
- Downward compaction — when a world model file approaches its size cap, it's rewritten to stay compact. Removed knowledge is preserved in an episodic note so it remains iscoverable.
Caveat: these two mechanisms are designed but not yet tested; this is one of my high-priority todos. Feedback especially welcome here.
Happy to answer questions — looking forward to feedback!
> Instead of modifying spacetime, the theory – dubbed a “postquantum theory of classical gravity” – modifies quantum theory and predicts an intrinsic breakdown in predictability that is mediated by spacetime itself. This results in random and violent fluctuations in spacetime that are larger than envisaged under quantum theory, rendering the apparent weight of objects unpredictable if measured precisely enough.
I have so many questions about this paragraph; how did they change quantum theory? Are they saying that breakdown in predictability at the quantum level is actually caused by space-time? Why does this result in violent fluctuations in space-time? And why are the space-time fluctuations larger than in the case of quantum gravity?
Related question, does anyone have experience with using the AMD MI100 for deep learning? With 32GB and a second hand price of ~1100 USD, it could be a good choice.
I’m wondering about the performance of such a cluster of repurposed mining GPUs. Mining doesn’t need high bandwidth interconnects nor large per GPU memory, on the other hand training deep learning models such as LLMs does need that.
I’m not saying it’s impossible to get good performance with custom training solutions; that would actually be very valuable as it would unlock the computing power of lower grade setups.
Like others mentioned in this thread, I don't think this is Apple's final vision for a Apple Silicon Mac Pro. To me, the only way forward is for Apple to, either use PCIe, or another proprietary high bandwidth bus solution, to connect together multiple Apple Silicon boards. This would result in a Mac Pro in which you can slide in multiple Apple Silicon modules, i.e. Mx Ultra modules.
For distributed Machine Learning workloads it also make sense to combine a CPU and a GPU per pluggable module. In such workloads, data that needs to be processed by a specific GPU (i.e. for training), is usually preprocessed by a CPU. In Intel machines with multiple Nvidia cards, this implies that you need a single beefy CPU with high core count. Having a CPU and GPU in each module makes sense from this perspective.
I'm wondering to what extend these benchmarks, especially the GPU benchmarks, are really optimised for the specific Apple M2 architecture? And to what extent the high GPU-GPU bandwidth is benchmarked, this also influences real-world performance.
This is the result of a system without any value judgment or morals, that’s the scary part. If these items are from existing lists it picked lists from authoritarian and totalitarian playbooks.
I’m a bit skeptical. ChatGPT can still hallucinate, generating information that seems correct, but is in fact nonsense. I’m wondering how they are going to deal with that.
Only tangentially related; why is Teams removing space indentations in chats on MacOS? It’s so annoying when you type a piece of (pseudo) code, all leading spaces are removed! It drives me nuts, why would anyone implement this!
I’m not using an M1 machine yet, but I think is extremely useful to be able to do some small scale experiments for testing and debugging purposes locally before starting a full-scale training job in the cloud.
Further, in general, I’m really charmed by the potential of having unified memory; the idea you can test some batch training iterations with a batch size that fills almost all 128MB is a unique capability!
Last, a large amount of unified memory allows to do inference and “prompt engineering” with very large models, locally. E.g. Using GPT-J like models (6B parameters).
I don’t know much about the black hole information loss problem and I do like Sabine’s skepticism. However, saying it has no use to work on mathematical solutions in this area is taking it one step too far. Maybe, based on the solutions other (indirect) experiments can be devised, maybe it will help solve other Physics, mathematics or engineering problems in the future.
Apparently they are going to use “HIP” to convert CUDA applications to be able to run on AMD:
> The OLCF plans to make HIP available on Summit so that users can begin using it prior to its availability on Frontier. HIP is a C++ runtime API that allows developers to write portable code to run on AMD and NVIDIA GPUs. It is essentially a wrapper that uses the underlying CUDA or ROCm platform that is installed on a system. The API is very similar to CUDA so transitioning existing codes from CUDA to HIP should be fairly straightforward in most cases. In addition, HIP provides porting tools which can be used to help port CUDA codes to the HIP layer, with no loss of performance as compared to the original CUDA application. HIP is not intended to be a drop-in replacement for CUDA, and developers should expect to do some manual coding and performance tuning work to complete the port.
Would this one day lead to a HIP based PyTorch? I hope so!
What software will they use to, for instance, train large deep learning models? Nvidia has CUDA, AMD has what? Are they writing new software from scratch? Maybe they have a lot of frameworks to solve problems in the “traditional” HPC space (eg weather forecasts), but in the ML space I only heard of ROCm which seems to be poorly supported.
AMD seems such an odd choice for “AI supercomputers”.
I decided a few weeks ago to transition to PyTorch (was using Keras before) and I must say that I really love it! How PyTorch is structured gives me the right balance between ease of use and the ability to make customisations. Further, using DistributedDataParallel, dividing the work over multiple processes, where each process uses one GPU, is very fast and GPU memory efficient.
Before my switch I tried out Keras for Tensorflow, and even got a lot of support from Google in my endeavours to resolve the issues I encountered (kudos to Google for that!). In the end I felt it was still not mature enough. Further, although I do believe TF and Keras are moving in the right direction, I still felt that in some cases the way the software was set up just didn't sit well with me.
Maybe it is worth to try again in a year or so, or by then I will tryout Swift for Tensorflow, which I think has a great future ahead.
Two mechanisms keep it sustainable, neither deletes anything, and both are discussed with you before being applied:
- Upward consolidation — when the episodic index grows large, mature stable knowledge from old episodes is extracted into the world model. Consolidated index entries move to a searchable archive; the original notes stay put. The active index keeps focused on recent work while the world model absorbs what's been learned.
- Downward compaction — when a world model file approaches its size cap, it's rewritten to stay compact. Removed knowledge is preserved in an episodic note so it remains iscoverable.
Caveat: these two mechanisms are designed but not yet tested; this is one of my high-priority todos. Feedback especially welcome here.
Happy to answer questions — looking forward to feedback!