I was an architect on the Anton 2 and 3 machines - the systolic arrays that computed pairwise interactions were a significant component of the chips, but there were also an enormous number of fairly normal looking general-purpose (32-bit / 4-way SIMD) processor cores that we just programmed with C++.
I love stuff like this! I have a Kaypro 2/84, which supports a display resolution of 160x100 with colors ranging from green, to 'less green' to black, and I went down a rabbit hole of trying to push the graphics: https://www.chrisfenton.com/exploring-kaypro-video-performan... - I was eventually able to get it do display (short!) 50 fps video clips.
I built a more whimsical version of this - my daughter and I basically built a 'junk robot' from a 1980s movie, told it 'you're an independent and free junk robot living in a yard', and let it go: https://www.chrisfenton.com/meet-grasso-the-yard-robot/
I did this like 18 months ago, so it uses a webcam + multimodal LLM to figure out what it's looking at, it has a motor in its base to let it look back and forth, and it use a python wrapper around another LLM as its 'brain'. It worked pretty well!
Mine was entirely mechanical (driven by punch cards and a hand-crank), and changed all of the pixels in parallel, but a lot of the mechanism development looked extremely familiar to me.
I think a quad-CPU X-MP is probably the first computer that could have run (not train!) a reasonably impressive LLM if you could magically transport one back in time. It supported a 4GB (512 MWord) SRAM-based "Solid State Drive" with a supported transfer bandwidth of 2 GB/s, and about 800 MFLOPS CPU performance on something like a big matmul. You could probably run a 7B parameter model with 4-bit quantization on it with careful programming, and get a token every couple seconds.
The Cray PVP line was also doing double precision floating point, and could overlap vector memory operations with math operations. My guess is that you would need a microcontroller operating at several hundred MHz to beat a Cray-1 in practice. The later Cray-1/S and /M variants also supported a 10gbps link to an SSD of several hundred megabytes, which is hard to beat in a microcontroller.
I took a different approach by just making an FPGA-based multi-core Z80 setup. One core is dedicated to running 'supervisor' CP/NET server, and all of the applications run on CP/NET clients and can run normal CP/M software. I built a 16-core version of this, and each CPU gets its own dedicated 'terminal' window, with all of the windowing handled by the display hardware (and ultimately controlled by the supervisor CPU). It's a fun 'what-if' architecture that works way better than one might expect in practice. It would have made an amazing mid-to-late 1980s machine.
I spend all day writing C++ or Python, and like playing around with Turbo Pascal on a circa-1984 Kaypro 2 as a hobby machine - the projects are certainly smaller, but my edit-compile-run loop on my Kaypro is usually faster in practice (even running on a 4 MHz Z80 with 64KB of RAM compared to an 8-core 3+ GHz workstation with 64GB of RAM) than my 'modern' work. It's genuinely crazy to me how usable a 40 year old machine is for development work.
FPGAs are particularly amazing for this sort of project where you would otherwise need a custom ASIC that could never be even remotely economical to build (see my 16-core Z80 laptop as another excellent example: http://www.chrisfenton.com/the-zedripper-part-1/). It lets you play 'fantasy computer architect.'
It updates in pretty close to realtime (ie the simulator timestep is extremely close to the actual wall clock time required to compute the next timestep), so “animation” isn’t really the right word. If you let it go for a few hours it leaves fun star trails though!
Author here! Most of my projects are kind of like this penny arcade comic (https://www.penny-arcade.com/comic/2011/06/13/the-hippothala...) - I saw where something was not, and said "No. This will not do." It's 2024 and a supercomputer wrist watch is possible, so now it exists =)