Oh, you're the MarshallH ? Thanks so much for everything you've done !
I'm just a nobody who wrote DotN64, and contributed a lil' bit to CEN64, PeterLemon's tests, etc.
For objn64, I don't think PNG was patched in. I only fixed a handful of things like a buffer overflow corrupting output by increasing the tmp_verts line buffer (so you can maximise the scale), making BMP header fields 4 bytes as `long` is platform-defined, bumping limits, etc. Didn't bother submitting patches since I thought nobody used it anymore, but I can still do it if anyone even cares.
Since I didn't have a flashcart to test with for the longest time, I couldn't really profile, but the current microcode setup seems to be more than fine.
Purely out of curiosity, as I now own a SC64, is the 64drive abandonware ? I tried reaching out via email a couple times since my 2018 order (receipt #1532132539), and I still don't know if it's even in the backlog or whether I could update the shipping address. You're also on Discord servers but I didn't want to be pushy.
I don't even mind if it never comes, I'd just like some closure. :p
There were many microcode versions and variants released over the years. IIRC one of the official figures was ~180k tri/sec.
I could draw a ~167,600 tri opaque model with all features (shaded, lit by three directional lights plus an ambient one, textured, Z-buffered, anti-aliased, one cycle), plus some large debug overlays (anti-aliased wireframes for text, 3D axes, Blender-style grid, almost fullscreen transparent planes & 32-vert rings) at 2 FPS/~424 ms per frame at 640x476@32bpp, 3 FPS/~331ms at 320x240@32bpp, 3 FPS/~309ms at 320x240@16bpp.
That'd be between around 400k to 540k tri/sec. Sounds weird, right ? But that's extrapolated straight from the CPU counter on real hardware and eyeballing, so it's hard to argue.
I assume the bottleneck at that point is the RSP processing all the geometry, a lot of them will be backface culled, and because of the sheer density at such a low resolution, comparatively most of them will be drawn in no time by the RDP. Or, y'know, the bandwidth. Haven't measured, sorry.
Performance depends on many variables, one of which is how the asset converter itself can optimise the draw calls. The one I used, a slight variant of objn64, prefers duplicating vertices just so it can fully load the cache in one DMA command (gSPVertex) while also maximising gSP2Triangle commands IIRC (check the source if curious). But there's no doubt many other ways of efficiently loading and drawing meshes, not to mention all the ways you could batch the scene graph for things more complex than a demo.
Anyways, the particular result above was with the low-precision F3DEX2 microcode (gspF3DLX2_Rej_fifo), it doubles the vertex cache size in DMEM from 32 to 64 entries, but removes the clipping code: polygons too close to the camera get trivially rejected. The other side effect with objn64 is that the larger vertex cache massively reduces the memory footprint (far less duplication): might've shaved off like 1 MB off the 4 MB compiled data.
Compared to the full precision F3DEX2, my comment said: `~1.25x faster. ~1.4x faster when maxing out the vertex cache.`.
All the microcodes I used have a 16 KB FIFO command buffer held in RDRAM (as opposed to the RSP's DMEM for XBUS microcodes). It goes like this if memory serves right:
1. CPU starts RSP graphics task with a given microcode and display list to interpret from RAM
2. RSP DMAs display list from RAM to DMEM and interprets it
3. RSP generates RDP commands into a FIFO in either RDRAM or DMEM
4. When output command buffer is full, it waits for the RDP to be ready and then asks it to execute the command buffer
5. The RDP reads the 64-bit commands via either the RDRAM or the cross-bus which is the 128-bit internal bus connecting them together, so it avoids RDRAM bus contention.
6. Once the RDP is done, go to step 2/3.
To quote the manual:
> The size of the internal buffer used for passing RDP commands is smaller with the XBUS microcode than with the normal FIFO microcode (around 1 Kbyte). As a result, when large OBJECTS (that take time for RDP graphics processing) are continuously rendered, the internal buffer fills up and the RSP halts until the internal buffer becomes free again. This creates a bottleneck and can also slow RSP calculations. Additionally, audio processing by the RSP cannot proceed in parallel with the RDP's graphics processing. Nevertheless, because I/O to RDRAM is smaller than with FIFO (around 1/2), this might be an effective way to counteract CPU/RDP slowdowns caused by competition on the RDRAM bus. So when using the XBUS microcode, please test a variety of combinations.
> You could almost treat the DSPs as the compute units of modern GPU architectures: they definitely processed the vertices, and nothing stops you from adding a "vertex shader" pass, however, because it's not directly integrated into the graphics pipeline, it's harder to emulate a true pixel shader, you might be limited to full-screen pixel shading since there's no feedback from the rasteriser about which pixels exactly get written to memory.
Actually, you could probably just use the depth buffer as write-only with constant values to identify objects as different kinds, and use a "pixel shader" pass to apply effects on those annotated pixels, but I assume this might be a pretty expensive technique.
... Now that I think about it, it's not unlike modern rendering engines.
Computer graphics has a very interesting history, and like many things, there's a lot to learn from studying it. Surprisingly, many techniques and principles are still relevant to this day.
Thankfully, the Internet is full of documentation, post-mortem accounts and reference implementations to learn more about all this.
Here's a bunch of random noteworthy things and references I forgot to link (sorry for potentially digging rabbit holes):
The best information tends to come from official documentation detailing almost everything you want to know about the inner workings of systems. Additionally, unofficial community documentation can also be of great quality and complement official sources.
The architecture overview posts from Rodrigo Copetti (https://www.copetti.org/writings/consoles/) pack a lot of accurate information at a glance, and are a great starting point if a specific topic piques your interest.
Emulators' progress reports can reveal a lot about the details and intricacies required for accurately replicating these systems' features: https://dolphin-emu.org/blog/ is one such amazing source of information, but all other major emulation efforts have equally interesting content.
Transparency sorting of surfaces (https://en.wikipedia.org/wiki/Order-independent_transparency) is a hard problem to solve for traditional scanline renderers (most PC/console GPUs), to the point that even today's releases can ship with somewhat obvious rendering errors.
On the other hand, tiled renderers (used in the Dreamcast, mobile hardware, and Apple silicon) are able to solve this problem due to their very nature, albeit trading another set of drawbacks, though the completely different hardware approach is a nice read (https://en.wikipedia.org/wiki/Tiled_rendering).
Someone once shared a video demonstrating a 3D package from ancient Lisp machines (https://youtu.be/gV5obrYaogU?t=30), and it was almost shocking to see how many things were familiar and done right from a modern perspective.
Reimplementing new techniques on old hardware. For example, someone implemented real-time shadows and toon shading on N64 (https://youtu.be/VqDAxcWnq3g).
For fun, you can grab RenderDoc (https://renderdoc.org/) and copies of your favourite games to analyse their frames (even via emulation): see how developers implement or fake visual effects and generally how these games render their world.
For instance, Dolphin emulates the GameCube's semi-programmable texture environment unit (TEV) via a pixel shader, and its shader source code is directly visible and editable, with the resulting output shown in the texture viewer. With the aid of textures, you can implement refraction, reflection and heat haze, among other effects.
TL;DR yes and no (thanks to DSPs, technically some were an integral part of the GPU):
In the era of semi-programmable graphics pipelines, of which there was the N64, PS2 and GameCube, you would not be able to solve general-purpose problems using only that pipeline: given a handful of colour mixing formulas, registers, textures and vertices to control the pixels to write back to memory, I fail to come up with any tangible problem you could hack them into solving.
However, what the 3D generation of video game consoles had to bundle with the hardware in order to process complex scenes at interactive frame rates were digital signal processors (DSPs). Think of them as regular processors, but with an extra vector unit, allowing you to crunch enormous amounts of data in parallel (single instruction, multiple data instructions, or SIMD).
This was essential to perform geometry transformations, which benefit greatly from parallelism, in order to build graphics commands to pass down to the rasteriser.
I am unsure if the SEGA Saturn and PS1's vector-accelerated instructions are sufficiently general-purpose to allow them perform any computation you wish to do, but at least starting with the Nintendo 64, you could write microcode to accelerate any task you'd like, provided you were brave enough to deal with the harsh memory constraints of microcode and obscure documentation.
To give concrete examples, the PS1 came with a dedicated chip (the MDEC, or motion decoder) to decode MPEG video frames from the CD directly into the VRAM to display on screen.
While the N64 doesn't have dedicated video decoding chips, its signal processor was designed in such a way to allow microcode to do an equivalent job (not to mention YUV texture decoding on the rasteriser's side).
So this did not stop a talented Resident Evil 2 developer fresh out of school to write an MPEG video decoder microcode to be able to cram two CDs worth of video data (around 1 gigabyte, down to a 64 megabyte cartridge), of course with other smart decisions like data deduplication and further compression/interlacing.
Another one was Rare developers writing an MP3 audio decoder microcode to store large amounts of dialogue.
And finally, in recent times, a developer managed to write an H.264 video decoder microcode, for a machine that was released almost a decade prior to this codec's birth.
You can also accelerate physics and anything else that would benefit from SIMD, really, and in fact, while more modern CPUs integrate SIMD instructions in their tool chest that compilers can take advantage of, the PS3's Cell processor was a brief throwback to the old hardcore ways, before GPGPU became king.
You could almost treat the DSPs as the compute units of modern GPU architectures: they definitely processed the vertices, and nothing stops you from adding a "vertex shader" pass, however, because it's not directly integrated into the graphics pipeline, it's harder to emulate a true pixel shader, you might be limited to full-screen pixel shading since there's no feedback from the rasteriser about which pixels exactly get written to memory.
I'm just a nobody who wrote DotN64, and contributed a lil' bit to CEN64, PeterLemon's tests, etc.
For objn64, I don't think PNG was patched in. I only fixed a handful of things like a buffer overflow corrupting output by increasing the tmp_verts line buffer (so you can maximise the scale), making BMP header fields 4 bytes as `long` is platform-defined, bumping limits, etc. Didn't bother submitting patches since I thought nobody used it anymore, but I can still do it if anyone even cares.
Since I didn't have a flashcart to test with for the longest time, I couldn't really profile, but the current microcode setup seems to be more than fine.
Purely out of curiosity, as I now own a SC64, is the 64drive abandonware ? I tried reaching out via email a couple times since my 2018 order (receipt #1532132539), and I still don't know if it's even in the backlog or whether I could update the shipping address. You're also on Discord servers but I didn't want to be pushy.
I don't even mind if it never comes, I'd just like some closure. :p
Thanks again !