The crate itself is further split between the realtime lighting plugin, base "raytracing scene" plugin that could be used for your own custom raytracing-based rendering, and the reference pathtracer I use for comparing the realtime lighting against.
There were some small changes to the rest of Bevy, e.g. adding a way to set extra buffer usages for the buffers we store vertex/index data in from another plugin https://github.com/bevyengine/bevy/pull/19546, or copying some more previous frame camera data to the GPU https://github.com/bevyengine/bevy/pull/19605, but nothing really major. It was added pretty independently.
I've been working on raytraced lighting in the Bevy game engine, using wgpu's new support for hardware raytracing in WGSL. The initial prototype is launching with the release of Bevy 0.17 tomorrow, but there's still a ton left to improve. Lots of experimenting with shaders and different optimizations.
I've been evaluating texture compression options for including in Bevy https://bevy.org, and there's just, not really any good options?
Requirements:
* Generate mipmaps
* Convert to BC and ASTC
* Convert to ktx2 with zstd super-compression
* Handle color, normal maps, alpha masks, HDR textures, etc
* Open source
* (Nice to have) runs on the GPU to be fast
I unfortunately haven't found any option that cover all of these points. Some tools only write DDS, or don't handle ASTC, or want to use basis universal, or don't generate mipmaps, etc.
Neural nets are great for replacing manually-written heuristics or complex function approximations, and 3d rendering is _full_ of these heuristics. Texture compression, light sampling, image denoising/upscaling/antialiasing, etc.
Actual "generative" API in graphics is pretty rare, at least currently. That's more of an artist thing. But there's a lot of really great use cases for small neural networks (think 3-layer MLPs, absolutely nowhere near LLM-levels of size) to approximate expensive or manually-tuned heuristics in existing rendering pipeline, and it just so happens that the GPUs used for rendering also now come with dedicated NPU accelerator things.
> Metal also definitely has a healthy balance between convenience and low overhead - and more recent Metal versions are an excellent example that a high performance modern 3D API doesn't have to be hard to use, nor require thousands of lines of boilerplate to get a triangle on screen.
Metal 4 has moved a lot in the other direction, and now copies a lot of concepts from Vulkan.
I wrote most of the virtual geometry / meshlet feature, so thanks for the kind words!
It's not quite at the level of Nanite, but I'm slowly getting there. Main limiter is that I do this in my spare time after work, since I don't have any dedicated funding for my work on Bevy. So, expect progress, but it's going to take a while :)
I've used LiveView a little in the past, and besides the lack of static typing (at the time, I think Elixir has gotten some type annotations since), I really liked it and found it easy to work with.
Ecto though, I could never figure out how to use. If I could just make SQL queries in an ORM-style I would understand it, but the repositories and auto generated relations and such I couldn't figure out. Do you have any good resources for learning ecto? I didn't find the official docs helpful.
If you want to make nice looking materials and effects, you need a combination of good lighting (comes from the rendering engine, not the material), and artistic capabilities/talent. Art is a lot harder to teach than programming I feel, or at least I don't know how to teach it.
Programming the shaders themselves are pretty simple imo, they're just pure functions that return color data or triangle positions. The syntax might be a little different than you're used to depending on the shader language, but it should be easy enough to pick up in a day.
If you want to write compute shaders for computation, then it gets a lot more tricky and you need to spend some time learning about memory accesses, the underlying hardware, and profiling.
> An exception that doesn't change the rule, where are the Vulkan extensions for DirectX neural shaders, and RTX kit?
DirectX "neural shaders" is literately the VK_NV_cooperative_vector extension I mentioned previously, which is actually easier to use in Vulkan at the moment since you don't need a custom prelease version of DXC. Same for all the RTX kit stuff, e.g. https://github.com/NVIDIA-RTX/RTXGI has both VK and DX12 support.
> NVidia and AMD keep designing their cards with Microsoft for DirectX first, and Vulkan, eventually.
Not really. For instance NVIDIA released day 1 Vulkan extensions for their new raytracing and neural net tech (VK_NV_cluster_acceleration_structure, VK_NV_partitioned_tlas, VK_NV_cooperative_vector), as well as equivalent NVAPI extensions for DirectX12. Equal support, although DirectX12 is technically worse as you need to use NVAPI and rely on a prerelease version of DXC, as unlike Vulkan and SPIR-V, DirectX12 has no mechanism for vendor-specific extensions (for good or bad).
Meanwhile the APIs, both at a surface level and how the driver implements them under the hood, are basically identical. So identical in fact, that NVIDIA has the nvrhi project which provides a thin wrapper over Vulkan/DirectX12 so that you can run on multiple platforms via one API.
> And not only are we getting tile/block-level primitives and TileIR
As someone working on graphics programming, it always frustrates me to see so much investment in GPU APIs _for AI_, but almost nothing for GPU APIs for rendering.
Block level primitives would be great for graphics! PyTorch-like JIT kernels programmed from the CPU would be great for graphics! ...But there's no money to be made, so no one works on it.
And for some reason, GPU APIs for AI are treated like an entirely separate thing, rather than having one API used for AI and rendering.
> But the biggest problem I'm having is management of buffer space for intermediate objects
My advice for right now (barring new APIs), if you can get away with it, is to pre-allocate a large scratch buffer for as big of a workload as you will have over the program's life, and then have shaders virtually sub-allocate space within that buffer.
> Are you arguing for a better software abstraction, a different hardware abstraction or both?
I don't speak for Raph, but imo it seems like he was arguing for both, and I agree with him.
On the hardware side, GPUs have struggled with dynamic workloads at the API level (not e.g. thread-level dynamism, that's a separate topic) for around a decade. Indirect commands gave you some of that so at least the size of your data/workload can be variable if not the workloads themselves, then mesh shaders gave you a little more access to geometry processing, and finally workgraphs and device generated commands lets you have an actually dynamically defined workload (e.g. completely skipping dispatches for shading materials that weren't used on screen this frame). However it's still very early days, and the performance issues and lack of easy portability are problematic. See https://interplayoflight.wordpress.com/2024/09/09/an-introdu... for instance.
On the software side shading languages have been garbage for far longer than hardware has been a problem. It's only in the last year or two that a proper language server for writing shaders has even existed (Slang's LSP). Much less the innumerable driver compiler bugs, lack of well defined semantics and memory model until the last few years, or the fact that we're still manually dividing work into the correct cache-aware chunks.
You can. There are API extensions for persistently mapping memory, and it's up to you to ensure that you never write to a buffer at the same time the GPU is reading from it.
At least for Vulkan/DirectX12. Metal is often weird, I don't know what's available there.
Fast light transport is an incredibly hard problem to solve.
Raytracing (in its many forms) is one solution. Precomputing lightmaps, probes, occluder volumes, or other forms of precomputed visibility are another.
In the end it comes down to a combination of target hardware, art direction and requirements, and technical skill available for each game.
There's not going to be one general purpose renderer you can plug into anything, _and_ expect it to be fast, because there's no general solution to light transport and geometry processing that fits everyone's requirements. Precomputation doesn't work for dynamic scenes, and for large games leads to issues with storage size and workflow slow downs across teams. No precomputation at all requires extremely modern hardware and cutting edge research, has stability issues, and despite all that is still very slow.
It's why game engines offer several different forms of lighting methods, each with as many downsides as they have upsides. Users are supposed to pick the one that best fits their game, and hope it's good enough. If it's not, you write something custom (if you have the skills for that, or can hire someone who can), or change your game to fit the technical constraints you have to live with.
> Nobody has a good solution to this yet. What does the renderer need to know from its caller? A first step I'm looking at is something where, for each light, the caller provides a lambda which can iterate through the objects in range of the light. That way, the renderer can get some info from the caller's spatial data structures. May or may not be a good idea. Too early to tell.
Some games may have their own acceleration structures. Some won't. Some will only have them on the GPU, not the CPU. Some will have an approximate structure used only for specialized tasks (culling, audio, lighting, physics, etc), and cannot be generalized to other tasks without becoming worse at their original task.
Fully generalized solutions will be slow be flexible, and fully specialized solutions will be fast but inflexible. Game design is all about making good tradeoffs.
1. Fail in the first X amount of time
2. Fail towards the end of their rated lifespan
So buying used drives doesn't seem like the worst idea to me. You've already filtered out the drivers that would fail early.
Disclaimer: I have no idea what I'm talking about