GPU processes are separated due to driver instability and unsafe code, so this means more than a GPU process crash, or a content process crash (tab crashed), something in the orchestration layer or main process crashed. There are probably massive amounts of memory being moved around which could easily be stressing the system or running into large limits. Don't think this is related to the security model or a compromise of an origin / content process.
Just wander itself is almost working in a browser - kind of hacked together right now, but will be pushed to the open-source soon. The same Wasm payload should be able to run in the browser or through wasmtime.
For the rive-renderer / 2D integration, it is going to be a much longer path to get working in a browser together with wander.
No, the goal is not to create a C++ API to give you GPU functions.
The C++ API for wander is used to embed the WebAssembly module of graphics code into the application. The API footprint is very small - load a file, pass parameters to it, iterate through the tree it produces.
This could be viewed as logically equivalent to programmatically loading a flash/swf file. Or similar to what Rive has built with a .riv, although this is static content, not code.
> 1) Why use WASM?
You're loading arbitrary, third-party code into an app - that is the renderlet. The benefit is to have a sandboxed environment to run code to put data on the GPU.
2) Why use renderlet instead of webGPU, which is already a high level cross platform abstraction including shader definitions?
WebGPU is a low-level API. If you are a graphics programmer, and want to build an app around WebGPU, go for it! A renderlet is more of a graphics plugin system than an entire first-party app.
> The renderlet compiler is currently in closed preview - please contact us for more information.
This is the system to build the renderlet. This is not writing raw C++ code to talk to WebGPU, this can be higher-level functions (build a grid, perform a geometric extrusion, generate a gradient) - you can see in the video it is a yaml specification. The compiler generate the necessary commands, vertex buffers, textures, etc, and soon, shaders to do this, and builds a Wasm module out of it.
> Is this like a 'alternative to webGPU' with a different API / easy mode tooling?
I certainly wouldn't describe it as an alternative to WebGPU, but easy(er) tooling to build graphics, yes.
> What is the use-case for 'I've compiled a part of my application only into a cross platform binary renderlet and I can now run that cross platform ... after I've compiled the rest of my application into a platform specific binary for the platform I'm running it on?'
Let's take an example - Temporal Anti-Aliasing. There are libraries that exist to implement this, or you can implement it through raw code. This requires changes structural changes to your pipeline - to your render targets, additional outputs to your shaders, running additional shaders, etc. Wouldn't it be nice to easy connect a module to your graphics pipeline that contains the code for this, and the shader stages, and works across platforms and graphics APIs, with data-driven configuration? That is the vision.
> ... rest of your application into WASM/platform native code... is that not strange? It seems strange to me
There is not really such a thing as a standalone Wasm application. It has seen great success as a data-driven plugin model. In a browser, it is hosted with / interacts with JavaScript. Even built for pure WASI, as a standalone app where everything is compiled into a single module, there is stil a runtime/host environment.
Interesting! Had never heard of it before, will check it out. The point of Haxe seems to be as a meta-compiler to generate code for a bunch of different languages/compilers? The same spirit of the Wasm dev experience but without the runtime.
Text / fonts is very much on the roadmap! For input and audio I would have to think through the scope.
That's exactly it. With renderlet, the goal is to compile the "frontend" code that's driving the rendering pipeline to WebAssembly, and provide a runtime that embeds that in any app, with the host app providing any configuration necessary to connect renderlet modules and use its canvas.
On the "backend", we will switch fully to wgpu as we retool around wasi-webgpu. I explicitly don't want to rebuild a project like wgpu, and everybody should commit upstream to that - we will likely have stuff to upstream as well.
This is awesome! I'm not fluent with Flutter/Dart but would like to dig in to how the build / Wasm packaging works.
The state of shared memory for Wasm is not great, although raw SharedArrayBuffers work ok in a browser for running multiple guests. Getting multi-memory properly working through llvm is likely a better solution.
We've got a bundle size issue as well even with -O3. I thought it was due to the amount of templated glm simd code we run, but now am convinced its deeper than that into Emscripten. Haven't been able to look into deeply yet.
The renderlet is a bundle of WebAssembly code that handles data flow for graphics objects. Input is just function parameters, output writes serialized data to a specific place in Wasm linear memory. With the Wasm Component Model, in the future can use much more complex types as input and output.
LoadFromFile() - Instantiates the Wasm module
Render() - runs the code in the module, wander uploads the output data to the GPU
Functions on the render tree - do things with the uploaded GPU data - like bind a texture to a slot, or ID3D11DeviceContext::Draw, for example.
There's some nuance about shading. In the current version, the host app is still responsible for attaching a shader, so should be no issue using the data in a deferred shading pipeline. In the future, the renderlet needs to be able to attach its own shaders, in which case it would have to be configured to use a host app's deferred shading pipeline. I think it is possible, but complicated, to build an API for this, where the host and then the renderlet are both involved in a lighting pass.
Of course, if all shading is handled within the renderlet, it entirely the concept of deferred shading, and this becomes an easier problem to solve.
I haven't worked with stereo setups with this codebase yet, but as it is just wrapping underlying platform-specific GPU APIs, it should be a similar performance profile.
On average, running the Wasm guest code is about 80% of the speed of a native build I use. That is both dependent on what is running in Wasm and not a very scientific measurement - wander needs better benchmarks. We think that performance profile is sufficient for anything that needs a GPU except the highest-performance 3D games.
Lots of interesting points in there, and working on Bevy I'm sure you have much more extensive WebGPU expertise than me.
I agree that the feature set around WebGPU is constrained and becoming outdated tech compared to native platforms. It shouldn't have taken this long just to get compute shaders into a browser, but here we are. The lack of programmable mesh pipelines is a barrier for a lot of games, and I know that's just the beginning.
For memory, architecturally, that's why I'm treating wander as a tree of nodes, each containing Wasm functions - everything gets its own stack, and there is a strategy to manage Store sizes in wasmtime. Deleting that is the only way to free memory vs a singular application compiled to Wasm with one stack/heap/etc. More of a data-driven visualization framework than a full engine like Bevy, which I still think is one of the most elegant ways to build browser based games and 3d environments.
> Currently it is the low level, cross platform layer that is the most complex and the biggest hurdle towards making a game engine viable
I couldn't agree more. My goal is not to simply build "a better game engine", but to make this kind of low-level tech accessible at a higher level and with much better dev tools to a broader class of developers and applications
> Don't underestimate what an individual or small teams can produce if they are operating on a solid platform
This gets into my motivations for building a company - larger companies have the resources to build moats, but often can't quickly realign themselves to go after novel technical opportunities. It's not either / or - both models exist for very valid reasons.
To me, this reads like the intersection of "Web Components as Wasm" and "The Browser as an OS" - almost something analogous to WASI as browser APIs that are delivered via Wasm ABI instead of JS/WebIDL. It's an interesting take, and as long as it can operate alongside existing code, I'm all for that.
There are strong parallels to what we're building - small modules of Wasm graphics code that can interoperate across a common interface.
Check the repo for the GPU integration - it's like a super trimmed down version of wgpu, where graphics data is copied out of Wasm linear memory and a host specific API (WebGPU/OpenGL/DirectX) takes care of the upload to the GPU. There is a wasi-webgpu WebAssembly L1 proposal that I am involved with in the works, driven by Mendy Berger, and at some point all of this will be tooled on top of that with wgpu as a backend.
For renderlet the company, the goal is to build developer tools that make it easy to build renderlets and these kinds of applications without having to write raw Wasm code. The meta-compiler in the video is the first step in that direction! The runtime itself will always be open-source.