Xilem: An Architecture for UI in Rust(raphlinus.github.io)

457 pointsby raphlinus4 lata temu103 comments

raphlinus.github.io

https://raphlinus.github.io/rust/gui/2022/05/07/ui-architecture.html

30 comments

Great technical article as always, and apologies for not responding to the meat of the article, but I tend to look at declarative UI with some degree of skepticism. Raph, do you think this general approach can eventually be scaled up to develop complex and highly interactive applications such as DAWs, video editors, and graphic design software? At least based on my experience with SwiftUI, I find the paradigm easy and satisfying to get started with, and great for prototyping and widgets, but quickly run into roadblocks whenever trying to build creative software that's not just a view around a database. Or do you see this as just another tool in the UI toolbox, and not necessarily mutually exclusive with imperative and immediate mode UI? Curious to hear your thoughts!

raphlinus4 lata temu

I think your skepticism is well placed, and I think you are asking the right question.

Here is why I'm hopeful. SwiftUI is as you say wonderful, but is very much a closed ecosystem. You can't really implement your own custom views, rather you can assemble the premade ones in various (cool and interesting!) ways. If you wanted grid layout before iOS 14, you were on your own.

By contrast, in what I'm building, everything is open-ended, and you are invited to build fully custom versions of every piece of the system - change propagation, async resource loading, layout, drawing, animation, everything.

So yes, I am hopeful this approach will give good results for especially those highly intensive applications you mention. And if not, we'll learn something why not. I'm looking forward to trying!

elcritch4 lata temu

I’ll have to checkout your project. I’ve been experimenting with something similar, but in Nim. It’s interesting to devise declarative or immediate mode UIs that make use of compiled languages with good async. Animations become very easy. I think those more complicated UIs may work well with declarative frameworks.

I’m curious to see how you’re handling events. Best of luck!

1: https://github.com/elcritch/fidgetty

elcritch4 lata temu

The node id path is interesting. I disabled it for performance reasons in Fidgetty since it wasn’t used, but was contemplating using it for a theming system similar to CSS. Do you have any ideas of combining declarative style UIs and CSS theme-ing?

archagon4 lata temu

Good luck! I sometimes wonder if it would be valuable to study the taxonomy of interactive software and compare/contrast the kind of architecture and data modeling best suited for each category. Perhaps the framing is too broad, but I often find myself wondering why an approach that clearly works great for my CRUD app can't really be generalized to effectively work with game state, or vice versa. (And part of me wonders if there's some yet-undiscovered paradigm that works remarkably well for almost every use case.)

tayistay4 lata temu

Ok I'm not Raph but, FWIW, my 3d sculpting app is written using SwiftUI (https://sculptura.app), which is not a view around a database (in fact it was previously UIKit... SwiftUI is so much better). I recently implemented a piano-roll editor in SwiftUI: https://github.com/AudioKit/AudioKitUI/tree/main/Sources/Aud.... I have little doubt a DAW, at least, could be implemented in SwiftUI easier than with UIKit or AppKit.

conradev4 lata temu

Have you thought about backing the persistent widget tree with native widgets as the final layer? Or, at least supporting it as a render target. That is, UI/NSViews on Apple platforms, DOM nodes on web, GTK objects on Linux, etc. Essentially using them as the final “render tree”.

SwiftUI takes this approach, as ComponentKit did before it. The framework still does most everything (state updates, layout, animation, etc), but the host compositor handles the actual rendering.

This approach has a few advantages in my opinion. These widgets integrate nicely with the host accessibility system, for one. They also can come with built-in styling to make them “fit into” the target platform.

There are also performance benefits to leveraging the system compositor – Firefox switched to have platform native layers on macOS and it helped immensely with battery life.

It is worth noting that SwiftUI and ComponentKit components don’t have 1:1 mappings with native widgets – they both perform flattening (i.e. for drawing paths and layout only nodes) to optimize their performance.

I think most importantly, though, it allows for rewriting code incrementally as that approach naturally supports bidirectional embedding. Having rewritten the Shortcuts app editor from UIKit into ComponentKit into SwiftUI (maybe one of the larger projects written in SwiftUI?), incremental rewriting is crucial for existing software projects.

The other concern is mobile – it just is not feasible to rewrite the text input stack on mobile, for example, so being able to use native text editing views would very much be necessary.

This all looks fantastic!

raphlinus4 lata temu

I've thought about it. Your "etc" in the first paragraph is doing a lot of work, especially on Windows. The idea that there is a "native" widget set is increasingly a fiction. I agree though that on mac and iOS it makes quite a bit of sense.

A somewhat galaxy-brain approach to this is to make views generic over Cx, and add "factory" methods to Cx for creating button, stack, slider, etc.

All that said, my personal feeling is that while this would give promising early results for creating simple property-sheet like UI, it will be extremely difficult to make polished, truly native-feeling UI in it, as ultimately the seams will show. The first 90% will go well, but the second 90% will be painful.

I don't want to discourage people from trying it though!

pkphilip4 lata temu

Personal opinion but I think the old RAD approach in tools like Delphi where the components are laid out visually using the IDE but having the option of generating the components programmatically during runtime including attaching/detaching event handlers etc with state handled application-wide or within the context of the "form" is a much faster way to develop than to use the declarative approach in tools like Flutter where state management is really complex.

Also, one could develop custom components very easily in Delphi with custom draw events which would draw just the component.

tayistay4 lata temu

I haven't used Delphi, but what you're describing sounds like Interface Builder, which I've used a lot. The biggest problem I had with IB was that it got quite tedious to specify all the constraints to make an app responsive to different screen/window sizes. Plus, you'd have some constraint in there and you wouldn't remember why it was there! IIRC you couldn't add comments. Also you couldn't diff the files because it was a ton of XML and merging them was really scary. I think bigger teams tended to stay away from IB/Storyboards altogether (storyboards were really bad because it was all centralized in one file).

pkphilip4 lata temu

It isn't that difficult to do responsive design in Delphi. This is an older video (2013) which explains the basic concepts: https://www.youtube.com/watch?v=QmbV4rAuZL4

But yes, you can't add comments and yes, the .dfm file which is generated is difficult to diff.

danappelxx4 lata temu

Really cool work. My impression after reading the article is that it’s significantly inspired by SwiftUI, but without the magic annotations (@State, @Binding, @EnvironmentObject, @StateObject, etc.). It will be interesting to see how Rust handles the fully-statically-typed view tree, which has been really pushing the limits of the Swift compiler.

Question for the author: perhaps I missed it, but how do you plan to handle view trees that change based on state (ie SwiftUI’s IfElseView + viewbuilder)?

raphlinus4 lata temu

Good catch! Yes, the plan is to implement _ConditionalContent. It would look something like this:

    if_view(bool_predicate, || view1(...then...), || view2(...else...))

Whether we end up having a proc macro that has similar functionality as ViewBuilder in SwiftUI is an open question. For the time being, I'm seeing how far I can get with just vanilla Rust.

I'm generally pretty hopeful about the ability of the Rust compiler to handle big complex types, but it is a risk. There other projects out there that also stress it, and the compiler team is pretty serious about making this work well.

Barrera4 lata temu

From the first paragraph:

> ... Architectures that work well in other languages generally don’t adapt well to Rust, mostly because they rely on shared mutable state and that is not idiomatic Rust, to put it mildly. ...

The author doesn't mention Redux (the architecture), which is surprising. There are three principles[1]:

1. The global state of your application is stored in an object tree within a single store.

2. The only way to change the state is to emit an action, an object describing what happened.

3. Changes are made with pure functions.

In other words, the components of an application never mutate the state tree directly. Rather, they emit actions which re-generates the state tree without mutation.

This style of state management is compatible with Rust's ownership model.[2] The emphasis on pure functions (that clone state rather than mutate) means that it's not necessary for your application to alias mutable references, which I'm guessing underlies the "generally don't adapt well to Rust" part of the claim.

[1] https://redux.js.org/understanding/thinking-in-redux/three-p...

[2] https://github.com/jaredonline/redux-rs

raphlinus4 lata temu

The other responses have this right. I think the Redux pattern is similar enough to Elm that I didn't feel a need to make a finer distinction.

I'll also say this: the tools that Rust provides for reasoning about mutation are powerful and principled. A central philosophy of Rust is that mutation isn't the problem, it's shared mutable state. If you believe that philosophy (and I do), then restricting yourself to pure functions over clonable state feels like tying one hand behind your back. I hope I've made the case that providing finer grained access to mutable app state is an approach at least worth exploring.

______-_-______4 lata temu

I think Redux is similar enough to Elm that it's not worth mentioning separately. Redux plays fast and loose with types whereas Elm uses types strictly; that's probably why Elm is mentioned more often in the context of Rust.

He does mention Redux in passing if you expand "Advanced topic: comparison with Elm"

mkishi4 lata temu

Apart from the architectural similarity, Elm also predates both Redux and Flux, even being referenced in Redux's Prior Art page [1].

[1]: https://redux.js.org/understanding/history-and-design/prior-...

cultofmetatron4 lata temu

redux architecture is just a global scan(). more or less what the elm architecture is as well.

dwmbt4 lata temu

it's... it's beautiful... wishing Ralph good luck! his blog posts have taught me sooo much in the past. i particularly enjoyed the disclaimer at the end, a lot of these Rust projects aren't production ready but learning about their respective architectures is a great way to passively consume 'academic' paradigms/concepts. if anything, this is what keeps me interested in Rust! i don't have a formal background in CS, so new projects and their inspiration serve as a great gateway into more rigorous study.

edit: the potential for Python bindings is very interesting, it seems to me that Python and Rust are developing a sweet kinship :,)

infogulch4 lata temu

Very interesting! This somehow seems convergent with the model-level incrementalization approach that incr_dom [1] and its successor bonsai [2] are using. Have you had a chance to compare these?

[1]: https://github.com/janestreet/incr_dom | https://www.youtube.com/watch?v=R3xX37RGJKE

[2]: https://github.com/janestreet/bonsai/blob/master/docs/blogs/...

raphlinus4 lata temu

I would say that they were even greater inspirations for the Druid architecture that predated this latest work - we were hoping that a lot of the incremental/reactive patterns could be expressed using combinators over immutable data structures. That didn't work out as well as hoped; I think it's possible to build things that way, but it's also pretty hard going and the community never really reached critical mass.

A semi-explicit goal of this work (that somehow didn't make it into the blog post) is that developing for this architecture, both building the UI components and using them, should be a lot more fun than before. Of course, that's a slippery goal to quantify. We'll just have to see how it goes, but I'm hopeful.

[deleted]4 lata temu

ogoffart4 lata temu

> The problem is that it requires shared mutable access to that state, which is clunky at best in Rust (it requires interior mutability).

I don't see the problem with using interior mutability (Rc<RefCell<T>> or Arc<Mutex<T>>)

With Slint [1], we just embrace it, and rely on interior mutability for the shared state, and that works well.

[1] https://github.com/slint-ui/slint

tayistay4 lata temu

I tried using interior mutability in rui [1] but the clunkiness appeared in having to call clone on the Rc/Arc too often. I'd have a few clones before moving into a closure, like this:

        let text = self.text.clone();
        focus(move |has_focus| {
            let text = text.clone();
            state(TextEditorState::new(), move |state| {
                let text = text.clone();
                let text2 = text.clone();
                let cursor = state.with(|s| s.cursor);
                let state2 = state.clone();

currently looks like this:

    focus(move |has_focus| {
        state(TextEditorState::new, move |state, cx| {
            let cursor = cx[state].cursor;
            canvas(move |cx, rect, vger| {

(Note the context (cx) passed to callbacks to look things up.)

[1] https://github.com/audulus/rui

CyberRabbi4 lata temu

Seems like a large part of the complexity is enabling the creation and use of reusable UI components that work in a variety of UI hierarchies and modify a variety of backend models (or app states), in a type-safe way. Is that right or are there other problems attempting to be solved?

The simple way to do this is a callback system. Why is that not appropriate for Rust? Does it require custom ownership dynamics that the borrow checker does not support?

steveklabnik4 lata temu

Callbacks are often isopmorphic to a big ball of mutable state, which Rust makes very painful.

It’s like the classic Joe Armstrong quite about getting the whole jungle when all you wanted is the banana.

You can do it, and if you check out the Gtk/QT bindings you’ll see the boilerplate it introduces. Not insurmountable but many people are interested in figuring out if there’s a better way.

nicoburns4 lata temu

I’m sure I’m being naive, but could this be worked around by passing a mutable reference to the state into the callback rather than it closing over the state? Assuming there’s only one UI thread, then only one callback can run at once anyway…

gpm4 lata temu

I think the fundamental problem with that approach is that even in single threaded code rust makes it illegal to have to have two pointers where at least one is mutable to the same state. So I can pass in a `&mut State` pointer, but then I can't also pass in a `&mut AnElementInSomeListInState` pointer. Nor can I really have `AnElementInSomeListInState` just have a parent pointer to the rest of the state, because someone has to have a `&mut State` pointer, and they would conflict (also because doubly linked lists are hard in rust).

GolDDranks4 lata temu

The problem is that multiple callbacks might require a reference to the same state, and closing over those references and retaining that callbacks, makes two mutable references to the same state, which Rust makes an illegal pattern. That is indeed safe in single-threaded code, but Rust prevents it nevertheless; there is a famous (in Rust circles) blog post about this: https://manishearth.github.io/blog/2015/05/17/the-problem-wi...

There is a workaround called "internal mutability", an ability to mutate state pointed by a shared pointer. That is syntactically slightly messier, and generally frowned upon. The blog post mentions about this workaround, and also states that it's not ideal so we should strive for better.

_dh544 lata temu

> There is a workaround called "internal mutability", an ability to mutate state pointed by a shared pointer. That is syntactically slightly messier, and generally frowned upon.

I don’t see why this would be frowned upon. It seems to be a runtime version of the borrow checker. For sufficiently dynamic code I don’t see how you can get around checking mutability access at runtime (or asserting that multiple blocks of code aren’t concurrently requesting a mutable reference).

As a comparison, there are many cases where the compiler can prove that a bounds check is unnecessary for accessing an element in a vector but there are also many useful cases where it’s simply not possible to do at compile time. It would be silly to frown upon runtime bounds checks when the requested index or the size of the vector is not known at compile time, a common occurrence in many interesting programs.

I get the motivation to make APIs as statically checkable as possible but it doesn’t seem to always be practical. Reusable UI components can be used in a variety of contexts, e.g. situations with multiple callbacks for different backends. The information is just not always there at compile time.

jcelerier4 lata temu

As someone who had literal paid gigs in c++ where I had to remove all sorts of reference counts to reach performance targets, it's depressing that there doesn't seem to be a better way in rust

CyberRabbi4 lata temu

The way to do this in Rust is to use “unsafe.” It essentially means “trust the programmer that this pointer is live.”

I don’t think there is a way around this. You’re dealing with a high-level runtime condition. There are no tractable ways to get compilers to understand these higher level runtime conditions, so conditions must be reproven at runtime. It needs an oracle. That oracle should be you but you’ve decided that you cannot be trusted. This appears to be somewhat of a contradiction. Should you even be writing this program in the first place?

GolDDranks4 lata temu

The reason it's frowned upon is that it makes the code not thread safe. This is fine in application code, but if you are a library author, you generally want your code to be as flexible as possible, and able to be used in various situations. Making the code not thread safe brings upon some limitations how the code can be used by your users. This is why using internal mutation sparingly is the default.

CyberRabbi4 lata temu

I’m not clear on what you mean it’s not thread safe. Arc<> is thread safe.

[deleted]4 lata temu

stormbrew4 lata temu

Borrow checking gets a lot more complicated with closures that live past their scope. It’s usually much more frustrating than it’s worth.

tcmart144 lata temu

I think I just ran into this issue about a week ago. I started working on a task manager written in Rust using the Cursive library, which provides like an ncurses TUI. Heavy use of callbacks, but all callbacks require a <'static> lifetime. All the structures I make for information on the database of course don't have <'static> lifetimes. I eventually figured out that cursive has a function where you can kind of give it the data, but to pull the data back out requires a lot of cloning and boiler plate code.

LegionMammal9784 lata temu

Isn't the standard solution to pass the data within an Rc or Arc? That way, the closure still owns all of its data.

tcmart144 lata temu

I'll need to play around with that. I just started doing a serious dive into Rust about a month ago.

pornel4 lata temu

Tip for you: when the compiler says you need a 'static lifetime, it's actually trying to say that all temporary references are forbidden and won't work. Arc is a reference, but isn't temporary. Add a mutex or atomic when you need to modify data behind arc.

This is the "interior mutability" approach that the article mentions.

stavros4 lata temu

This is very useful, so useful that I can't help but think that the compiler actually should say what you said.

stormbrew4 lata temu

I definitely think the error could be improved but I'm not sure it should really say "just shove it in an `Arc<Mutex>`," because that really is only sometimes the right solution and it might lead people to make poor choices by default.

I think a lot of the problem is really with the confusingness of the concept of static lifetime in rust to begin with, where it's kinda used for both "lives forever" and "doesn't reference anything that lives longer than it"[1]. I hope someday these meanings get different names, tbh.

But when you see that error the naive thought is like... "ok better make it live forever" but it really just means you need to make it something that you control, and there are other ways to do that than a refcounting mutex.

[1] https://doc.rust-lang.org/rust-by-example/scope/lifetime/sta...

stavros4 lata temu

That's very useful, thanks! I need to learn more Rust.

stavros4 lata temu

That's very useful, thanks!

viraptor4 lata temu

Try raising an issue on GitHub. Rust people are interested in improvements like that. I'm not saying it will be accepted, but it won't be ignored.

thurn4 lata temu

Animation is usually the biggest pain point with frameworks like this -- and it usually feels like a complete afterthought for framework designers. Of course you always have the simple CSS-style stuff (here's a list of properties you can attach transitions to) but as soon as you get into anything more complicated it all falls apart.

grayrest4 lata temu

> Animation is usually the biggest pain point with frameworks like this -- and it usually feels like a complete afterthought for framework designers.

It tends to be. The only (popular) JS framework where it's a first class concept is with Svelte. There's a basic set of transitions and the framework handles the post-out-transition removal (which is annoying in a lot of frameworks) but there's also custom CSS animations where a JS-defined curve is rendered into CSS [1], semi-automated FLIP animations for non-transitioning nodes [2], and deferred transitions where the outro of one element is sync'd with the intro of another [3]. There's a few other things like springs and whatnot but I consider the package as a whole to be a significant advancement in the state of the art.

[1] https://svelte.dev/tutorial/custom-css-transitions [2] https://svelte.dev/tutorial/animate [3] https://svelte.dev/tutorial/deferred-transitions

The only other framework I know of that's tackled it in core is Inferno but I haven't read through their implementation in detail.

IshKebab4 lata temu

Flutter is also very good at animation. I suspect that a significant part of the motivation for Flutter was Android developers getting frustrated with the repeated unsatisfactory attempts at an animation API for Java Android. I think there are like 4 different animation APIs in Java Android now.

ink4044 lata temu

The compose APIs for animation are quite nice on Android now

chriskrycho4 lata temu

CSS transitions are fairly restricted, but animation in CSS is substantially more capable than transitions if you’re using the animations API instead: https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Animati...

infogulch4 lata temu

CSS is pretty flexible. What kind of animations would you expect to be difficult to express?

webmaven4 lata temu

> CSS is pretty flexible. What kind of animations would you expect to be difficult to express?

Interpolations of anything beyond basic properties such as size, orientation, opacity, color, stroke, and fill; vector shapes, particle systems, and generative textures come to mind.

There are intermediate possibilities. In theory you could "compile" the animation of a skeletal system with inverse kinematics to CSS, but I'm not aware of anyone going to the trouble.

infogulch4 lata temu

If you're going all the way to skeletons, kinematics, particle systems, etc maybe you just need a game renderer. Say, Bevy, if you want a rust flavor.

webmaven4 lata temu

No need to go that far. If you're just looking for the edge of where the capabilities of CSS animation fails, consider the pseudo-physics of various direct manipulations in touch UIs: eg. the bounce when you swipe to scroll and reach the last item, or swipe down again to refresh, or swipe to fling cards aside left or right, or pinch to zoom past the last zoom level, etc.

In some cases you could get a good-enough result by animating from arbitrary (ie. user controlled) starting point to one or more fixed end-states (eg. Three "flick right" animations, one each for flick up and to the right, right, or down and to the right), but at that point you might as well just use an animation library.

webmaven4 lata temu

To restate things from a bit more mathematical perspective, interpolating position (even with control over easing etc.) only gets you so far. Ideally you would be able to set and interpolate higher derivatives of position over time (speed, acceleration, etc.) as well, without resorting to JS.

wtetzner4 lata temu

How important are animations in a UI? I typically find them annoying (because you have to wait for them).

viraptor4 lata temu

The basic stuff is incredibly important, but there's also lots of fluff in many applications which is why you find it annoying.

My two go-to examples:

- The shopping app I use has some kind of points system and some items give you "boost". I hate the concept and don't even care what it does because adding those items animates a bar filling up and take extra 4s while I'm trying to do this as quickly as possible. This is a terrible use of animations.

- Any app where you can rearrange a list needs an animation for when you hold an item and try to move it. Specifically, other entries shouldn't instantaneously jump around when you cross some threshold. There's lots of subtle ways to achieve this, but one of my favorites is greying out the old space and springing items up/down as you move the chosen one around. This animation is almost required, because the experience is jarring otherwise.

wtetzner4 lata temu

But this feels like something that can be handled with instantaneous state changes. I feel like pretty much any GUI implementation could handle it. I assumed the kind of animation support referred to was timed animations, which need multiple frames to complete.

chriskrycho4 lata temu

Poor animation design has exactly the problem you describe. Animation in general is a very useful affordance, though: it allows you to convey the relationships between the various nodes and modes in your UI. That ranges from being able to do more than hard toggles on colors of buttons for different modes to actually being able to do push/pop on “stacks” within the UI, which is a concept mobile UIs in particular make heavy use of. Overuse of animation can be an accessibility problem, but having no animations or transitions can actually also be an accessibility problem because it can make it hard for people to build a mental model of what happened when navigation occurred.

berkes4 lata temu

I'd go so far as to state that anyone who 'hates animations' uses them daily.

The ones you don't notice are most often good - yet they are there. You'll notice degrading UX when they aren't. The ones you notice are quite likely bad, but that doesn't make the majority of aniation bad.

Anything, from a blinking cursor, to the highlight effect on your smartphone keyboard to the check-uncheck of a checkbox is animated. Subtle, but crucial.

wtetzner4 lata temu

I don't think these are the kinds of animations being referred to. They're instantaneous state changes. E.g., for the highlight effect you can just change the key image on press. It doesn't need multiple frames of animation.

So while it is technically an animation, I don't think it really requires any special GUI toolkit support to implement.

ativzzz4 lata temu

Simple sliding animations for going back/forward in mobile screens add so much more confirmation to what exactly is happening when you click an element. Or when you tap a button. That feedback you get when you interact with an element is often some kind of animation.

golergka4 lata temu

In game development, extremely important. And to increase complexity, they can also “delay” state updates from the actual state to what is presented to the user. The player's gold purse might already have that 10gp, but the gold counter at the top should only be updated when the gold icon completes it flight from the center of the screen. Also, this icon? It was originally rendered in the game layer underneath the ui, and has to change layers (which usually have very different layout algorithms) and un-chain itself from the camera movement during this animation, completely unnoticed by the player.

pjmlp4 lata temu

In modern UIs, even screen transitions rely on animations.

wtetzner4 lata temu

Sure, but I guess my point was, do they really need to? How much does it add?

I'm not necessarily saying they don't matter, more that I'd like to find out how much they actually matter, vs how much people think they matter.

thatguy_4 lata temu

Really interesting post.

One thing I think might be problematic for you would be fallibility in the `Adapt` callback process. i.e. if it would not be sound to change the state of a child due to some emerging app state inconsistency or similar.

To put it differently, I understand your architecture outputs a new _statically typed_ tree every time through kind of recursive state mutation. However, being statically typed, it is clear from the start of the operation what the end complete static type will be (to the compiler at least). If the transformation of one of the children is not possible (although the rest might be fine), what do you do?

The obvious way would be to make each conversion fallible, but this might be a pain to use/propagate. Otherwise, you might have state that all operations involved in state mutation must be infallible.

Anyway just my 2c. I might have misunderstood the mechanism, though.

Thank you for all your impressive work!

chaosprint4 lata temu

Very insightful post. Although my work is mainly about audio not GUI, I still learn a lot from your idea. For my audio work, I also use declarative style and diff algorithm for updating the audio graph. But when rewriting it, I found sending messages is also very convenient:

https://github.com/chaosprint/glicol/tree/main/rs/synth

This might be interesting for you as I found that you also have a synth project (https://github.com/raphlinus/synthesizer-io). Admittedly, there is still some way to go for this audio lib. Will further study this post when I got more spare time.

melony4 lata temu

What are your thoughts on Sycamore? It uses Svelte-style compile-time reactivity.

https://github.com/sycamore-rs/sycamore

raphlinus4 lata temu

To be honest, I haven't looked too deeply into Sycamore. A lot of the reactive machinery looks pretty similar to Dioxus (threading a context scope, using explicit observable objects for change propagation). I think it's worth comparing more carefully, and would be more than happy to link a writeup to such a comparison.

rektide4 lata temu

Author Raphlinus has an incredible history of great, superb technical posts here, many which have spawned great discussions[1].

Referenced in this article are: Xi-Editor, discussed in Xi-Editor Retrospective[2] and Druid, discussed in Rust 2021: GUI[3], which follows closely after Principled Reactive UI[4] (describing a prototype for Druid called Crochet).

> I have long believed that it is possible to find an architecture for UI well suited to implementation in Rust, but my previous attempts (including the current Druid architecture) have all been flawed.

These have all been very very good & technical posts on what toolkits really support & enable ui (amid other great technical topics too). It's delightful seeing such an ongoing continuation, an evolcing refinememt of ideas & self-review from someone of such expert caliber.

Rarely do we get such an intimate view into what the real hunt for rightness is, see how we ever hunt for perfection. Personally I believe that hunt for better is one of the undertold aspects of hackerdom, a less visible less knowable reciprocal to fast-and-dirty. Tapping & enabling this creative, knowledge & intellect based capability is a core spring from where greatness emerges.

> I have studied a range of other Rust UI projects and don’t feel that any of those have suitable architecture either.

It's also notable how widely raphlinus travels to get the best persepctive available. This post begins with a a vast field survey of other ui libraries & their origins. I lack the energy to search down & link HN discussions on each of these, for there are many! But seeing how everyone else is doing, looking wide & far to explore their peer's attempts, is also a notable characteristic I admire here.

[1] https://news.ycombinator.com/from?site=raphlinus.github.io

[2] https://raphlinus.github.io/xi/2020/06/27/xi-retrospective.h... https://news.ycombinator.com/item?id=23663878 (538 points, 26 months ago, 157 points)

[3] https://raphlinus.github.io/rust/druid/2020/09/28/rust-2021.... https://news.ycombinator.com/item?id=24631611 (374 points, 31 months ago, 244 comments

[4] https://raphlinus.github.io/rust/druid/2020/09/25/principled... https://news.ycombinator.com/item?id=24599560 (234 points, 31 months ago, 97 comments)

Klasiaster4 lata temu

Would be great to eventually have different backends for this, like Java had where the app looked native in Windows, Gtk, etc. One could even think of having a TUI backend besides an HTML or Canvas backend.

That would be restricting of course when one needs features available in only one backend and developers could opt-in for more control and require a certain backend like Gtk or not require but detect the backend and get extra features.

berkut4 lata temu

I've never persevered with immediate-mode UIs deeply enough to get to the point I needed to solve this, given I mostly deal with complex nested UIs and in my (possibly limited/incomplete?) experience, immediate-like UIs don't really work well for that type of complex and dynamic setup, but it sounds like the Widget tree persists in this model (at least more than the other trees), but it's not clear if it's update-able as well? (there is mention of it being rebuild-able, so I guess so?).

I wonder what happens regarding state (say, selection state, or visibility/enabled state) in the case where you might want to allow the user to re-arrange entire UI components (say the user draging a nested tab/pane from one window of the app to another, and docking it into another different hierarchy): would the trees have to be completely re-built (I guess sub treelets could still be kept?) along with re-building the id paths. Would that mean diffing is hard/impossible in some cases to transfer across a large re-build of these trees?

raphlinus4 lata temu

First question is easy: yes, the widget tree can be updated. That's generally done by diffing data stored in view nodes, but in fact the View trait is open-ended and you can implement the rebuild method to do anything you like. And yes, selection state lives in widgets and it is absolutely a goal to have that persist.

The second question is more challenging (see the "advanced topic" under identity for a little more background). View id's cannot be re-parented, in other words when a parent relationship is expressed in an id path (by virtue of having the child id follow the parent in a path), that relationship cannot be changed. However, widget ids and view ids are not necessarily the same, though they can be. I think what's needed for your use case is a level of indirection so the view id paths remain stable, but the widget id structure relationships can be changed. I haven't worked out all the details, but think it can be done, and if it's done right it wouldn't require any rebuilding of widget subtrees.

berkut4 lata temu

Thanks for the info!

flohofwoe4 lata temu

Obviously Raph knows what he's doing, so this isn't a critique, but a question:

What's the reason for replicating the "React idea" outside the web browser?

The way I understand the motivation behind React is that it tries to work around the too high level and too rigid DOM by mapping one way to describe an UI (the React API) to another (the DOM), for the only reason that there's no realistic way to bypass the DOM (except doing everything yourself - including fundamentals like text rendering - via a 2D- or WebGL canvas).

But if you don't have something as rigid as the DOM as lowest layer to begin with, what's the point of building a React-style system with 'tree diffing' against data that persists across frames? Is it really worth the complexity managing intermediate data that sits between the UI description that (most likely) needs to be updated each frame, and the low-level rendering instruction stream - which most likely also needs to be generated each frame?

Or is it all about Rust's language restrictions?

loxs4 lata temu

As someone who writes exclusively Rust on the backend and React on the frontend... To me React is a "language", not the technique of the "tree-diffing". And I think it's the best one there is for expressing an UI. The tree-diffing (and the speed deterioration) is only the (unfortunate) way to compile it to the DOM. Still worth it IMO. With Rust it can probably be done both safely and imperatively (i.e., fast) while retaining the declarative language.

Haven't really thought it over fully, but so far it sounds really nice.

mikevm4 lata temu

What about various libraries that don't use a virtual DOM and have vastly better performance than React, for example https://www.solidjs.com or Svelte?

loxs4 lata temu

Svelte is nice, although a lot less ergonomic than React as it is not "native JS" which creates the need for "two domains" of programming - one, the Svelte domain (which is later compiled) and one JS domain which is called as a library. Haven't tried SolidJS yet.

Cannot really say if Svelte is worse than React, as I have much more experience with React itself. My main blocker to adopt it (Svelte) is mostly library support - with React one can use "everything" in all kinds of "hackish" ways and with TypeScript, which wasn't really the case with Svelte last I checked, maybe 2 years ago.

miohtama4 lata temu

Svelte might not be internally as ergonomic as React, but the fact that it ditches “everything needs to be native JS” principle, allows one to build components with much less boilerplate. This translates to more readable code, less boilerplate and lines to type. Effectively Svelte sacrifices some of the purism principles in the favour of better developer productivity.

The trick with Svelte is that “two domains” are very, very, close to each other. A JavaScript programmer can learn and understand Svelte after 2 days tutorials.

Library support is something that can be improved only over time.

loxs4 lata temu

Yeah, I agree, but I suspect that React is still the "right" tool for 80% of the software out there (having all of these things in mind). Though I should give Svelte another go soon maybe :)

kuon4 lata temu

I am currently designing an UI framework in rust and I took a similar approach. It is intended for medical monitoring systems. I should be able zo open source it at some point. But for now, all I can say is that this approach seems the way to go.

webmaven4 lata temu

Some of your previous explorations were motivated (IIRC) by UIs that had (potentially) many thousands of objects for interactive visualizations of datasets. Is that still a motivating example for this evolved approach?

raphlinus4 lata temu

Yes, most definitely. My main work these days is on supporting high performance drawing of such large data sets on GPU. It's not entirely wrong to say that my continued explorations into UI architecture are motivated by wanting to drive the renderer with really high-bandwidth input.

webmaven4 lata temu

Okay. That said, are there any particular conclusions or implications from this approach for that use case?

For that matter, are you next going to extend this simple exploration toward more complexity (eg. more widget types, more interactions, more layout constraints, etc.) or toward more data?

As I've followed your efforts in this area, the power and utility of selecting the right motivating example has been coming into focus. The parallels with a startup's MVP are fairly clear, the parallels to selecting a (dissertation) research topic a bit handwavy (in part because guidelines for the latter are extraordinarily vague). So far you haven't really chosen a new motivating end-user application since suspending development of Xi. There are lots of directions you could go, from making an equivalent to Processing, to creating a game engine, or eventually restarting Xi development with an eye toward exploration and visualization of large codebases, etc. Personally I'm starting to become intrigued by the need to dig into large neural nets and the representations encoded within (related subtopics: latent spaces, feature vectors, orthogonality):

https://distill.pub/2019/activation-atlas/

Meanwhile, I've been casting about for a proper (Rust) motivating example app/library of my own, and the closest I've come so far is something like a HTML + CSS rendering engine, though not targeted at the Web per-se, but instead at ebooks (esp. EPUB), a much smaller set of required functionality.

In theory a desktop ebook library + reader app with features like MathML support, advanced typography, and visualizing connections between millions of works and annotations seems like it would be a better fit with the direction you're (currently) going than any existing framework.

raphlinus4 lata temu

These are very good questions. I'm not 100% sure where this will go. Your comments are helpful in helping define the contours of the space to explore.

webmaven4 lata temu

Glad to help, in however small a way. Perhaps a poll, survey, or even a DX study to gather feedback more broadly might be appropriate at this stage?

______-_-______4 lata temu

Big fan of your work, Raph.

One small typo:

> {anonymous function of type FnMut(u32) -> ()}

It looks like the param type should be `&mut u32`. And in that simple case the whole thing could probably just be `fn(&mut u32)` since the closure doesn't capture any locals.

raphlinus4 lata temu

Heh yes, somebody else caught that. I think the current state is ok. This closure won't capture any locals, but in general closures in the view tree will. I'll take a PR if you think it should be improved further :)

eximius4 lata temu

I think the answer to this is no, but I'll ask anyway.

Is there any concern with the allocation for the view/widget tree every cycle? I know it's retained mode so the most important resources (graphics) are cached between cycles, but I'm wondering if the trees ought to be allocated out of arenas or something so that the allocating/freeing every cycle isn't undue performance penalty.

(I'm not sure what existing regained frameworks do.)

yencabulator4 lata temu

As someone who has written some 10,000 lines of Rust, if this comma is intentional this is just frankly unreadable:

    adapt(
        |data: &mut Arc<Parent>, thunk| {
            let mut child = data.child.clone();
            thunk.call(&mut child),  // <-- HERE
            if !Arc::ptr_eq(&data.child, &child) {
                Arc::make_mut(data).child = child;
            }
        },
        child_view(...) // which has Arc<Child> as its app data type
    )

Maybe it was meant to be a semicolon?

patrickberkeley4 lata temu

Related and possibly helpful:

* How Glimmer.js implemented autotracking: https://www.pzuraq.com/blog/how-autotracking-works

* Discussion specifically on Lamport Clocks and autotracking: https://v5.chriskrycho.com/journal/autotracking-elegant-dx-v...

adamnemecek4 lata temu

Is this inspired by adaptron by any chance? If no, do you have an opinion on said work?

raphlinus4 lata temu

Adapton is definitely an inspiration, and I have cited it in previous iterations (and even had it in an earlier draft - if you check the Markdown source, the link def is still there).

Here's my current thinking on the topic. It all depends on whether you want to model your problem as a tree or a graph (see also [1] for some great discussion of that tension). If you model your problem as a graph, then a general purpose incremental computation engine like Adapton (or Incremental or Salsa) is great. However, if your problem is a tree, then constructing the explicit dependency graph requires a nontrivial amount of ceremony. What Xilem does is represent the most common tree-structured flows of information in a very lightweight manner (mostly just plain code that builds views), while allowing you to insert arbitrary graph edges if you like with more work.

I think it's a discussion worth continuing. One thing you could do is integrate Adapton and Xilem together, where the implementation of most view nodes in the latter becomes queries into the Adapton engine. How well would that work? Really only one way to find out.

[1]: https://glazkov.com/2022/02/06/tension-between-graphs-and-tr...

infogulch4 lata temu

What happens if two child components need mutable access to the same data? Say, a group of dropdowns to filter/sort a table plus a pie chart with clickable slices that also change the filtering.

raphlinus4 lata temu

If two components generate events in the same cycle, they get queued, and the corresponding callbacks are run sequentially, each time with a mutable borrow of the app state. This seems like the most correct and ergonomic approach to me.

infogulch4 lata temu

That makes sense.

[deleted]4 lata temu

AbuAssar4 lata temu

kudos for explaining the meaning of the name, alot of projects just puck a random name and call it a day.

> The name “Xilem” is derived from xylem, a type of transport tissue in vascular plants, including trees. The word is spelled with an “i” in several languages including Romanian and Malay, and is a reference to xi-editor, a starting place for explorations into UI in Rust (now on hold).

[deleted]4 lata temu

aaaaaaaaaaab4 lata temu

Another UI framework that re-renders everything each cycle? Yeah, it's gonna be awesome for todo lists (as long as you don't have more than ~100 todo items). Meanwhile high-performance apps will still be written in retained-mode UI toolkits.

eximius4 lata temu

This is targeting retained mode.

aaaaaaaaaaab4 lata temu

Ummmm... no?

"As is completely standard in declarative UI, it is done by diffing the old view tree against the new one, in this case calling the rebuild method on the View trait. This method compares the data, updates the associated widget if there are any changes, and also traverses into children. The view tree is retained just long enough to do event propagation and to be diffed against the next iteration of the view tree, at which point it is dropped. At any point in time, there are at most two copies of the view tree in existence."

eximius4 lata temu

Retained mode is distinct from immediate mode where it _retains_ what is _drawn on the screen_, only redrawing visual components that have changed. Immediate mode redraws the entire application each time, replacing all pixels it's responsible for even if nothing has changed.

Throwing away some internal state is not relevant to this distinction.

aaaaaaaaaaab4 lata temu

What? No, that’s not the definition of retained mode graphics…

https://en.m.wikipedia.org/wiki/Retained_mode

coryvirokmobile4 lata temu

Apologies for the noob question, but why reinvent the wheel when we have HTML/DOM/CSS?

chriskrycho4 lata temu

Besides the (correct!) sibling comment: the idea here is to provide underlying primitives which can then be used to express UI in a variety of contexts, first of all native UIs but (as the post notes at the end) also including the DOM. For example, people have implemented experiments which use SwiftUI for authoring HTML, and you can use React to author native UI (React Native, but also e.g. the Raycast widget system), text UIs, etc.

__jem4 lata temu

So that you don't have to ship an entire web browser for your application.

[deleted]4 lata temu