The Linux Graphics Stack(blog.mecheye.net)
blog.mecheye.net
The Linux Graphics Stack
http://blog.mecheye.net/2012/06/the-linux-graphics-stack/
8 comments
I can understand that Wayland does not want to pay for the complexity of network transparency, but I'd be sorry to see it go entirely. I'm one of the few people who do make use if X11's network transparency on a regular basis. Being able to open a web browser on a machine that is several ssh hops away is awfully convenient. I hope that Wayland has some provisions for reproducing or emulating X11 even if it isn't built into the core.
This may be incorrect; I have only vague familiarity with the subject. I don't know if there are any non VNC-like solutions being worked on.
Wayland doesn't have network transparency as a first-class citizen, but it's easy to fit it into the mix. A bit of background:
In X11, the client sends the actual drawing commands (draw a line here, a smooth gradient here, a pixmap here, a curved line here, etc.) over a network pipe. The X server on the other end takes those commands and builds the resulting image.
In Wayland, your program just transfers (well, using shared memory) complete images of the entire window to the Wayland server.
The X11 solution is absolutely excellent and very efficient assuming your programs primarily use the X11 primitives. Most modern programs and toolkits do not use the drawing primitives; they just build the images they want and send a bunch of pixmaps to the X server. They do this primarily because X11 doesn't have a number of useful drawing primitives, like anti-aliased text. The result is that you're doing more or less what Wayland intends to do. (Though sending things like Xclock or Xeyes will be less efficient under Wayland, seeing as they use the X11 drawing primitives.)
In the end, Wayland will probably have something very much like per-window VNC. Your program will produce an image, some stub Wayland server will send that image over the network, and a Wayland compositor on the other end will receive it and display it.
Wayland doesn't have network transparency as a first-class citizen, but it's easy to fit it into the mix. A bit of background:
In X11, the client sends the actual drawing commands (draw a line here, a smooth gradient here, a pixmap here, a curved line here, etc.) over a network pipe. The X server on the other end takes those commands and builds the resulting image.
In Wayland, your program just transfers (well, using shared memory) complete images of the entire window to the Wayland server.
The X11 solution is absolutely excellent and very efficient assuming your programs primarily use the X11 primitives. Most modern programs and toolkits do not use the drawing primitives; they just build the images they want and send a bunch of pixmaps to the X server. They do this primarily because X11 doesn't have a number of useful drawing primitives, like anti-aliased text. The result is that you're doing more or less what Wayland intends to do. (Though sending things like Xclock or Xeyes will be less efficient under Wayland, seeing as they use the X11 drawing primitives.)
In the end, Wayland will probably have something very much like per-window VNC. Your program will produce an image, some stub Wayland server will send that image over the network, and a Wayland compositor on the other end will receive it and display it.
I used to as well, but I discovered that VNC completely beats X forwarding in terms of responsiveness, reliability, speed, clipboard management and OpenGL usability. Modern apps simply draw so much so complex things that the recipe is heavier than the effect.
The philosophy is different, I admit, and it needs certain setup, but it is totally worth it.
The philosophy is different, I admit, and it needs certain setup, but it is totally worth it.
Yes, I use that all the time too. There are certainly alternatives around for getting a remote window displayed locally but the simplicity of "ssh -X" is hard to beat.
That would be XWayland, essentialy running X on Wayland the way you'd run it on Windows or MacOS X now, which should go a long way towards easing the transition.
As much as X may suck, it's actually faster for me to start LibreOffice through ssh -X over Wifi on my Linux PC than running it on my MacBook. That's quite good for an obsolete technology.
And while I've never actually needed to do this, it's comforting to know it's there.
I'm doing this at the very moment.
It's planned for the future, but it isn't done now.
I wish that unoccluded or maybe just the frontmost window could be unredirected too, even if it's not fullscreen. The decorations and shadow could still be drawn in the Compositor Overlay Window.
I work on Linux on a program which is entirely drawn by OpenGL (I target ARM devices, but I test on the desktop before doing an ARM build). Application GL performance suffers immensely under a compositor, so on most machines I can't use GNOME3 (though I would like to). On machines where I get acceptable GL performance in my app, vsync is broken so there is constant tearing.
I'm sure there are some caveats with unredirecting the frontmost window -- like when it becomes occluded it would have to be redirected and that requires a round trip to the client (which might hang, etc), but there are probably other situations like this which the window manager has to deal with already (resize timers?) so maybe there is a way to implement it...
I work on Linux on a program which is entirely drawn by OpenGL (I target ARM devices, but I test on the desktop before doing an ARM build). Application GL performance suffers immensely under a compositor, so on most machines I can't use GNOME3 (though I would like to). On machines where I get acceptable GL performance in my app, vsync is broken so there is constant tearing.
I'm sure there are some caveats with unredirecting the frontmost window -- like when it becomes occluded it would have to be redirected and that requires a round trip to the client (which might hang, etc), but there are probably other situations like this which the window manager has to deal with already (resize timers?) so maybe there is a way to implement it...
I wish I could see this kind of discussion in the wayland/name-your-compositor mailing lists. Currently it would be easy to implement a new attribute in the ICCCM so that applications that don't need foreground unredirection (ie: they actively want to be transparent, which is a minority) can signal so, but it looks like that with the unification of the window manager and compositor we are losing this kind of extendability.
I do realtime graphics with GL for a living, and I'm really sad at the state of the linux desktop, but especially worried at the future. GL performance with any modern distribution is worrysome due to the compositors, and removing it is the first thing that I need to tell any customer lamenting performance issues. Being unable to set vsync on a per-window/context basis is really a major problem, and the performance hit that you get is unacceptable. It seems that removing vsync is all hip these days for games and toys, yet tearing is unacceptable in any other context when you can actually get triple buffering with modern hardware.
Why are we optimizing for toys, when what matters are actual applications and everyday's performance? People don't seem to realize that while the GPU can be used as a general purpose computing unit, the usage pattern is vastly different. The process handling the screen often must have either higher or total control over it many times. You immediately notice video stutter and frames lagging.
But what do I know... I do the same with audio trying to squeeze decent latency with PortAudio and/or alsa directly, while people go through the pulseaudio pipeline...
I do realtime graphics with GL for a living, and I'm really sad at the state of the linux desktop, but especially worried at the future. GL performance with any modern distribution is worrysome due to the compositors, and removing it is the first thing that I need to tell any customer lamenting performance issues. Being unable to set vsync on a per-window/context basis is really a major problem, and the performance hit that you get is unacceptable. It seems that removing vsync is all hip these days for games and toys, yet tearing is unacceptable in any other context when you can actually get triple buffering with modern hardware.
Why are we optimizing for toys, when what matters are actual applications and everyday's performance? People don't seem to realize that while the GPU can be used as a general purpose computing unit, the usage pattern is vastly different. The process handling the screen often must have either higher or total control over it many times. You immediately notice video stutter and frames lagging.
But what do I know... I do the same with audio trying to squeeze decent latency with PortAudio and/or alsa directly, while people go through the pulseaudio pipeline...
Part of the problem is also that the compositor often can't know if the frontmost window (or any window, for that matter) is 100% opaque. If your window has partial transparency, or (for example) implements non-rectangularity by using an ARGB window and setting the alpha for some pixels to 0, then unredirecting it will break that.
I suppose you could just restrict the optimization to windows with an RGB visual, though. But then you need to make sure there isn't something else modifying the opacity of the window. I've written a standalone compositor before (that is, a compositor that runs independently of the window manager, as a separate process), so you can't always assume that the compositor "knows best"; for example, the WM may want to make the window decorations semi-transparent. Or maybe the user has specified slight transparency for all their windows. Or something.
Hmm... in the end, though, you could probably come up with a heuristic that works in most cases.
I suppose you could just restrict the optimization to windows with an RGB visual, though. But then you need to make sure there isn't something else modifying the opacity of the window. I've written a standalone compositor before (that is, a compositor that runs independently of the window manager, as a separate process), so you can't always assume that the compositor "knows best"; for example, the WM may want to make the window decorations semi-transparent. Or maybe the user has specified slight transparency for all their windows. Or something.
Hmm... in the end, though, you could probably come up with a heuristic that works in most cases.
I think that Mac OS X unredirects unoccluded windows; I've noticed windowed computer games suddenly getting much slower when I move another window in front of them.
It appears to, or has some other trick... My coworker has a huge monitor, and his MBA doesn't hit 60 FPS on any of the fullscreen animations, but individual window contents can go at 60 making me think they don't repaint the world every time a window changes.
Absolutely fascinating.
Some questions...
Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol?
Is anyone working to create a non-reference optimized implementation of libwayland?
Have the Wayland developers said anything about supporting multiple monitors yet?
Some questions...
Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol?
Is anyone working to create a non-reference optimized implementation of libwayland?
Have the Wayland developers said anything about supporting multiple monitors yet?
> Would this mean that window managers themselves would have to be re-implemented in order to support the Wayland protocol?
Yes, an a lot more than that will have to be partially or fully rewritten. Thankfully, many if not most desktop apps use a widget toolkit like GTK+ or Qt for UI, so it's enough to make the widget toolkits work on Wayland.
Yes, an a lot more than that will have to be partially or fully rewritten. Thankfully, many if not most desktop apps use a widget toolkit like GTK+ or Qt for UI, so it's enough to make the widget toolkits work on Wayland.
Yep, the Wayland developers' solution to window management is for anyone who wants a different window manager to replace Wayland with a different compositor+window manager speaking the same protocol. Their suggested way of supporting proprietary graphics drivers is also replacing the compositor+window manager with one specific to that hardware which speaks the Wayland protocol, sort of like in the early days of X when there was a seperate X server binary for every piece of hardware. So your window management choices under Wayland depend on which graphics drivers you're using.
Not quite. Wayland itself doesn't require any graphics-driver-specific code; it uses the standard kernel interfaces and graphics stack (Mesa, DRI, libdrm, KMS, etc). That also works fine with radeon and nouveau, and it would work fine with the proprietary drivers as long as they support the standard graphics stack. It just won't work with proprietary drivers that implement their own entirely un-integrated graphics stack, which seems unsurprising to me.
Your window manager of choice can just link into Wayland/Weston and ignore all the underlying graphics driver bits; they'll Just Work.
Your window manager of choice can just link into Wayland/Weston and ignore all the underlying graphics driver bits; they'll Just Work.
It's been years and years since I knew the details, but at one point the nvidia drivers replaced pretty everything from libX11 down, and I don't think you could hook in at a middle point. (ie, they don't support the "standard" graphics stack.) Would appreciate if anyone could comment on how things stand today.
Somewhat better, but still awful. They don't replace libX11, but they do replace libGL, and they bypass the entire kernel graphics stack in favor of their own.
You're describing how it should work, but AFAIK the proprietary drivers don't support KMS and have even talked about never supporting it.
But rather than hacking Weston to support proprietary drivers, I would advocate using it as a carrot to beat AMD/Nvidia into submission.
But rather than hacking Weston to support proprietary drivers, I would advocate using it as a carrot to beat AMD/Nvidia into submission.
While its true that a new "WM" in wayland would be a diffent binary its not really the same as a different X server. All the stuff that X does wrt hardware and stuff is not really in the wayland compositor, but in the drivers, and they are the same with both WMs.
And the WMs would link to the wayland server library, so a lot of code would be shared between them.
I don't think properietary drivers need to have their own wayland server. However, they do need to add EGL support, so that wayland can use the driver outside of X, and a few specific GL extensions that wayland needs. See e.g: http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-way...
And the WMs would link to the wayland server library, so a lot of code would be shared between them.
I don't think properietary drivers need to have their own wayland server. However, they do need to add EGL support, so that wayland can use the driver outside of X, and a few specific GL extensions that wayland needs. See e.g: http://ppaalanen.blogspot.fr/2012/03/what-does-egl-do-in-way...
Wow, does that mean you could end up with a "KDE-only" nvidia driver and a "GNOME-only" ATI/AMD driver?
libwayland being a reference implementation does not mean its not implemented. It just means that the specified thing is the protocol, not the library API. This is similar to libX11, its just a reference implementation, and other implementations like escher (a pure java X11 protocol implementation) exists. Most wayland servers will still use libwayland.
There is nothing special about multiple monitors in wayland. The KMS/Mesa drivers allows setup and rendering to multiple monitors fine, then its up to the wayland server to decide what to draw on each monitor.
There is nothing special about multiple monitors in wayland. The KMS/Mesa drivers allows setup and rendering to multiple monitors fine, then its up to the wayland server to decide what to draw on each monitor.
then its up to the wayland server to decide what to draw on each monitor
This is the kind of "mechanism without any policy" thinking that I was hoping to escape from. What does Weston actually do in the presence of multiple monitors?
This is the kind of "mechanism without any policy" thinking that I was hoping to escape from. What does Weston actually do in the presence of multiple monitors?
Now, this is the sort of article I want to see on Hacker News. Excellent read.
instead of going over the network, they’ll go over a UNIX socket, and the kernel doesn’t have to copy data around.
This isn't really true. Even if the kernel does cow pipes (on both ends), the writer is almost certainly going to trash the page and cause a fault. The data is just copied around without being bundled into TCP packets.
There is also a SHM extension which does use shared memory, but that's separate and mostly only used for pixmaps and the like.
This isn't really true. Even if the kernel does cow pipes (on both ends), the writer is almost certainly going to trash the page and cause a fault. The data is just copied around without being bundled into TCP packets.
There is also a SHM extension which does use shared memory, but that's separate and mostly only used for pixmaps and the like.
> While it’s very apparent that X is going to go away in favor of Wayland some time soon...
Isn't it a bit early to be saying something like that? While I hope that Wayland will succeed, it has an equal chance of ending up as vaporware.
Isn't it a bit early to be saying something like that? While I hope that Wayland will succeed, it has an equal chance of ending up as vaporware.
At minimum we'll get at least one distro attempting to utilize Wayland eventually: Ubuntu. See the bottom of https://wiki.ubuntu.com/Wayland where it reads:
Canonical is investing in making Compiz Wayland-enabled and running Unity on top.
While it does not completely refute your concern of Wayland becoming vaporware, a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone. Put another way: it's more reliable than a random user on the Arch forums* talking up grand plans to rebuild WMFS to work with Wayland.
*A purely random example, and not a knock on Arch forums, which are an awesome/informative place.
Canonical is investing in making Compiz Wayland-enabled and running Unity on top.
While it does not completely refute your concern of Wayland becoming vaporware, a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone. Put another way: it's more reliable than a random user on the Arch forums* talking up grand plans to rebuild WMFS to work with Wayland.
*A purely random example, and not a knock on Arch forums, which are an awesome/informative place.
>a commitment from Canonical is about as close as anything in the Linux Desktop world ever comes to being a hard milestone
Furthermore, most Xorg and desktop developers are pro-Wayland if the OP is to be believed.
Furthermore, most Xorg and desktop developers are pro-Wayland if the OP is to be believed.
I think if anyone really doubts this they should spend some time programming something directly on top of Xorg to convince themselves that when the ball of mud is gone there will be at least a bit more sanity.
I haven't used the raw stream protocol directly, but if you mean Xlib, I actually find it more usable than most toolkits. (Although a big part of that is being able to use select(), whereas most toolkits force you to do everything in callbacks while the toolkit drives the main loop.)
If you've been to a few Linux conferences, you should have seen how much support Wayland had from Xorg developers. So it's here to stay, although even if it's 80% ready, there's still 80% of the work left.
And X isn't going away since we'll probably be stuck using Xwayland for quite sometime for backward-compatibility (yes they've thought of that).
And X isn't going away since we'll probably be stuck using Xwayland for quite sometime for backward-compatibility (yes they've thought of that).
Is Wayland ready for basic use yet?
I'd like to have it run a single local firefox window for instance.
So wait... Wayland does nothing, makes existing things harder and requires the Linux ecosystem to greenspun X in perpetuity?
Not sure why you're being pessimistic. VNC demonstrates that graphical access can be networked at the application level rather than as a fundamental architecture decision we pay for constantly. Wayland is an attempt to cash in on the savings.
Except it doesn't look good, and copy/paste is even shittier.
X runs fine on top of Wayland, just like X runs fine on top of Quartz.
[deleted]