Wayland 3D Compositor with Oculus Rift and Razer Hydra [video](youtube.com)
youtube.com
Wayland 3D Compositor with Oculus Rift and Razer Hydra [video]
https://www.youtube.com/watch?v=Dgtba_GpG-U
12 comments
Luckily these goggles won't stay at the current resolution for long, we should be seeing 4k and higher in the next few years.
This is really cool. I ordered a DK2 to try out some similar things in a similar domain. The idea of a 120° workspace which can be expanded indefinitely seems like a reasonable alternative to getting bunches of physical screens on a desk.
This is also 100% the reason I ordered one.
I've spent quite a deal of time thinking about how you make a 3D interface actually work for usability, and its interesting to see some of those ideas implemented in this example (allowing 3D views to either pop out in the workspace, or be pushed back into their own world).
I've spent quite a deal of time thinking about how you make a 3D interface actually work for usability, and its interesting to see some of those ideas implemented in this example (allowing 3D views to either pop out in the workspace, or be pushed back into their own world).
The beauty of this approach is that it frees up window managers from managing desktop spaces and instead allows you to manage windows and the relation of the windows relative to other windows. Being freed of borders and boundaries presents amazing opportunities. Defining things in absolute space is very confining. Defining things in relativistic space, gives you the opportunity to move into multi-view/multi-screen modes with multiple participants much more easily. It's not longer about desktops, but about taskspaces.
I don't want to say "put window A on the lefthand side of the desktop and put window B on the righthand side of the desktop." Instead I want to be able to say "put A to the left of B in when mode X" or "put C on top of D when performing task Y."
It's simply much more natural and more intuitive to tell the computer where your focus should be (or if the focus should be split equally between 2 or more windows), and then defining all the peripheral windows that should appear around the focused window. Ideally these wouldn't even separate windows, but simply a rectangular stream of pixels from the buffers of your choice, so that the same application could be multiplexed to layout modes. This would basically be tmux for windows/apps in 3D space.
A prolog-like declarative language with relations and predicates for describing windows and how they are organized relative to one another under variable conditions would be awesome. Type one command to switch to the window you want to have focus in the center of your screen. Once that window is in place, bring into focus the last used "mode" for that window, with all the peripheral windows coming in from behind you or zooming in from the back (time machine style, but with your primary focused window held constant). Then you can activate switching between modes with another key stroke combination.
I don't want to say "put window A on the lefthand side of the desktop and put window B on the righthand side of the desktop." Instead I want to be able to say "put A to the left of B in when mode X" or "put C on top of D when performing task Y."
It's simply much more natural and more intuitive to tell the computer where your focus should be (or if the focus should be split equally between 2 or more windows), and then defining all the peripheral windows that should appear around the focused window. Ideally these wouldn't even separate windows, but simply a rectangular stream of pixels from the buffers of your choice, so that the same application could be multiplexed to layout modes. This would basically be tmux for windows/apps in 3D space.
A prolog-like declarative language with relations and predicates for describing windows and how they are organized relative to one another under variable conditions would be awesome. Type one command to switch to the window you want to have focus in the center of your screen. Once that window is in place, bring into focus the last used "mode" for that window, with all the peripheral windows coming in from behind you or zooming in from the back (time machine style, but with your primary focused window held constant). Then you can activate switching between modes with another key stroke combination.
This is pretty cool as far as tech goes, but also seems rather ridiculous practically. Productivity with a HMD will likely amount to a lot more than the same 2D dynamic images we already have(screens) being placed arbitrarily in 3D space. What will it look like instead? one may ask, to which I have no definitive response.
What we have here seems like an extrapolation of what we have now, which may prove fruitful if we have nothing else to go off of (and is the nature of incremental progress). If I had many more monitors I'd place a class (or any other abstraction) on every screen, so maybe every class will have its own 2D plane of text and they'll be organized semantically & dynamically in 3D space. Also resolution here needs to be really really high. Idk, just interesting food for thought.
What we have here seems like an extrapolation of what we have now, which may prove fruitful if we have nothing else to go off of (and is the nature of incremental progress). If I had many more monitors I'd place a class (or any other abstraction) on every screen, so maybe every class will have its own 2D plane of text and they'll be organized semantically & dynamically in 3D space. Also resolution here needs to be really really high. Idk, just interesting food for thought.
As you say, it's an interesting tech demo (which looks like it would be even more fun if you coupled it with a physics engine btw), but not particularly practical.
The way I could see it working though (without a significant change in OS UI) is by arranging your 2D desktop across multiple virtual 2D screens in 3D space, ie a bank of monitors angled towards you. You could use a conventional keyboard and mouse to navigate the 2D space (with perhaps mouse movement assisted or replaced by hand/head/eye tracking), and drag windows between screens, set programs to open on certain ones, with full screen clipping to screen edges - all basically as many OSes do already.
But because they're virtual, you could resize, split and merge screens, curve large ones, have ones which float in front of your vision to use for toolbar windows and HUDs. You could even use existing technology like VMs or VNC to use multiple OSes at the same time.
Mount your keyboard and mouse to a swivel chair which tilts, and you could comfortably have a 360 degree working area (assuming you can touch type) 4 or so screens deep, which would put traders' setups to shame (and at a fraction of the price). Throw in some panoramic backgrounds and you can move your office to a forest or a beach. Take it further you could even set up some sort of virtual office, being able to virtually get up from your desk and visit the workspaces of remote-working colleagues and friends to collaborate.
The exciting thing is that none of this is a massive leap from where we are now, and it would map pretty seamlessly to the software and UI we already use. The main thing holding it back would seem to be the resolution of 3D goggles, but with 4k rising, even that can't be too far off.
The way I could see it working though (without a significant change in OS UI) is by arranging your 2D desktop across multiple virtual 2D screens in 3D space, ie a bank of monitors angled towards you. You could use a conventional keyboard and mouse to navigate the 2D space (with perhaps mouse movement assisted or replaced by hand/head/eye tracking), and drag windows between screens, set programs to open on certain ones, with full screen clipping to screen edges - all basically as many OSes do already.
But because they're virtual, you could resize, split and merge screens, curve large ones, have ones which float in front of your vision to use for toolbar windows and HUDs. You could even use existing technology like VMs or VNC to use multiple OSes at the same time.
Mount your keyboard and mouse to a swivel chair which tilts, and you could comfortably have a 360 degree working area (assuming you can touch type) 4 or so screens deep, which would put traders' setups to shame (and at a fraction of the price). Throw in some panoramic backgrounds and you can move your office to a forest or a beach. Take it further you could even set up some sort of virtual office, being able to virtually get up from your desk and visit the workspaces of remote-working colleagues and friends to collaborate.
The exciting thing is that none of this is a massive leap from where we are now, and it would map pretty seamlessly to the software and UI we already use. The main thing holding it back would seem to be the resolution of 3D goggles, but with 4k rising, even that can't be too far off.
It'd be fantastic to have an Oculus-based desktop environment! Imagine not having to drag giant monitors around. Combine it with xmonad and you're golden. I hope this becomes reality soon, at least for development purposes.
Shut up and someone take my money already.
Shut up and someone take my money already.
Xmonad is a way to optimize on 2D spatial scarcity. I don't think Xmonad will exist in any similar implementation in 3D viewing space. It might end up being similar, but I think that depth, being what humans with camera vision are used to, will become a much more dominating organizational pattern.
Drop xmonad, what you want is no-longer the virtual desktops, but the virtual rooms! (As a bonus it has been proved humans memory interact with entering and leaving rooms.) We will be hanging applications on the walls like pictures.
I always thought what you need is instead a kind of body model.
So you have things which are attached to your head, things which follow position relative to your body, and things which follow your absolute position in the world.
Most people would work in body-space, and maybe have several different overlays they switch between for that.
On top of that, you probably need the concept of the private, shared and public objects in all these spaces - you can interact or allow people into a limited view of your 3D environment.
So you have things which are attached to your head, things which follow position relative to your body, and things which follow your absolute position in the world.
Most people would work in body-space, and maybe have several different overlays they switch between for that.
On top of that, you probably need the concept of the private, shared and public objects in all these spaces - you can interact or allow people into a limited view of your 3D environment.
Thanks, but no thanks.
Especially when graphic programming using flow-charts will be implemented.
Everything old is new again! ;)
http://c2.com/cgi/wiki?JaronLanier
http://www.jaronlanier.com/vpl.html
http://c2.com/cgi/wiki?BounceLanguage
http://www.art.net/~hopkins/Don/lang/bounce/bounce.html
http://www.vrs.org.uk/virtual-reality-profiles/vpl-research....
http://www.nic.funet.fi/index/graphics/misc/siggraph/courses...
https://w2.eff.org/Misc/Publications/John_Perry_Barlow/being...
http://c2.com/cgi/wiki?JaronLanier
http://www.jaronlanier.com/vpl.html
http://c2.com/cgi/wiki?BounceLanguage
http://www.art.net/~hopkins/Don/lang/bounce/bounce.html
http://www.vrs.org.uk/virtual-reality-profiles/vpl-research....
http://www.nic.funet.fi/index/graphics/misc/siggraph/courses...
https://w2.eff.org/Misc/Publications/John_Perry_Barlow/being...
Another 3D Wayland compositor example: https://www.youtube.com/watch?v=_FjuPn7MXMs
nice one. It reminds me The Lawnmower Man movie. Soon, you will be able to physically dive into the code.
The one effect I really really want is when you switch from 2D -> 3D mode with the Goggles I kind of want the 2D desktop to do the laser activation effect from Tron Legacy.
I hope the author ports this to something else than Oculus, since I bailed on it when it sold out.
The next version is up for pre-order. http://www.oculusvr.com
He means, he became emotionally detached to Oculus when they made the decision to collaborate with Facebook.
Thanks... you put it better than I did. That was poorly worded on my part. More 'averse' than emotionally detached, though.
(No, I'm not the author.)
Code: https://github.com/evil0sheep/qtwayland-motorcar-compositor
Thesis: https://github.com/evil0sheep/MastersThesis/raw/master/thesi...
Defense Slides: https://docs.google.com/presentation/d/1svgGMxxbfmcHy_KuS5Q9...
Code: https://github.com/evil0sheep/qtwayland-motorcar-compositor
Thesis: https://github.com/evil0sheep/MastersThesis/raw/master/thesi...
Defense Slides: https://docs.google.com/presentation/d/1svgGMxxbfmcHy_KuS5Q9...
If you doubt it, try setting your monitor to something other than its native resolution and coding for a while using the interpolated resolution. I'm afraid text in floating 3d windows on vr sets is going to look like that for a long time to come.
I'm still going to try, dammit.