Not all things that look like "interpolators" need traditional lookforward lag.
Mice and head trackers can already run at 1000 Hz. It's the GPU that cannot keep up.
Instead of black-box interpolators (e.g. Sony MotionFlow), a smart interpolator can be made to know the high-frequency controller inputs in realtime, and doesn't even need to use guesswork-based interpolation for everything.
Just shift everything around based on the high-refresh 1000Hz controller input. (In other words, "reprojection").
Also, knowing more data about the source (e.g. near-zero-lag controller input stream) eliminates lots of interpolation guesswork. It's much like how H.264 (video compression) is heavily interpolation-based mathematics during the video codec, but it had full awareness of the source video material, to successfully compress it virtually artifact-free.
So basically, you are simply giving a smart interpolator full awareness of things like geometry & input at a higher rate than the GPU renders. To avoid guesswork on those kinds of items.
Things like future multilayer Z-buffers can help solve a lot of parallax-reveal problems of trying to create intermediate frames, and there are future tweaks they are working on to eliminate reprojection artifacts. Like artifacts or reprojection distortions around edges of objects in front of objects. So adding intermediate frames with full parallax effects can eventually become artifact free because of the GPU's knowledge-in-advance of what-behind-what. Basically, more advanced reprojection algorithms that can create near-flawless intermediate GPU frames (without lookforward) without a full polygonal rerender.
Prediction helps (as it does for Oculus), but remember, we have controllers that already go at ultra high frequencies, and it is expected headtrackers will eventually become ultra high frequency too -- and that extra data can reduce the need to do lookforward prediction.
It's all very complex, with many researchers working on multiple solutions, but it can reduce the average processing-power-required per extra frame, and it can theoretically allow high reprojection ratios without lookforward lag (e.g. theoretical future 10:1, such as multiplying 100fps to 1000fps, at least with 1000Hz input devices like 1000Hz gaming mice, and 1000Hz head trackers).
Several VR scientists have indeed advocated the need for 1000Hz eventually, someday in humankind, as there are confirmed tangible immersion benefits to getting that high and beyond.
That's why I wrote that article full of motion demos explaining the visual science concepts of why 1000Hz displays are needed. It will be useful for passing a theoretical future Holodeck Turing Test (not telling apart a VR headset versus transparent ski goggles in a reality-versus-VR blind test), in terms of Morarity-style or Matrix-style "it's real" VR.
Many tricks layers upon each other, to achieve what's being achieved today, and this creativity will only continue. Lagless lookbehind-only interpolation (utilizing ultra-high-Hz controller input to reproject new 3D position). Foveated rendering too, yes. Realtime beamtracing with realtime denoising (NVIDIA scientist paper), perhaps. Maybe even all piled on top of each other simultaneously, perhaps.
Holy server hug, batman! (Chief Blur Buster here, I noticed the traffic spike).
BTW, GPU is a problem, but we're expecting Frame Rate Amplification Technologies to solve the problem. Basically improved versions of Oculus Spacewarp that can do large framerate multiplication factors with zero parallax artifacts (unlike today).
The gist is that within five to ten years, we'll have many tricks to increase framerates with the same number of transistors, without needing to reduce detail levels or make textures/edges blurry, without input lag, and without interpolation artifacts.
If you're running "ToastyX Strobelight" (easy lightboost programming utility) on a 120Hz LightBoost-compatible monitor -- hit Control+Alt+Plus to enable LightBoost strobing, then Control+Alt+Minus to disable LightBoost strobing. Control+Alt+1 will program the strobing to 1.4 milliseconds, while Control+Alt+0 will program the strobing to 2.4 milliseconds (oscilloscope measurements). The 1.4ms-vs-2.4ms is actually noticeable in the castle at the top of http://www.testufo.com/#test=photo&pps=1440
It's a very interesting exercise in hacking LightBoost 2D as a programmable-persistence computer monitor (100% software hack; no hardware modifications needed; no 3D kit needed)
There are some excellent new web-based motion test animations, that demonstrates eye-tracking-based motion blur, which is quite relevant to Mike's paper:
View these links in Chrome or another web browser that supports perfect VSYNC animations. The pages will attempt to detect if your browser supports VSYNC. IE10+ works well (up to 100Hz), Chrome works well (up to >144Hz), and Safari6+ (iPad's work too). FireFox 22 will judder too much; FireFox 24+ pre-beta adds VSYNC support -- I am the person who convinced Mozilla to add native support for 120fps animations as Bugzilla@Mozilla #856427; now in FF24 pre-beta.
Isn't Mount Sharp ALREADY in the 360 panorama? I SEE IT. It just only looks "small". The mountain in the same direction in the shadow?
The mountain looks small mainly due to camera distortion (but it could also be incorrectly stretched image). But it's not a small mountain -- that's a big one. It is an optical effect caused by the panorama stretching. Zoom in the mountain, and it's exactly identical to the photograph by NASA.
http://mars.jpl.nasa.gov/msl/multimedia/images/?ImageID=4271
...
Look at the details; they match exactly the mountain that's already in the panorama. (the little-looking 'hill' in the direction rover of the shadow, and slightly to the right, is actually Mt Sharp)
Mount sharp just looks small in the panorama, due to optical effect. But zoom in using the mousewheel, and you'll see the surface features match Mount Sharp, even though it's a little smeared (low-resolution). Let's wait for higher-def transmissions, it's still a slow dial-up-style link over 500 million miles...
Mice and head trackers can already run at 1000 Hz. It's the GPU that cannot keep up.
Instead of black-box interpolators (e.g. Sony MotionFlow), a smart interpolator can be made to know the high-frequency controller inputs in realtime, and doesn't even need to use guesswork-based interpolation for everything.
Just shift everything around based on the high-refresh 1000Hz controller input. (In other words, "reprojection").
Also, knowing more data about the source (e.g. near-zero-lag controller input stream) eliminates lots of interpolation guesswork. It's much like how H.264 (video compression) is heavily interpolation-based mathematics during the video codec, but it had full awareness of the source video material, to successfully compress it virtually artifact-free.
So basically, you are simply giving a smart interpolator full awareness of things like geometry & input at a higher rate than the GPU renders. To avoid guesswork on those kinds of items.
Things like future multilayer Z-buffers can help solve a lot of parallax-reveal problems of trying to create intermediate frames, and there are future tweaks they are working on to eliminate reprojection artifacts. Like artifacts or reprojection distortions around edges of objects in front of objects. So adding intermediate frames with full parallax effects can eventually become artifact free because of the GPU's knowledge-in-advance of what-behind-what. Basically, more advanced reprojection algorithms that can create near-flawless intermediate GPU frames (without lookforward) without a full polygonal rerender.
Prediction helps (as it does for Oculus), but remember, we have controllers that already go at ultra high frequencies, and it is expected headtrackers will eventually become ultra high frequency too -- and that extra data can reduce the need to do lookforward prediction.
It's all very complex, with many researchers working on multiple solutions, but it can reduce the average processing-power-required per extra frame, and it can theoretically allow high reprojection ratios without lookforward lag (e.g. theoretical future 10:1, such as multiplying 100fps to 1000fps, at least with 1000Hz input devices like 1000Hz gaming mice, and 1000Hz head trackers).
Several VR scientists have indeed advocated the need for 1000Hz eventually, someday in humankind, as there are confirmed tangible immersion benefits to getting that high and beyond.
That's why I wrote that article full of motion demos explaining the visual science concepts of why 1000Hz displays are needed. It will be useful for passing a theoretical future Holodeck Turing Test (not telling apart a VR headset versus transparent ski goggles in a reality-versus-VR blind test), in terms of Morarity-style or Matrix-style "it's real" VR.
Many tricks layers upon each other, to achieve what's being achieved today, and this creativity will only continue. Lagless lookbehind-only interpolation (utilizing ultra-high-Hz controller input to reproject new 3D position). Foveated rendering too, yes. Realtime beamtracing with realtime denoising (NVIDIA scientist paper), perhaps. Maybe even all piled on top of each other simultaneously, perhaps.