Nvidia's G-Sync: Attempting to Revolutionize Gaming via Smoothness(anandtech.com)
anandtech.com
Nvidia's G-Sync: Attempting to Revolutionize Gaming via Smoothness
http://www.anandtech.com/show/7436/nvidias-gsync-attempting-to-revolutionize-gaming-via-smoothness
154 comments
Here's an example of what's going on:
1. The game decides what happened since the last frame: an opponent appeared on the screen
2. Game -> GPU (3D geometry)
3. GPU -> Frame buffer (geometry is rasterized into pixels)
4. Another part of the GPU handles the DisplayPort protocol... Frame buffer -> DisplayPort -> Monitor
5. This is where G-sync works. Based on all the images supplied on the NVidia site, the $100 G-sync board is a drop-in _replacement_ for the controller board in your monitor. So obviously, it's only guaranteed to work on the ASUS VG248QE. Monitor's DisplayPort input -> Frame buffer
6. Where G-sync shines: a vanilla controller board in your monitor should not buffer very much of the pixel data. But they sometimes buffer waaaay more than necessary. To oversimplify, you could receive 1 row of pixels on the DisplayPort and send them out as analog signals to the transistors in the LCD panel while filling up the buffer again for the next line. NVidia's uses something about the vertical blank packet to tell the board that a new frame is coming _now_.
7. The transistors react to the change in voltage and either block light or allow it to pass through.
8. You see the change and shoot your opponent.
Because of the way the liquid crystals respond to voltage, the analog signals to the panel are anything but simple [1]. The display can't just be left "on" and can't be expected to instantly react to changes. So the G-sync board has to be panel-specific for the Asus display, but is smarter than your average display controller.
The oversimplified example above breaks down because the transistors embedded in the panel need a varying signal, so the controller is actually driving the panel at rates much higher than 144Hz per full frame. This way, each transistor experiences an average voltage something like [1] caused by the rapid updates coming from the controller. Separate LCD driver chips disappeared over 10 years ago; now all LCD panels are designed to be driven by a high-frequency digital signal that averages out to the voltage the panel needs. In car audio it's called a "1-bit DAC," but inside an LCD it's called a "wire." :)
Interestingly, G-Sync may actually be capable of shortening the panel's life. Since it is capped at 144Hz which is the panel's maximum rated speed, it may be perfectly safe. But any time you go and change the analog signals to the panel there could be harmful effects: slow changes in color calibration, ghosting, or even dead pixels.
[1] Image only: http://commons.wikimedia.org/wiki/File:LCD_Panel_drive_%28Ac...
I looked for a good explanation of the physics behind LCD drive voltage, but Google apparently can't find any good articles out there...
1. The game decides what happened since the last frame: an opponent appeared on the screen
2. Game -> GPU (3D geometry)
3. GPU -> Frame buffer (geometry is rasterized into pixels)
4. Another part of the GPU handles the DisplayPort protocol... Frame buffer -> DisplayPort -> Monitor
5. This is where G-sync works. Based on all the images supplied on the NVidia site, the $100 G-sync board is a drop-in _replacement_ for the controller board in your monitor. So obviously, it's only guaranteed to work on the ASUS VG248QE. Monitor's DisplayPort input -> Frame buffer
6. Where G-sync shines: a vanilla controller board in your monitor should not buffer very much of the pixel data. But they sometimes buffer waaaay more than necessary. To oversimplify, you could receive 1 row of pixels on the DisplayPort and send them out as analog signals to the transistors in the LCD panel while filling up the buffer again for the next line. NVidia's uses something about the vertical blank packet to tell the board that a new frame is coming _now_.
7. The transistors react to the change in voltage and either block light or allow it to pass through.
8. You see the change and shoot your opponent.
Because of the way the liquid crystals respond to voltage, the analog signals to the panel are anything but simple [1]. The display can't just be left "on" and can't be expected to instantly react to changes. So the G-sync board has to be panel-specific for the Asus display, but is smarter than your average display controller.
The oversimplified example above breaks down because the transistors embedded in the panel need a varying signal, so the controller is actually driving the panel at rates much higher than 144Hz per full frame. This way, each transistor experiences an average voltage something like [1] caused by the rapid updates coming from the controller. Separate LCD driver chips disappeared over 10 years ago; now all LCD panels are designed to be driven by a high-frequency digital signal that averages out to the voltage the panel needs. In car audio it's called a "1-bit DAC," but inside an LCD it's called a "wire." :)
Interestingly, G-Sync may actually be capable of shortening the panel's life. Since it is capped at 144Hz which is the panel's maximum rated speed, it may be perfectly safe. But any time you go and change the analog signals to the panel there could be harmful effects: slow changes in color calibration, ghosting, or even dead pixels.
[1] Image only: http://commons.wikimedia.org/wiki/File:LCD_Panel_drive_%28Ac...
I looked for a good explanation of the physics behind LCD drive voltage, but Google apparently can't find any good articles out there...
> Also, some of us have (in business environments) $1k+ S-IPS 30"+ monitors — the quality of these monitors is way above that of consumer models like the VG248QE and others. If there is no way to generically mod monitors without onboard DSPs, I could see that hindering adoption.
I think Nvidia is targeting hardcore gamers first and foremost. Most gamers are not gaming at 2560x1600/1440. Some are, but most aren't.
The most popular monitors by pro gamers right now (Twitch/eSport players and enthusiasts) are 120/144hz 1ms monitors, such as the ASUS VG248QE. Color reproduction isn't as important to pro gamers as smoothness/framerates.
Also hardcore/pro players are dumping lots of money on the most expensive computer rigs, often upgrading to the latest and greatest every generation. They are a very important marketing group for Nvidia.
I think Nvidia is targeting hardcore gamers first and foremost. Most gamers are not gaming at 2560x1600/1440. Some are, but most aren't.
The most popular monitors by pro gamers right now (Twitch/eSport players and enthusiasts) are 120/144hz 1ms monitors, such as the ASUS VG248QE. Color reproduction isn't as important to pro gamers as smoothness/framerates.
Also hardcore/pro players are dumping lots of money on the most expensive computer rigs, often upgrading to the latest and greatest every generation. They are a very important marketing group for Nvidia.
> The most popular monitors by pro gamers right now (Twitch/eSport players and enthusiasts) are 120/144hz 1ms monitors, such as the ASUS VG248QE.
I wonder if that latency is noticeable to them or this this the same market as the audiophile market that sells gold-plated cables for 100x markup.
I wonder if that latency is noticeable to them or this this the same market as the audiophile market that sells gold-plated cables for 100x markup.
It's not really about the latency (5ms vs 1ms is negligible), but its about the pixel response to reduce/eliminate ghosting and other artifacts of LCD's persistent pixels. The speed that the pixels can update is proportional to the amount of ghosting. Interestingly enough, it won't eliminate it no matter how fast the pixels update. The real problem with ghosting turned out to be precisely the pixel-persistence. Even more interesting is that someone discovered a hack for the modern 3D monitors like the ASUS mentioned that completely eliminates ghosting: the strobing backlight functionality necessary for 3D completely eliminates ghosting when applied to 2D. I currently have this setup and its exactly like using a CRT. A flat, light, 1920x1080 CRT. It's beautiful.
That makes sense, thanks for explaining!
He's actually completely wrong. Persistence is about image quality, and can be mitigated by filtering that hardcore gamers always turn off, because it costs them latency.
Reducing latency isn't about how noticeable it is. Latency can be completely impossible to detect for you but still hurt you.
Input lag is the time between providing some input, such as clicking with your mouse, to getting feedback of this event on the screen. As the clicking will be prompted by things happening on the screen, input lag acts as a command delay to everything that is done. The most interesting feature of latency is that all latency is additive. It doesn't matter how fast or slow each part in the system is, none of them can hide latency for one another. Or, even if the game simulation adds 150ms and your stupid wireless mouse adds 15ms, the 2 ms added by the screen still matter just as much.
The second mental leap is that the human controlling the system can also be considered to be just another part in the pipeline adding latency. Consider a twitch shooter, where two players suddenly appear on each other's screens. Who wins depends on who first detects the other guy, successfully aims at him, and pulls the trigger. In essence, it's a contest between the total latency (simulation/cpu + gpu + screen + person + mouse + simulation) of one player against the other player. Since all top tier human players have latencies really close to one another, even minute differences, 2 ms here or there, produce real detectable effects.
Reducing latency isn't about how noticeable it is. Latency can be completely impossible to detect for you but still hurt you.
Input lag is the time between providing some input, such as clicking with your mouse, to getting feedback of this event on the screen. As the clicking will be prompted by things happening on the screen, input lag acts as a command delay to everything that is done. The most interesting feature of latency is that all latency is additive. It doesn't matter how fast or slow each part in the system is, none of them can hide latency for one another. Or, even if the game simulation adds 150ms and your stupid wireless mouse adds 15ms, the 2 ms added by the screen still matter just as much.
The second mental leap is that the human controlling the system can also be considered to be just another part in the pipeline adding latency. Consider a twitch shooter, where two players suddenly appear on each other's screens. Who wins depends on who first detects the other guy, successfully aims at him, and pulls the trigger. In essence, it's a contest between the total latency (simulation/cpu + gpu + screen + person + mouse + simulation) of one player against the other player. Since all top tier human players have latencies really close to one another, even minute differences, 2 ms here or there, produce real detectable effects.
This is completely wrong. When even the fastest human reaction time is on the order of 200ms, 5ms vs 1ms of monitor input lag has no effect on the outcome. Also consider that 5ms is within the snapshot time that servers run on, so +/- 5ms is effectively simultaneous to the server on average.
Pixel persistence is not about image quality and cannot be mitigated by anything, except turning off the backlight at an interval in sync with the frame rate you're updating the image. This is how CRTs work, and that's why they had no ghosting effects. The 3D graphics driver hack I mentioned does exactly that for 3D enabled LCD monitors.
Pixel persistence is not about image quality and cannot be mitigated by anything, except turning off the backlight at an interval in sync with the frame rate you're updating the image. This is how CRTs work, and that's why they had no ghosting effects. The 3D graphics driver hack I mentioned does exactly that for 3D enabled LCD monitors.
People can notice input latencies that are many times smaller than their reaction time. 200ms of input latency is going to be noticeable and bothersome to basically everyone for even basic web browsing tasks. Most gamers will notice more than 2-3 frames of latency, and even smaller latencies will be noticed in direct manipulation set-ups like touchscreens and VR goggles where the display has to track 1:1 the user's physical movements.
I think you misunderstood my point. In terms of actual advantage, 1ms vs 5ms is negligible, considering the fact that human reaction time is 200ms. So in the case of shooting someone as they popped out from behind a corner, the 200ms reaction time + human variation + variation in network latency + discreet server time, will absolutely dominate the effects.
I definitely agree that small latencies can be noticed, even latencies approaching 5ms (but not 5ms itself--I've seen monitor tests done that showed this).
I definitely agree that small latencies can be noticed, even latencies approaching 5ms (but not 5ms itself--I've seen monitor tests done that showed this).
> I think you misunderstood my point. In terms of actual advantage, 1ms vs 5ms is negligible, considering the fact that human reaction time is 200ms.
You did not understand the point of my post. The quality that matters is total latency. How long a human takes to react is completely irrelevant to what level of latency has an effect. Whether average human reaction time was 1ms or 1s doesn't matter. All that matters is that your loop is shorter than his, and your reaction time is very near his, so any advantage counts.
> the 200ms reaction time + human variation + variation in network latency + discreet server time, will absolutely dominate the effects.
Server tick time is the same for everyone. Top level gaming tourneys are held in lans, where the players typically make sure that the network latency from their machine to the server is not any greater than from anyone else. However, none of that matters to the question at hand.
Assume that total latency of the system, including the player, can be approximated by:
Human_reaction_time + network_lag + processing_lag + display_lag
and assume all are normally distributed random around some base value, except display lag, and you have:
(midpoint, standard deviation)
rand(200,20) + rand(20,5) + rand(16,2) + 15
while I have:
rand(200,20) + rand(20,5) + rand(16,2) + 5
The total latency is utterly dominated by the human processing time. Yet if we model this statistically, and assume that lower latency wins, the one with the faster screen wins 63% of time. That's enough of an edge that people pay money for it.
You did not understand the point of my post. The quality that matters is total latency. How long a human takes to react is completely irrelevant to what level of latency has an effect. Whether average human reaction time was 1ms or 1s doesn't matter. All that matters is that your loop is shorter than his, and your reaction time is very near his, so any advantage counts.
> the 200ms reaction time + human variation + variation in network latency + discreet server time, will absolutely dominate the effects.
Server tick time is the same for everyone. Top level gaming tourneys are held in lans, where the players typically make sure that the network latency from their machine to the server is not any greater than from anyone else. However, none of that matters to the question at hand.
Assume that total latency of the system, including the player, can be approximated by:
Human_reaction_time + network_lag + processing_lag + display_lag
and assume all are normally distributed random around some base value, except display lag, and you have:
(midpoint, standard deviation)
rand(200,20) + rand(20,5) + rand(16,2) + 15
while I have:
rand(200,20) + rand(20,5) + rand(16,2) + 5
The total latency is utterly dominated by the human processing time. Yet if we model this statistically, and assume that lower latency wins, the one with the faster screen wins 63% of time. That's enough of an edge that people pay money for it.
No I understood your point, I just don't agree that it results in any meaningful advantage. What you didn't model was the fact that the server does not process packets immediately as they are received. They are buffered and processed in batch during a server tick. If the two packets from different players are not received along a tick boundary, then the server will effectively consider them simultaneous.
And remember, we're considering 1ms vs 5ms, so the difference would be 4ms in this case. I would like to see what percentage an advantage someone has in this setup. Even 63% isn't anything significant considering skill comes down to game knowledge rather than absolute reaction time. People will pay for smaller/bigger numbers, sure. But that doesn't mean there is anything practically significant about it.
And remember, we're considering 1ms vs 5ms, so the difference would be 4ms in this case. I would like to see what percentage an advantage someone has in this setup. Even 63% isn't anything significant considering skill comes down to game knowledge rather than absolute reaction time. People will pay for smaller/bigger numbers, sure. But that doesn't mean there is anything practically significant about it.
But it doesn't average out. The 5ms player is continuosly 4ms behind the other player. As above poster explained - the times add up. So you have 200ms + one to five ms. If server tick is as little as 5ms the problem is even worse as in that case player A will with exact same reactions get a faster tick in 4 out of 5 times. I don't know how often it matters, but I'd expect top player to have pretty similar reaction times. So let's say 2 opponents are both between 200 and 220 ms reaction time - then constantly having 4ms more for one player definitely sounds like it will have an effect.
edit: Or in other words - it depends on how often the reaction of the opponents is with the 4ms difference. That certainly depends on the game and the players.
edit: Or in other words - it depends on how often the reaction of the opponents is with the 4ms difference. That certainly depends on the game and the players.
Most server ticks are nowhere near 5ms. Quake 3 ran on 20-30 tick, CS:S/CS:GO runs on 33 by default and up to 66 if you run on high quality servers. 100 tick was somewhat common in CS:S. Some really expensive servers claimed 500 tick but I never bought it. Either way no one's client would update that fast.
Furthermore, if you watch pro matches, you'll quickly realize their skill has nothing to do with having the fastest reaction time. Once you get to a certain skill level, it all comes down to game knowledge. Having a consistent 4ms advantage is absolutely negligible.
Furthermore, if you watch pro matches, you'll quickly realize their skill has nothing to do with having the fastest reaction time. Once you get to a certain skill level, it all comes down to game knowledge. Having a consistent 4ms advantage is absolutely negligible.
I just did go with the 5ms because you mentioned those in your previous post...
>I wonder if that latency is noticeable to them
If you get a chance, demo a 120hz monitor setup and spin around quickly in an FPS. It's quite noticeable. It almost feels extra surreal a la the Hobbit at 48fps until you get used to it.
If you get a chance, demo a 120hz monitor setup and spin around quickly in an FPS. It's quite noticeable. It almost feels extra surreal a la the Hobbit at 48fps until you get used to it.
What is the real size of the market of gamers who upgrade with every new generation of hardware? I've gotta say, I know many gamers (though no professional ones), and none of them upgrade that often. It's more like once every 2-3 years at the most.
It just seems like a bit of a sensational claim..
It just seems like a bit of a sensational claim..
Hardcore gamers are not a great source of income however a great marketing resource for nvidia. They are very influential on others when choosing products. They also represent a large portion of the review industry online.
Having the crown for best graphic card even translates to sales on low end laptops.
Having the crown for best graphic card even translates to sales on low end laptops.
Exactly, especially in the Twitch.tv and eSport era. Sponsored players flaunt their hardware, often linking to Amazon product pages or giving hardware away in their Twitch channels. These professional players have tens of thousands of viewers, and thousands of subscribers, on Twitch. There's a lot of marketing to be had.
>Is this technology going to require hardware manufacturers to cooperate with nVidia, or will their cooperation simply make things nicer/easier?
Carmack said on his twitter:
https://twitter.com/ID_AA_Carmack/status/391301331325321216
"great tech, but can it really survive as an NVIDIA only thing, versus an industry standard everyone could support?"
"I think everyone will wind up with it, or something similar. It is low hanging fruit."
Carmack said on his twitter:
https://twitter.com/ID_AA_Carmack/status/391301331325321216
"great tech, but can it really survive as an NVIDIA only thing, versus an industry standard everyone could support?"
"I think everyone will wind up with it, or something similar. It is low hanging fruit."
Carmack had some interesting notes on this, a couple being:
- This would be perfect for games common in emulation where frame rates are capped but vertical sync isn't used. https://twitter.com/id_aa_carmack/status/391303034745401344
- This technology will come later to laptop and mobile devices and sadly, he tried to get Apple to do this years ago. https://twitter.com/id_aa_carmack/status/391303627278925824
- This would be perfect for games common in emulation where frame rates are capped but vertical sync isn't used. https://twitter.com/id_aa_carmack/status/391303034745401344
- This technology will come later to laptop and mobile devices and sadly, he tried to get Apple to do this years ago. https://twitter.com/id_aa_carmack/status/391303627278925824
> and sadly, he tried to get Apple to do this years ago.
As an Apple user, this doesn't really surprise me. Apple has never liked games, although I'm hoping that was a Steve Jobs thing and the company will see the light.
In some ways they are behind on displays. Windows 7 and above supports 'Deep Color' (30-bit or more), but as far as I know Mountain Lion doesn't.
They're were out first with retina though.
It wouldn't surprise me if this doesn't come to OS X any time soon. Too bad.
As an Apple user, this doesn't really surprise me. Apple has never liked games, although I'm hoping that was a Steve Jobs thing and the company will see the light.
In some ways they are behind on displays. Windows 7 and above supports 'Deep Color' (30-bit or more), but as far as I know Mountain Lion doesn't.
They're were out first with retina though.
It wouldn't surprise me if this doesn't come to OS X any time soon. Too bad.
>As an Apple user, this doesn't really surprise me. Apple has never liked games, although I'm hoping that was a Steve Jobs thing and the company will see the light.
But Apple DOES have a thing for responsiveness and smoothness, and latency and smoothness is exaclty what this tech is improving. I think you could make a strong case without mentioning games. Not that you would have to anymore, since games are so popular on these devices...
But an example: I find the animations in IOS 7 look great on 5S but a bit choppy on regular 5. They clearly aren't maintaining 60 FPS on that device and you can see some hitching. This tech is most beneficial at making variable frame rates between 30 and 60 be more smooth so it could be a big help.
But Apple DOES have a thing for responsiveness and smoothness, and latency and smoothness is exaclty what this tech is improving. I think you could make a strong case without mentioning games. Not that you would have to anymore, since games are so popular on these devices...
But an example: I find the animations in IOS 7 look great on 5S but a bit choppy on regular 5. They clearly aren't maintaining 60 FPS on that device and you can see some hitching. This tech is most beneficial at making variable frame rates between 30 and 60 be more smooth so it could be a big help.
I used to think my next computer purchase would be whatever was required to drive the first 4k monitor under $2k. But I have seen that I was incorrect. Smoothness is far more important than resolution (to me), so if this technology proves to be useful, I will probably end up purchasing a monitor with this capability and a new computer to go with it.
I really have to applaud the engineers at Nvidia, and whoever else drove this initiative. I thought graphics were slowing down and I wouldn't need to upgrade for a while (it's already been two years). To come up with a product that proves me wrong is both surprising and delightful. Great work from a corporate perspective, great work from a gaming perspective, and great work from an engineering perspective. Just really fantastic all around.
I really have to applaud the engineers at Nvidia, and whoever else drove this initiative. I thought graphics were slowing down and I wouldn't need to upgrade for a while (it's already been two years). To come up with a product that proves me wrong is both surprising and delightful. Great work from a corporate perspective, great work from a gaming perspective, and great work from an engineering perspective. Just really fantastic all around.
Agreed! But I see no reason why we can't have both. Resolution and smoothness. I'd be so happy with both.
Especially if I can have it at Seiki prices. I picked up one of their $700 4Ks and the only thing holding Seiki back from cracking the desktop display market is the 30Hz limit they inherit from HDMI 1.4, making the monitor only suitable for work (but very well suited to work).
Especially if I can have it at Seiki prices. I picked up one of their $700 4Ks and the only thing holding Seiki back from cracking the desktop display market is the 30Hz limit they inherit from HDMI 1.4, making the monitor only suitable for work (but very well suited to work).
Unless the world economy collapses permanently, I have no doubt that we'll have both eventually. I don't have the patience to wait for inexpensive 4k in the face of this sync stuff though, which is why I can't have both in this generation.
Curious: how is the Seiki working out for you? I've heard its not a great TV, but might be an OK computer monitor.
I thought 4k was just for large tvs, can you tell a difference over a retina display? Aren't they called retina because that's the most your eye can see?
Genuinely curious, thanks!
Genuinely curious, thanks!
No. They're called "retina" because the people who name things at Apple are professionally focused on deceiving you. What a surprise.
I've seen lots of articles about this over time, here's one I googled just now:
http://www.cultofmac.com/173702/why-retina-isnt-enough-featu...
Combine this with what jamesaguillar already said, about wanting a larger screen and also wanting high PPI, and why wouldn't you want higher res?
I've seen lots of articles about this over time, here's one I googled just now:
http://www.cultofmac.com/173702/why-retina-isnt-enough-featu...
Combine this with what jamesaguillar already said, about wanting a larger screen and also wanting high PPI, and why wouldn't you want higher res?
> I thought 4k was just for large tvs
4k is nothing more than the double resolution from 1080p
> can you tell a difference over a retina display?
A 21" Retina Display iMac would be 4k (the current 21" iMac is 1920x1080). A 27" Retina Display iMac would be way beyond 4k (it would have a 5120x2880 display)
> Aren't they called retina because that's the most your eye can see?
That's more of a marketing moniker and incomplete. The original point/qualifier is that they fall beyond the eye's angular resolution so you can't "see" individual pixels anymore. That's not "the most your eyes can see" though, many arthropods create details & colors through nanometric structures.
4k is nothing more than the double resolution from 1080p
> can you tell a difference over a retina display?
A 21" Retina Display iMac would be 4k (the current 21" iMac is 1920x1080). A 27" Retina Display iMac would be way beyond 4k (it would have a 5120x2880 display)
> Aren't they called retina because that's the most your eye can see?
That's more of a marketing moniker and incomplete. The original point/qualifier is that they fall beyond the eye's angular resolution so you can't "see" individual pixels anymore. That's not "the most your eyes can see" though, many arthropods create details & colors through nanometric structures.
Actually, 4k screens are useless for TVs (despite what all the TV manufactures want you to believe), since you generally sit so far away from them. A 4k monitor would be very nice, since it would be large but still very high DPI (ie, "Retina").
My understanding is that a 4k 30" monitor is a retina display at 30 inches. I know that I can see pixel edges on my Dell Ultrasharp, and it's 2560x1440. My guess is that I would not be able to see pixel edges if the resolution was quadrupled (4k).
Why can't a display behave exactly as bit-mapped memory? That is, you set this pixel in memory, and that pixel changes on the screen, at that time.
That is, like a vector display, but using the LEDs of a flat screen. There's no electron ray scanning over the phosphors as there was in a cathode ray tube.
--
The LEDs can switch only so fast - but does this latency prevent them from being switched independently?
The LEDs need to stay on for long enough for the human visual system to perceive them (and without flickering etc) - but this could be managed in other ways.
I think the main reason, apart from inertia, is that the larger market for displays is as TVs, where the concept of updating the whole frame (frames per second) is even more entrenched - though, there's no reason why video couldn't be displayed in the same way, it's just pushing out a kind of compression to the display itself. Light from real objects is not emitted one frame at a time.
That is, like a vector display, but using the LEDs of a flat screen. There's no electron ray scanning over the phosphors as there was in a cathode ray tube.
--
The LEDs can switch only so fast - but does this latency prevent them from being switched independently?
The LEDs need to stay on for long enough for the human visual system to perceive them (and without flickering etc) - but this could be managed in other ways.
I think the main reason, apart from inertia, is that the larger market for displays is as TVs, where the concept of updating the whole frame (frames per second) is even more entrenched - though, there's no reason why video couldn't be displayed in the same way, it's just pushing out a kind of compression to the display itself. Light from real objects is not emitted one frame at a time.
I assume you're talking about LCD? Not too many LED displays around (though the principle is still similar).
While there's no scanning electron beam the electronics on these displays are still optimized for scanning. That is they push a whole bunch of adjacent pixels in every clock cycle. If you try to use those for random access your refresh rate is going to fall dramatically. The problem isn't whether or not each pixel can be switched independently it's how to efficiently address them and move data from the display controller to the display.
(editing with some more info)
If you changed the monitor's protocol to push X, Y, pixel and souped up the on-board electronics your pixel rate would probably fall by an order of magnitude. So your 80Hz display is now an 8Hz display (for full frames). In terms of how each pixel behaves they are independent (each has it's own transistor and capacitor) but the addressing is on a grid. So you can select your row and then set a whole bunch of pixels in this row (for example)...
While there's no scanning electron beam the electronics on these displays are still optimized for scanning. That is they push a whole bunch of adjacent pixels in every clock cycle. If you try to use those for random access your refresh rate is going to fall dramatically. The problem isn't whether or not each pixel can be switched independently it's how to efficiently address them and move data from the display controller to the display.
(editing with some more info)
If you changed the monitor's protocol to push X, Y, pixel and souped up the on-board electronics your pixel rate would probably fall by an order of magnitude. So your 80Hz display is now an 8Hz display (for full frames). In terms of how each pixel behaves they are independent (each has it's own transistor and capacitor) but the addressing is on a grid. So you can select your row and then set a whole bunch of pixels in this row (for example)...
That doesn't seem to agree with http://www.fujitsu.com/downloads/MICRO/fma/pdf/LCD_Backgroun... .
What specifically do you have a problem with?
Details will vary between different displays but my point is that due to the pixels being on a grid sequential access is going to be faster. This is not unlike memory.
EDIT: Here's a reference that discusses scanning in a TFT: http://www.electronicsforu.com//EFYLinux/efyhome/cover/March...
"The TFT-LCD panel of the AMLCD is scanned sequentially line-by-line from top to bottom. Each line is selected by applying a pulse of +20V to gate line Gn, which turns on the TFTs in that specific row. Rows are deselected by applying –5V to G n-1 and G n+1, which turns off all the TFTs in the deselected rows and then the data signal is applied from the source driver to the pixel electrode. The voltage applied from the source driver, called ‘gray-scale voltage,’ decides the luminance of the pixel. The storage capacitor (CS) maintains the luminance of the pixel until the next frame signal voltage is applied. In this way, the next line is selected to turn on all the TFTs, then the data signal is fed from the source driver and hence scanning is done."
Details will vary between different displays but my point is that due to the pixels being on a grid sequential access is going to be faster. This is not unlike memory.
EDIT: Here's a reference that discusses scanning in a TFT: http://www.electronicsforu.com//EFYLinux/efyhome/cover/March...
"The TFT-LCD panel of the AMLCD is scanned sequentially line-by-line from top to bottom. Each line is selected by applying a pulse of +20V to gate line Gn, which turns on the TFTs in that specific row. Rows are deselected by applying –5V to G n-1 and G n+1, which turns off all the TFTs in the deselected rows and then the data signal is applied from the source driver to the pixel electrode. The voltage applied from the source driver, called ‘gray-scale voltage,’ decides the luminance of the pixel. The storage capacitor (CS) maintains the luminance of the pixel until the next frame signal voltage is applied. In this way, the next line is selected to turn on all the TFTs, then the data signal is fed from the source driver and hence scanning is done."
Selecting rows like that is very similar to how DRAM works, just with the word size being the number of subpixels in a row, and with no read port. Bit-level random access is inefficient, but you don't have to write to the rows in sequential order, and you don't have to update all the other rows before issuing another update for the first row. That's purely a limitation of the current driving circuitry, but a replacement like G-SYNC doesn't have to be bound by sequential rasterization any more than it has to stick to a fixed refresh rate.
But it's most likely working that way because (analog) displays always worked like that and your display technology needed to be as compatible as possible for CRTs and TFTs.
From my point of view you have not explained why it's not technologically feasible. You're merely describing that the current display tech isn't working that way.. of course not..
From my point of view you have not explained why it's not technologically feasible. You're merely describing that the current display tech isn't working that way.. of course not..
It's hard to speak about feasibility in absolute terms here. I would disagree that the current tech is the way it is simply because of the history of CRTs (though there's definitely some influence). Displays have evolved to their current technology by optimizing for things like manufacturability, price and performance. Those obviously would come ahead of CRT compatiblity.
The motivation for the gridded layout is clear I think? You have this grid of transistors and you need to address them individually. Being able to drive an entire line and then select the columns is a good and relatively cheap solution. Now you can drive all pixels in one line concurrently if you need to and the performance of a single pixel becomes less of a bottleneck. So the row/col grid structure isn't a result of needing to be compatible with CRTs... Also naturally accessing in sequence allows you to simply send the data and clock down the line. Random access would require either multiplexing the coordinates or widening your bus.
I would imagine it's possible to design a random access LCD. You would need better performing individual pixels, you will almost certainly need more layers and more conductors, you will complicated your interfaces and protocols. So you end up with a more complex and expensive system for practically little benefit. In many applications (games, videos) all pixels change every frame.
Sub-scanning a rectangular portion of the display is maybe a more reasonable target.
The motivation for the gridded layout is clear I think? You have this grid of transistors and you need to address them individually. Being able to drive an entire line and then select the columns is a good and relatively cheap solution. Now you can drive all pixels in one line concurrently if you need to and the performance of a single pixel becomes less of a bottleneck. So the row/col grid structure isn't a result of needing to be compatible with CRTs... Also naturally accessing in sequence allows you to simply send the data and clock down the line. Random access would require either multiplexing the coordinates or widening your bus.
I would imagine it's possible to design a random access LCD. You would need better performing individual pixels, you will almost certainly need more layers and more conductors, you will complicated your interfaces and protocols. So you end up with a more complex and expensive system for practically little benefit. In many applications (games, videos) all pixels change every frame.
Sub-scanning a rectangular portion of the display is maybe a more reasonable target.
Because of overhead. It's much slower to communicate "turn pixel at X,Y on" to the display for millions of individual pixels than it is to communicate "here's all the millions of pixels you need to display in sequence" in one message.
Because telling the display to turn one pixel at a time will require crazy bandwidth, and plus that's not how monitors or videocards or drivers or software work.
I'm glad to see others have been working on this. I'd been batting the idea of Hz-less displays around in my head over the past few weeks. I was actually going to write a blog post about the idea, as I was surprised that it seemed to have gone unimplemented.
I don't see much info about partial screen updates. I hope the eventual standardization of this sort of thing in DisplayPort will provide for Hz-less partial screen updates. It would be nice to be able to run a 3D application and a movie at the same time, providing 24 or 30 updates/second for the movie, and a dynamic rate for the 3D application. It also allows for cleaner implementation of 3D applications that aren't running fullscreen. Generally, partial updates seem like a more flexible, more natural evolution of the Hz-less display idea, particularly with "panel self refresh" and "partial frame updates" already in DisplayPort.
The obvious downside is that you've got a classic example of resource contention, this time in the case of bandwidth. If you've got two areas that want to be redrawn at the same time on different parts of the screen, only one (or a portion of one) can be sent at a time. This leads to jitter or (depending on how you decide to deal with it) some other visual problem. But let not the enemy of "good" be "perfect": it's still a very useful feature.
The best half-solution to the bandwidth contention problem (IMO) would be to push all the decision making as to who gets access to the "pipe" to the OS. It can decide which application (if any) gets a jitter-free experience (perhaps with user preference taken into account), it can provide hints to applications about when the pipe will be free, etc. The OS really has to manage it in order to provide a good experience.
I don't see much info about partial screen updates. I hope the eventual standardization of this sort of thing in DisplayPort will provide for Hz-less partial screen updates. It would be nice to be able to run a 3D application and a movie at the same time, providing 24 or 30 updates/second for the movie, and a dynamic rate for the 3D application. It also allows for cleaner implementation of 3D applications that aren't running fullscreen. Generally, partial updates seem like a more flexible, more natural evolution of the Hz-less display idea, particularly with "panel self refresh" and "partial frame updates" already in DisplayPort.
The obvious downside is that you've got a classic example of resource contention, this time in the case of bandwidth. If you've got two areas that want to be redrawn at the same time on different parts of the screen, only one (or a portion of one) can be sent at a time. This leads to jitter or (depending on how you decide to deal with it) some other visual problem. But let not the enemy of "good" be "perfect": it's still a very useful feature.
The best half-solution to the bandwidth contention problem (IMO) would be to push all the decision making as to who gets access to the "pipe" to the OS. It can decide which application (if any) gets a jitter-free experience (perhaps with user preference taken into account), it can provide hints to applications about when the pipe will be free, etc. The OS really has to manage it in order to provide a good experience.
Me too. I thought about it a bunch when Abrash and Carmack were discussing VR early this year, and while thinking about 48fps movies. Really, I've wanted higher refresh rates ever since the switch to LCDs, and VFR displays would make smooth, high-refresh video much easier.
Since it's unlikely I'll ever be able to get into VFR display tech professionally, for the benefit of anyone working in the industry, here are my thoughts on the subject from February (focused more on video and movies):
In the modern digital age of LCD, LED, plasma, and DLP displays, there's not much need for refresh rates to be all that high, or even constant. In the days of CRTs it was necessary to have a constantly refreshing signal coming from the video source in order to drive the electron beam across the screen and refresh the image in the phosphors without visible flicker. With film, it was much easier to run projectors and cameras at a constant rate due to their mechanical nature (and probable lack of a standard for indicating when to switch frame rates).
PROPOSAL:
I propose the development of a complete variable frame rate video chain, from camera, through production, encoding and distribution, and an HDMI/DVI-like video interface, to display devices. My primary focus of thought thus far has been on the video interface and display devices.
APPLICATIONS:
Cinema: Recently, The Hobbit was released at 24fps in some theaters, 48fps in others. As more directors want to experiment with 48fps cinema, why not remove the restriction to a fixed frame rate entirely? Initially, with widespread device support, directors and cinematographers could switch between 24fps and 48fps on a scene by scene basis. Eventually though, why not allow the frame rate to be varied continuously on a per-frame basis? Special effect sequences could be presented at 120fps, with either a gradual or an abrupt transition down to 24fps or even lower as desired for emotional or psychological effect. Variable frame rate displays would also allow mixing of NTSC, PAL, and film content without the delay of switching video signal refresh rates or using nasty time stretching or pulldown techniques. Video cameras and encoding systems could be developed that automatically adjust frame rate based on the amount of movement detected (to some extent this already exists in video codecs).
Gaming: Video games would also benefit from variable frame rate displays. No longer would gamers and game developers have to choose between a tear-free but delayed game using vertical sync, and a low-latency experience at high frame rate but with tearing. In a VFR display, the frame would be sent to the display when the game is ready, rather than the other way around. The content should be in control, not the device. This way a game could still remain artifact free if the frame rate drops, without the lag induced by waiting for the vertical blanking region before swapping buffers.
Mobile: Finally, variable frame rate devices could use much less power for signal processing and transmission; this would be especially desirable in battery-powered devices. A tablet running a word processor on an external display only needs to send an update to the screen once or twice per second to flash the cursor, saving the power that would be used for reading from video RAM and driving the display interface.
AREAS OF STUDY:
A very incomplete list of some of the considerations that must be made when developing a VFR technology suite follows:
The clock rate of an individual frame on the video display interface must be determined. Should devices negotiate an optimum clock rate upon connection and use that rate for all frames transmitted? Or should they adjust their pixel clock rate based on the current frame rate? Using the maximum possible pixel clock supported by the devices and the copper (or fiber or RF spectra) connecting them would reduce transmission-induced latency but might increase power consumption slightly.
A signaling method would need to be devised. Should the display interface protocol be verbose, with the video source announcing frame rates or frame times in advance to the display device? Or should the video card just start scanning out pixels whenever it wants to, and the display just has to deal with it?
Buffering techniques in the display would need to be considered. Should the LCD (or other) panel be updated as the pixels come in, or should a full frame be buffered first? How quickly could a buffered frame be shifted into the display panel? Given the fact that gamers can adapt to and extract greater temporal information from a higher framerate signal with tearing on a fixed rate display, there may be some benefit to scanning out the rows of an image as they arrive (the top row would be displayed a full 16ms sooner than it would otherwise).
Software would need a method of informing the video card that it's done drawing a frame. To a large extent this already exists in the form of the buffer swap call, but in a variable refresh rate system, there would be less need for double buffering to prevent artifacts. The application could draw to the scanout framebuffer as it pleases, tell the video card to send a scan to the display, then wait for the video card to notify it that the scan is done. Double buffering would still be used in games and applications that don't want to wait for one frame to finish scanning before drawing the next.
FURTHER THOUGHTS:
It would be interesting to go beyond VFR and create a display standard that can update selected regions of the screen (AKA dirty rectangles). For example, a video card could send a hypothetical "Start Scan" packet to the display device that contains the location and size of the region being updated, then stream raw pixel data that the display device fills into the updated region. For that matter, the updated region needn't be rectangular.
What about variable resolution display updates as well? This seems to make less sense in discrete pixel digital displays, but might find application when displaying low-DPI content in a subset area of a high-DPI/"retina" display.
It seems that a VFR display chain is a lot like double or triple buffering was back in the CRT days, but one of the buffers has been moved into the display device itself.
Since it's unlikely I'll ever be able to get into VFR display tech professionally, for the benefit of anyone working in the industry, here are my thoughts on the subject from February (focused more on video and movies):
Feb. 21, 2013
-------------
In the modern digital age of LCD, LED, plasma, and DLP displays, there's not much need for refresh rates to be all that high, or even constant. In the days of CRTs it was necessary to have a constantly refreshing signal coming from the video source in order to drive the electron beam across the screen and refresh the image in the phosphors without visible flicker. With film, it was much easier to run projectors and cameras at a constant rate due to their mechanical nature (and probable lack of a standard for indicating when to switch frame rates).
PROPOSAL:
I propose the development of a complete variable frame rate video chain, from camera, through production, encoding and distribution, and an HDMI/DVI-like video interface, to display devices. My primary focus of thought thus far has been on the video interface and display devices.
APPLICATIONS:
Cinema: Recently, The Hobbit was released at 24fps in some theaters, 48fps in others. As more directors want to experiment with 48fps cinema, why not remove the restriction to a fixed frame rate entirely? Initially, with widespread device support, directors and cinematographers could switch between 24fps and 48fps on a scene by scene basis. Eventually though, why not allow the frame rate to be varied continuously on a per-frame basis? Special effect sequences could be presented at 120fps, with either a gradual or an abrupt transition down to 24fps or even lower as desired for emotional or psychological effect. Variable frame rate displays would also allow mixing of NTSC, PAL, and film content without the delay of switching video signal refresh rates or using nasty time stretching or pulldown techniques. Video cameras and encoding systems could be developed that automatically adjust frame rate based on the amount of movement detected (to some extent this already exists in video codecs).
Gaming: Video games would also benefit from variable frame rate displays. No longer would gamers and game developers have to choose between a tear-free but delayed game using vertical sync, and a low-latency experience at high frame rate but with tearing. In a VFR display, the frame would be sent to the display when the game is ready, rather than the other way around. The content should be in control, not the device. This way a game could still remain artifact free if the frame rate drops, without the lag induced by waiting for the vertical blanking region before swapping buffers.
Mobile: Finally, variable frame rate devices could use much less power for signal processing and transmission; this would be especially desirable in battery-powered devices. A tablet running a word processor on an external display only needs to send an update to the screen once or twice per second to flash the cursor, saving the power that would be used for reading from video RAM and driving the display interface.
AREAS OF STUDY:
A very incomplete list of some of the considerations that must be made when developing a VFR technology suite follows:
The clock rate of an individual frame on the video display interface must be determined. Should devices negotiate an optimum clock rate upon connection and use that rate for all frames transmitted? Or should they adjust their pixel clock rate based on the current frame rate? Using the maximum possible pixel clock supported by the devices and the copper (or fiber or RF spectra) connecting them would reduce transmission-induced latency but might increase power consumption slightly.
A signaling method would need to be devised. Should the display interface protocol be verbose, with the video source announcing frame rates or frame times in advance to the display device? Or should the video card just start scanning out pixels whenever it wants to, and the display just has to deal with it?
Buffering techniques in the display would need to be considered. Should the LCD (or other) panel be updated as the pixels come in, or should a full frame be buffered first? How quickly could a buffered frame be shifted into the display panel? Given the fact that gamers can adapt to and extract greater temporal information from a higher framerate signal with tearing on a fixed rate display, there may be some benefit to scanning out the rows of an image as they arrive (the top row would be displayed a full 16ms sooner than it would otherwise).
Software would need a method of informing the video card that it's done drawing a frame. To a large extent this already exists in the form of the buffer swap call, but in a variable refresh rate system, there would be less need for double buffering to prevent artifacts. The application could draw to the scanout framebuffer as it pleases, tell the video card to send a scan to the display, then wait for the video card to notify it that the scan is done. Double buffering would still be used in games and applications that don't want to wait for one frame to finish scanning before drawing the next.
FURTHER THOUGHTS:
It would be interesting to go beyond VFR and create a display standard that can update selected regions of the screen (AKA dirty rectangles). For example, a video card could send a hypothetical "Start Scan" packet to the display device that contains the location and size of the region being updated, then stream raw pixel data that the display device fills into the updated region. For that matter, the updated region needn't be rectangular.
What about variable resolution display updates as well? This seems to make less sense in discrete pixel digital displays, but might find application when displaying low-DPI content in a subset area of a high-DPI/"retina" display.
It seems that a VFR display chain is a lot like double or triple buffering was back in the CRT days, but one of the buffers has been moved into the display device itself.
Video of the panel discussion between Carmack, Sweeney, and Johan (lead on EA's Frostbite engine):
http://www.twitch.tv/linustech/b/471263848?t=2h27m
Carmack answered some questions about it on twitter right after as well:
https://twitter.com/ID_AA_Carmack/with_replies
http://www.twitch.tv/linustech/b/471263848?t=2h27m
Carmack answered some questions about it on twitter right after as well:
https://twitter.com/ID_AA_Carmack/with_replies
Thank you for the twitch link!
Submitted this a bit earlier with a link to the actual press release, but it never made front page.
http://www.geforce.com/whats-new/articles/introducing-nvidia...
http://www.geforce.com/whats-new/articles/introducing-nvidia...
Consider another revolutionary technology from the same company, Optimus. When it first came out, it took a number of years and a public roasting by Linus Torvalds [1] to convince nVidia to officially support it on Linux.
Hopefully, with Valve and increasing numbers of indies supporting Linux, nVidia will learn from their previous mistakes and offer official Linux support for G-Sync from the beginning.
[1] http://www.phoronix.com/scan.php?page=news_item&px=MTEyMTc
Hopefully, with Valve and increasing numbers of indies supporting Linux, nVidia will learn from their previous mistakes and offer official Linux support for G-Sync from the beginning.
[1] http://www.phoronix.com/scan.php?page=news_item&px=MTEyMTc
Optimus wouldn't have a reason to exist if say Nvidia made their "mid-range" dedicated notebook GPU's to clock at 200 Mhz for "normal use", and then go to 600+ Mhz in games, or other high-end apps.
Of course, perhaps the company that led to the making of Optimus in the first place is Intel, because they started bundling their GPU's and then charging OEM's more for the standalone CPU than from the bundle - and eventually OEM's were like "why not just get both Intel's GPU, and a higher-end Nvidia one?"
If you ask me, I think Intel's move should've been declared anti-competitive from the beginning. There's no way the bundle cost Intel less than the CPU, but they priced it that way because they had a monopoly, and could force OEM's to just accept the deal "or buy the more expensive CPU if they don't like it", which was obviously a non-option option.
Of course, perhaps the company that led to the making of Optimus in the first place is Intel, because they started bundling their GPU's and then charging OEM's more for the standalone CPU than from the bundle - and eventually OEM's were like "why not just get both Intel's GPU, and a higher-end Nvidia one?"
If you ask me, I think Intel's move should've been declared anti-competitive from the beginning. There's no way the bundle cost Intel less than the CPU, but they priced it that way because they had a monopoly, and could force OEM's to just accept the deal "or buy the more expensive CPU if they don't like it", which was obviously a non-option option.
This is absolutely brilliant. Pushing the clock from software seems like it'd help the issue with syncing the encoded framerate on streaming video to the display, as well. I wonder if the non-uniform refresh of a 60fps and 24fps video on-screen at the same time would look weird.
If they're adding some cpu and a framebuffer on the display, maybe they can start doing some compression for the cable link between GPU and display -- the raw bitrate is proportional to resolution² ✕ color depth ✕ framerate but the information rate doesn't go up nearly as fast as that increases. Even simple, lossless PNG-style compression would be a huge gain on a 240hz/48bit/4k display.
If they're adding some cpu and a framebuffer on the display, maybe they can start doing some compression for the cable link between GPU and display -- the raw bitrate is proportional to resolution² ✕ color depth ✕ framerate but the information rate doesn't go up nearly as fast as that increases. Even simple, lossless PNG-style compression would be a huge gain on a 240hz/48bit/4k display.
The flip side of the coin is that as soon as you're pushing more data than the link can hold, if you encounter incompressible data (say I just generated some recursive high-frequency Perlin noise) - you would start getting stutter and lag.
Yeah, it couldn't be perfectly general. You could still keep the decoder simple and just put a little more intelligence on the GPU side to handle quantizing on the time/space/color dimensions appropriately when you try to send a pathological image. Getting the few multiples of compression you'd need to run 4k@48bit 240hz on current interfaces would probably be safe, trying to put HD over a serial cable will probably go less well.
This could also be nice for watching simple videos. It's subtle, but on smooth pans, you can see the stuttering of a 24p video on a 60Hz display. If the video player could tell the display to refresh at 47.952Hz (or any other integer multiple) instead, it would clean that right up.
Combine that with morphing frame interpolation, and you could be watching movies at just about exactly the rate your hardware can manage to push them out.
Combine that with morphing frame interpolation, and you could be watching movies at just about exactly the rate your hardware can manage to push them out.
It is kind of surprising that CRTs have been dead for so long but we're still driving our displays in pretty much the same fashion, except with reduced blanking intervals. We still treat video connections as constant data-rate streams, when we should be bursting frames to the monitor as soon as they finish rendering and letting the monitor worry about when it will be able to update which part of the screen.
That's how it would work with OLED, right? So we should have that in a few years once density and cost catch up to traditional LED-backed LCDs.
That's how it could work on any of the common display technologies that don't use a single electron beam tracing across the screen. Active-matrix displays - be they OLED or TFT LCD - don't require that pixels be updated in any particular order or at any specific frequency save for the minimum update frequency that is analogous to DRAM refreshing.
The way we currently send pixel data to monitors is basically optimized to make the monitor's electronics as simple as possible and to minimize the bandwidth used at all times, even if the hardware is capable of communicating at higher speeds. Just simply changing DisplayPort to always send pixel data at the highest speed even when operating at less than the maximum resolution supported by that link would result in a significant reduction in latency, by no longer taking 16ms to send each frame (which almost all monitors fully buffer in order to apply color transformations or scaling). The next step after that would be to allow frames to start on irregular intervals, which is apparently what NVidia's implementing. But it's still all just about how the contents of the GPU's framebuffer are transmitted to the monitor's framebuffer, and is in no way dependent on what kind of technology is downstream of the monitor's framebuffer.
The way we currently send pixel data to monitors is basically optimized to make the monitor's electronics as simple as possible and to minimize the bandwidth used at all times, even if the hardware is capable of communicating at higher speeds. Just simply changing DisplayPort to always send pixel data at the highest speed even when operating at less than the maximum resolution supported by that link would result in a significant reduction in latency, by no longer taking 16ms to send each frame (which almost all monitors fully buffer in order to apply color transformations or scaling). The next step after that would be to allow frames to start on irregular intervals, which is apparently what NVidia's implementing. But it's still all just about how the contents of the GPU's framebuffer are transmitted to the monitor's framebuffer, and is in no way dependent on what kind of technology is downstream of the monitor's framebuffer.
Sounds great. But it should be in a standard, keeping this proprietary to nvidia means it will only be available on a limited number of monitors and cross-fingers AMD doesn't do their own version.
Put it in the next HDMI spec (and the rev the spec fast, not another 2 years or so).
Put it in the next HDMI spec (and the rev the spec fast, not another 2 years or so).
I'm curious how similar this is to Embedded DisplayPort's panel self refresh feature. It would be neat to be able to run the display at exactly 24 FPS for movies, down lower when staring at code (to save power), and then dial it up to 100+ Hz for scrolling and animations.
I've never understood what is so difficult about making software that can generate frames at a fixed rate, and I don't understand this product- I don't understand how having a variable refresh rate would do anything other than harm smoothness, and encourage more bad software.
Maybe someone can clarify this for me, but what is so wrong with writing software that can just meet the frame deadline? Maybe the hardware innovation should be hardware and drivers that helps you do vertical sync more reliably?
Maybe someone can clarify this for me, but what is so wrong with writing software that can just meet the frame deadline? Maybe the hardware innovation should be hardware and drivers that helps you do vertical sync more reliably?
If your renderer can draw 40fps in the highest density scenes (lots of polygons, particles, and effects) on specific hardware, then that is the most you'll be able to guarantee to that user without sacrificing detail. That will be your "fixed frame rate".
However, in simpler scenes the same renderer is likely to output significantly more frames (even up to thousands) at the same detail level.
So in this scenario, all you're doing by setting a fixed rate is throwing away tons of frames in low density scenes. Most gamers would prefer to have the most fps possible at any given moment, even if it means variability.
The most hardcore gamers I know use 120hz monitors, and machines that can deliver 241fps (120 * 2 + 1) in the highest detail scenes. They then set the engine to cap frames at 241fps which will eliminate tearing, negating the need for this technology. However, their gaming machines cost a LOT, so this would deliver similar results for a much wider range of hardware.
However, in simpler scenes the same renderer is likely to output significantly more frames (even up to thousands) at the same detail level.
So in this scenario, all you're doing by setting a fixed rate is throwing away tons of frames in low density scenes. Most gamers would prefer to have the most fps possible at any given moment, even if it means variability.
The most hardcore gamers I know use 120hz monitors, and machines that can deliver 241fps (120 * 2 + 1) in the highest detail scenes. They then set the engine to cap frames at 241fps which will eliminate tearing, negating the need for this technology. However, their gaming machines cost a LOT, so this would deliver similar results for a much wider range of hardware.
If you're generating thousands of frames and throwing them away, you wrote your rendering software wrong. I ask again, what is so impossible about just rendering the 60fps (just a bit under the limit of the rate at which a human is able to perceive any difference), and then not rendering any more? instead of rendering as fast as you can, do the different trade of of always meeting the deadline.
sigh why am I explaining this again? is it really hard to understand? why?
sigh why am I explaining this again? is it really hard to understand? why?
Your question was answered; it has nothing to do with the case where you're "generating thousands of frames and throwing them away". What you want is a rendering engine that will perform at 60fps in the worst case. What engine devs want to write is an engine that can do better (even much much better) than 60fps in the average case, and be allowed to slip in those pathological cases. Gamers want more frames. More frames than is noticeable. They want some slack so that if something totally unrelated to the game ties up the machine, the framerate drop is not noticeable. They want to be able to double it so that they can drive a 3D display but still have the same effective framerate per eye.
Having a consistent 40fps is much worse (for a gamer) than a variable framerate that will dip down to 40fps for 1% (or 10%) of the play time. Having to limit your most complex scene to what can be guaranteed rendered at 60fps is much less appealing to a developer than making sure all the likely scenes can render at 60fps.
Having a consistent 40fps is much worse (for a gamer) than a variable framerate that will dip down to 40fps for 1% (or 10%) of the play time. Having to limit your most complex scene to what can be guaranteed rendered at 60fps is much less appealing to a developer than making sure all the likely scenes can render at 60fps.
> Having a consistent 40fps is much worse (for a gamer) than a variable framerate that will dip down to 40fps for 1%
stuttering animations is better than smooth animations?
Stuttering is better than smooth. gotcha.
> Your question was answered;
for someone who doesn't appear to understand what I'm asking you have a high degree of confidence that I've been answered.
What is so terrible about having a lower complexity budget that guarantees 60fps? What if you had 60 fps no exceptions as a constraint in your hardware and software design, how far could you really go with some creativity? Think about it- is having a complexity ceiling the only possible way to ensure 60fps?
stuttering animations is better than smooth animations?
Stuttering is better than smooth. gotcha.
> Your question was answered;
for someone who doesn't appear to understand what I'm asking you have a high degree of confidence that I've been answered.
What is so terrible about having a lower complexity budget that guarantees 60fps? What if you had 60 fps no exceptions as a constraint in your hardware and software design, how far could you really go with some creativity? Think about it- is having a complexity ceiling the only possible way to ensure 60fps?
>What is so terrible about having a lower complexity budget that guarantees 60fps
>Think about it- is having a complexity ceiling the only possible way to ensure 60fps?
I'm really not following you. Are you asking what is the benefit of this technology when games come out every day at 60fps even now? This technology allows them to get the same fidelity and smoothness on less powerful hardware, with more complicated simulations, and lower latency.
>Think about it- is having a complexity ceiling the only possible way to ensure 60fps?
I'm really not following you. Are you asking what is the benefit of this technology when games come out every day at 60fps even now? This technology allows them to get the same fidelity and smoothness on less powerful hardware, with more complicated simulations, and lower latency.
>I ask again, what is so impossible about just rendering the 60fps
Are you asking why renders can't maintain perfectly steady frame rates without going above or below 60fps, regardless of WHAT they are rendering on screen or what is happening in the simulation? You can't see how that's a 'non trivial' problem?
The most notable example of a game using dynamic tradeoffs to maintain a solid 60FPS is ID's Rage engine -- written by Johh Carmack, one the people on stage at this very presentation, who was lauding this technology and saying he has been pushing GPU and monitor manufacturers to implement this for years.
Carmack notes that while they were able to stay at 60 with an incredible amount of work, if they had been able to target 90% of 60fps with this technology there would have been little visual difference but the gameplay and visual complexity ceiling would have been vastly higher.
Are you asking why renders can't maintain perfectly steady frame rates without going above or below 60fps, regardless of WHAT they are rendering on screen or what is happening in the simulation? You can't see how that's a 'non trivial' problem?
The most notable example of a game using dynamic tradeoffs to maintain a solid 60FPS is ID's Rage engine -- written by Johh Carmack, one the people on stage at this very presentation, who was lauding this technology and saying he has been pushing GPU and monitor manufacturers to implement this for years.
Carmack notes that while they were able to stay at 60 with an incredible amount of work, if they had been able to target 90% of 60fps with this technology there would have been little visual difference but the gameplay and visual complexity ceiling would have been vastly higher.
It's "non trivial" is all you have?
I'm asking someone to explain this. because I don't get it.
Isn't meeting the frame deadline kind of more important than scene complexity?
I'm asking someone to explain this. because I don't get it.
Isn't meeting the frame deadline kind of more important than scene complexity?
Look into what is involved in modern 3D rendering of high-detail scenes. It is NON-TRIVIAL, and I can tell you this as someone who has done 3D programming for 17 years.
Pro Tip: If an entire industry of experienced people finds something very hard, and you don't know anything about the topic but you don't see why it would be hard, maybe the relevant factor here is the "you don't know."
It reminds me of my mom who said on multiple occasions "All these rockets are dangerous and they explode; I don't see why the scientists don't just use the majestic forces that keep the planets in their orbits to move the rocket."
Pro Tip: If an entire industry of experienced people finds something very hard, and you don't know anything about the topic but you don't see why it would be hard, maybe the relevant factor here is the "you don't know."
It reminds me of my mom who said on multiple occasions "All these rockets are dangerous and they explode; I don't see why the scientists don't just use the majestic forces that keep the planets in their orbits to move the rocket."
Yes, I don't know. that's what I am saying.
I am not in this thread saying "You're wrong and I'm right", and I'm not asking you to say to me "no, you're wrong"
I am asking you to explain it*. not just say "well, it's hard, and this guy and this industry says it's hard" and call it a day.
do you understand? "because it's hard" is not an interesting answer. It's a boring and contentless answer.
do you understand? "because it's hard" is not an interesting answer. It's a boring and contentless answer.
This has to do with the way the software interacts with hardware on PC.
Basically, a GPU is a very complex computer with several hierarchies of execution streams e.g. there are vector SIMD streams that execute same code over different data, there are threads of such streams that preempt each other, there multiple processing units each running a set of such threads, and there are even structures of such processing units. Yet all of this is hidden from the programmer and only API there is an abstract "scene" description. E.g. you can say "first render these polygons with such and such settings, then render other polygons with other settings, then show what is rendered so far and return to the default state".
Going from such a high level description to the thousands of execution streams that GPU will execute is a very complex procedure that changes with each driver version and is not fully understood by any single person. On top of this you have other processes running on your machine while playing the game and they can and will steal CPU and the OS scheduling slots, adding a lot of variance to your frame time.
You can render the same data set several times and sometime it will take 10ms but other times it will take 100ms depending on what other processes decided to do at the time so it's impossible to guarantee constant frame time on PC.
On consoles it's not a big deal as you can program the GPU directly and don't compete with other processes. A great number of games do run with constant frame rate - it's not trivial but it's not a rocket science either.
Basically, a GPU is a very complex computer with several hierarchies of execution streams e.g. there are vector SIMD streams that execute same code over different data, there are threads of such streams that preempt each other, there multiple processing units each running a set of such threads, and there are even structures of such processing units. Yet all of this is hidden from the programmer and only API there is an abstract "scene" description. E.g. you can say "first render these polygons with such and such settings, then render other polygons with other settings, then show what is rendered so far and return to the default state".
Going from such a high level description to the thousands of execution streams that GPU will execute is a very complex procedure that changes with each driver version and is not fully understood by any single person. On top of this you have other processes running on your machine while playing the game and they can and will steal CPU and the OS scheduling slots, adding a lot of variance to your frame time.
You can render the same data set several times and sometime it will take 10ms but other times it will take 100ms depending on what other processes decided to do at the time so it's impossible to guarantee constant frame time on PC.
On consoles it's not a big deal as you can program the GPU directly and don't compete with other processes. A great number of games do run with constant frame rate - it's not trivial but it's not a rocket science either.
thank you for your detailed and thoughtful answer. This makes a lot of sense.
Well, I left the start of an explanation in another reply. But the problem is that you are asking a question where it takes years to really understand the answer, and certainly hours for a commenter to write a summary. For someone to put that much effort in, they have to be motivated to put in the effort; but you are coming across in a very unpleasant way, and not offering anything in return, so why would anyone put in the effort just for you?
I have no control over how you perceive my posts. I write them in a neutral way, produce text, and it's up to you to read in a tone of voice and mood. You are free to perceive anything about them you want. You are certainly welcome to refuse to reply to them and you always were. I was apparently pleasant enough for you to give glib responses but not enough to reply with anything much of substance more than "you're asking a stupid question and you can never be as smart as me HAHAHA!!" I mean, if that's your answer then you probably are better off not responding to me at all, leaving my question hanging, and not polluting the conversation with such negativity.
As for the start of an answer, what was wrong with just .. starting with that? What was the point of all the other stuff you wrote? Think about it. what is your mission here? to be informative or just to attempt to make me feel bad for being curious?
As for the start of an answer, what was wrong with just .. starting with that? What was the point of all the other stuff you wrote? Think about it. what is your mission here? to be informative or just to attempt to make me feel bad for being curious?
The mission is to make you a little more self-aware of how your posts are being read and perceived. "why am I explaining this again? is it really hard to understand? why?" does not sound like a neutral tone to most people, it sounds condescending, which is confusing when (as you admit) you are the one who is asking for an explanation for something you do not understand.
Yes I don't understand something and I'm asking for an explanation. If you, or someone else does not understand my actual question, and has nothing more to contribute than "It's hard" then I am not interested in their answers or their condescension and I couldn't give two tosses if you turn around and perceive me as being condescending. It's projection. And I wasn't asking you anyway. But since I'm here, what is the point of YOUR post? I don't need more self awareness, people on hacker news need to stop answering questions they don't understand with bullshit nonsense condescension and getting pissy when the victim of their idiocy gets annoyed by it.
Seriously I will not tolerate this curiosity shaming, the ethic that one should feel embarrassed about asking questions. I do not buy the idea that it is condescending to be dissatisfied with shallow lazy nothing answers. I have no control over you perceiving it that way. It is just a flaw in your background that you should perhaps be more self aware about.
Seriously I will not tolerate this curiosity shaming, the ethic that one should feel embarrassed about asking questions. I do not buy the idea that it is condescending to be dissatisfied with shallow lazy nothing answers. I have no control over you perceiving it that way. It is just a flaw in your background that you should perhaps be more self aware about.
You are being rude to people that you are asking a favor from. Stop blaming everyone else; isolate the common factor. They're not shaming you because of the asking itself.
Why don't you look for some writeup about challenges during Rage development and tradeoffs that were needed to keep it at constant 60fps?
Meeting the frame deadline is a function of scene complexity.
>I ask again, what is so impossible about just rendering the 60fps, and then not rendering any more?
There seem to be two ways to interpret your question:
>1.) Why can't games render 60 frames per second always?
Because some scenes are more complex than others. Rendering a complex scene can longer than 16.7 ms, and there is no way around that.
>2.) If a game comes far below the frame completion deadline (e.g., completes a frame in 0.5 ms, where the deadline is 16.7 ms), why doesn't it simply stop doing anything until the deadline has passed?
I do not know the answer. But I can say that many games actually do this. It is usually referred to as a frame cap.
There seem to be two ways to interpret your question:
>1.) Why can't games render 60 frames per second always?
Because some scenes are more complex than others. Rendering a complex scene can longer than 16.7 ms, and there is no way around that.
>2.) If a game comes far below the frame completion deadline (e.g., completes a frame in 0.5 ms, where the deadline is 16.7 ms), why doesn't it simply stop doing anything until the deadline has passed?
I do not know the answer. But I can say that many games actually do this. It is usually referred to as a frame cap.
Thanks I think this is an insightful interpretation of my question. I think I have been satisfactorily answered elsewhere on the thread. It totally makes sense now, but on the other hand I still don't see how this tech helps much. It would improve things marginally, in theory but it seems like a bandaid to what you really want, which is a steady stream of animation frames. it feels like a concession to the impossibility of creating rich smooth realtime animation in the presence of a multiprocess operating system.
60 fps means a budget of 16.6 ms per frame. If it takes you 16.7 ms per frame, you don't get 59.8 fps - you get 30 fps when vsync is enabled. So renderers in practice have to get a good way under that time budget to get a reliable 60 fps.
Furthermore, 60 fps is not "just under the limit" for what a human can see; read e.g.:
http://www.100fps.com/how_many_frames_can_humans_see.htm
Furthermore, 60 fps is not "just under the limit" for what a human can see; read e.g.:
http://www.100fps.com/how_many_frames_can_humans_see.htm
On consoles you used to get a vsync interrupt. A reliable signal-- an event which you could control very precisely from your game software.
Nowadays when I program games, I cannot get a promise that a frame event will fire. I can get a "well, this code may run, unless the operating system needs those cycles".
So, so is skipping a whole frame, OR halving the frame rate really the best that is possible? Why is it either/or? What are operating systems, hardware vendors and driver writers doing about it?
Nowadays when I program games, I cannot get a promise that a frame event will fire. I can get a "well, this code may run, unless the operating system needs those cycles".
So, so is skipping a whole frame, OR halving the frame rate really the best that is possible? Why is it either/or? What are operating systems, hardware vendors and driver writers doing about it?
The problem is that you can only start sending a frame to the display at 16.67ms intervals. If you miss that deadline, you can either swap buffers now (causing tearing), or you can swap buffers at the next vertical blank interval (which results in an effictive 30Hz refresh rate). These are the only two options that are supported by current displays. There's nothing you can do on the computer side to get around these constraints, because the limitation comes from display connections like VGA, HDMI, etc. that only deal in fixed refresh rates. Trying to drive such connections with a variable refresh rate would be like changing the baud rate of a serial connection on the fly without any way to let the receiving device know about the change.
This is an interesting point. The point is made elsewhere in the thread that if you miss the deadline, but send what you have drawn so far anyway, with modern typical rendering software/hardware you'd get an incomplete drawing with holes and missing layers etc.
But that is not what tearing is. Tearing is, you've missed the deadline, so you finish the rendering to completion, then swap the buffer in the middle of a vertical scan. What you're saying is, you can do that, or wait until the next vertical scan before swapping the buffer. And further more, that can either result in simply a skipped frame, or the game switches down to a 30fps mode, perhaps based on some running statistical about frame render durations.
I'm reminded about a discussion Carmack (was it him? or am I having a brain fart) about mitigating the tearing and framerate problem by doing per scanline rendering instead of per frame rendering.
hmmm! I will have a long think
But that is not what tearing is. Tearing is, you've missed the deadline, so you finish the rendering to completion, then swap the buffer in the middle of a vertical scan. What you're saying is, you can do that, or wait until the next vertical scan before swapping the buffer. And further more, that can either result in simply a skipped frame, or the game switches down to a 30fps mode, perhaps based on some running statistical about frame render durations.
I'm reminded about a discussion Carmack (was it him? or am I having a brain fart) about mitigating the tearing and framerate problem by doing per scanline rendering instead of per frame rendering.
hmmm! I will have a long think
is this what you are looking for? http://www.hialgo.com/main.html
Huh?
supposedly - "HiAlgoBoost changes the resolution on the fly in order to increase frame rate (FPS). "
Meeting the frame deadline is about scene complexity, not code. Keeping scene complexity under the limit is nearly impossible in large, rich game worlds. On small console games it is done by putting in walls and strictly limiting the visual budget, but that would remove much of the appeal of a game like Skyrim.
Skyrim exists on at least one console. Isn't it possible to make a game with a similar level of richness of skyrim within a budget? can hardware help with enforcing the budget? I intuit that there must be some kind of trademark you can make to get under the deadline- like for instance, perhaps, progressive rendering of a frame, so that if the frame doesn't finish in time you get a half resolution frame, instead of just the top half of a frame?
See my Rocket comment above. But in reply to this specific comment I will drop you a hint (this hint is still just a small piece of the whole situation):
3D rendering is so deeply pipelined that it is difficult or even impossible for the program to know if a frame render is going to finish on time. It takes a long time to get information about completed results back from a GPU; on PCs you almost certainly can't get that info during the same frame you are rendering, unless you are rendering tremendously slowly.
In order to make an estimate about whether the frame is going to be done in time, you would have to guess. Okay, then, so now you decided to stop rendering this frame, what do you do? Leave a giant hole in the scene? Turn off the postprocess? Draw low-detail versions of some things (hint: still very slow)?
Your program does not even really know for sure which pieces of the scene are fast to render and which are slow. It does not know if specific textures are going to be paged out of VRAM by the time you get to a specific mesh, or not. etc etc
3D rendering is so deeply pipelined that it is difficult or even impossible for the program to know if a frame render is going to finish on time. It takes a long time to get information about completed results back from a GPU; on PCs you almost certainly can't get that info during the same frame you are rendering, unless you are rendering tremendously slowly.
In order to make an estimate about whether the frame is going to be done in time, you would have to guess. Okay, then, so now you decided to stop rendering this frame, what do you do? Leave a giant hole in the scene? Turn off the postprocess? Draw low-detail versions of some things (hint: still very slow)?
Your program does not even really know for sure which pieces of the scene are fast to render and which are slow. It does not know if specific textures are going to be paged out of VRAM by the time you get to a specific mesh, or not. etc etc
So you are saying it's completely impossible to start out just rendering the lowest complexity scene and progressively refine it, so that if you stop at any point in time, you still have something reasonable to show for it? And that GPU manufacterers have been working on making steady frame rates more and more difficult instead of easier?
What you are suggesting could lead to horrific flicker. If you render one frame in low complexity follow by another frame at medium complexity, followed by another frame in low complexity... etc., you'd get flashing as (for example) shadows appeared and disappeared repeatedly.
Well, wouldn't that depend on what you mean by "low complexity" ? That's what we mean nowadays, but could you not design a version of "low complexity" that reduces the appearance of flicker?
The problem is that if the low quality render differs in any appreciable way from the high quality one, there will be flickering. So the low quality renders have to be extremely similar to the high quality ones, in which case why not just use the low-quality ones all the time?
Though come to think of it, there is one example that almost does what you describe. Dynamic resolution scaling has some artifacts, but is being used in some games (and is notably used in Dead Rising 3). Though one has to decide before the frame is rendered what resolution to use, so you still get frames that take longer than 16.7ms or whatever your target is.
One could do something similar with render quality, but it has the same drawback that you have to decide beforehand what quality you want to use. One would also have to ramp up and down the quality slowly, which is difficult as the complexity of a given scene can vary wildly over the space of a single second. It also wouldn't help with spikes in rendering time.
Though come to think of it, there is one example that almost does what you describe. Dynamic resolution scaling has some artifacts, but is being used in some games (and is notably used in Dead Rising 3). Though one has to decide before the frame is rendered what resolution to use, so you still get frames that take longer than 16.7ms or whatever your target is.
One could do something similar with render quality, but it has the same drawback that you have to decide beforehand what quality you want to use. One would also have to ramp up and down the quality slowly, which is difficult as the complexity of a given scene can vary wildly over the space of a single second. It also wouldn't help with spikes in rendering time.
I like this post (or at least the direction it begins to take at the end).
"One could do something similar with render quality. There are possible solutions, and I'm thinking creatively about it."
Thank you for taking the time to answer me. <3
"One could do something similar with render quality. There are possible solutions, and I'm thinking creatively about it."
Thank you for taking the time to answer me. <3
I think you have a misapprehension that rendering is or can be done via progressive enhancement and that's never really been a direction games have pushed into.
Rendering a scene is kind of like constructing a building. If you stop at an arbitrary point in the construction process, you don't get a pretty good building, you get a non functional structure.
The real reason is that there might possibly be some way of doing a progressively enhanced, rendered game but it's never been high enough priority for anyone to have done serious work in that area.
Rendering a scene is kind of like constructing a building. If you stop at an arbitrary point in the construction process, you don't get a pretty good building, you get a non functional structure.
The real reason is that there might possibly be some way of doing a progressively enhanced, rendered game but it's never been high enough priority for anyone to have done serious work in that area.
This is an interesting point, and I can kind of see the outline of some vaguely political reasoning here. Let us clarify that we are talking about realtime rendering for games. Of course, rendering in general has been done in all sorts of different ways, and it's pretty straightforward to demonstrate a raytracer that progressively enhances is rendering over time.
The point is, this isn't a direction that cutting edge games programming has gone in, because getting more detail into the game has always been a higher priority than having a steady predictable framerate- And to make a further point, practically speaking, painter's/z-buffer algorithm is easier to optimise than some other rendering algorithms. Though, the others are not impossible, just not fruit that is quite as low hanging.
The point is, this isn't a direction that cutting edge games programming has gone in, because getting more detail into the game has always been a higher priority than having a steady predictable framerate- And to make a further point, practically speaking, painter's/z-buffer algorithm is easier to optimise than some other rendering algorithms. Though, the others are not impossible, just not fruit that is quite as low hanging.
Yes, it is not possible. Learn about what a Z Buffer is and how it works.
Dude seriously, I have been doing this a long time. I am going to stop replying after saying just one more thing:
If you are running on an OS like Windows (which this product is targeted at), you do realize that the OS can just preempt you at any time and not let you run? How do you predict if you are going to finish a frame if you don't even know how much you will be able to run between now and the end of the frame?
Dude seriously, I have been doing this a long time. I am going to stop replying after saying just one more thing:
If you are running on an OS like Windows (which this product is targeted at), you do realize that the OS can just preempt you at any time and not let you run? How do you predict if you are going to finish a frame if you don't even know how much you will be able to run between now and the end of the frame?
I am not talking about predicting in advance, I am talking about just doing as much as you can, and then when the deadline comes, sending the result. With the way software is written now this results in tearing, because pixels are simply rendered left to right, top to bottom. but what if you rendered them in a different order such as with an interlaced jpeg, which would show you a low res version of the image when partially downloaded.
The render pipeline is more or less, compute a bunch of geometry on the cpu, send it to the gpu with some textures and other things, tell it to render, and once the gpu has rendered the frame, you can tell it when to switch.
There's no point at which you have a usable partial frame to display, and it doesn't make sense to compute every other pixel, and if there's time come back and get the rest, because computing a pixel's neighbor will need a lot of the same intermediate work, and you probably don't have the resources to keep that. Parallel rendering generally divides the rendering tasks into different regions of the screen for each unit, not interleaved pixels.
To answer your question regarding why not just use 40fps, instead of going up and down; If you cap framerate at 40fps and your monitor doesn't refresh at an even multiple of 40, you're going to end up with consistent judder which is probably worse than occasional framerate dropping.
If you look at it from the other side, look at all the extra buffers that are needed to support fixed framerate monitors when frame generation really doesn't need to be fixed. At the desktop, if nothing on the screen moves, there's no need to transmit the buffer 60x per second, except for legacy reasons, in a graphics intensive application, the time to generate a buffer may vary. Video usually was recorded at fixed frequency, but it often doesn't match the frequency of the monitor. CRTs absolutely required that the electron beam trace every pixel several times a second, but LCDs don't.
There's no point at which you have a usable partial frame to display, and it doesn't make sense to compute every other pixel, and if there's time come back and get the rest, because computing a pixel's neighbor will need a lot of the same intermediate work, and you probably don't have the resources to keep that. Parallel rendering generally divides the rendering tasks into different regions of the screen for each unit, not interleaved pixels.
To answer your question regarding why not just use 40fps, instead of going up and down; If you cap framerate at 40fps and your monitor doesn't refresh at an even multiple of 40, you're going to end up with consistent judder which is probably worse than occasional framerate dropping.
If you look at it from the other side, look at all the extra buffers that are needed to support fixed framerate monitors when frame generation really doesn't need to be fixed. At the desktop, if nothing on the screen moves, there's no need to transmit the buffer 60x per second, except for legacy reasons, in a graphics intensive application, the time to generate a buffer may vary. Video usually was recorded at fixed frequency, but it often doesn't match the frequency of the monitor. CRTs absolutely required that the electron beam trace every pixel several times a second, but LCDs don't.
I only used 40fps as an example. I know you'd probably want 60fps, 30fps or 20fps
(or possibly 50 or 25 if your refresh rate has more of a PAL bent)
Here's an idea I am curious about now. If you can almost but not quite reliably generate full res images at 60 FPS, can you generate quarter resolution (that is, half the pixels on each dimension for a quarter of the pixels) at 240 fps, or does the overhead for each render outstrip the efficiency from generating fewer pixels? That is, how much of a fixed overhead for a frame is there, and can it be spread over 4 frames, with slightly offset camera?
Here's an idea I am curious about now. If you can almost but not quite reliably generate full res images at 60 FPS, can you generate quarter resolution (that is, half the pixels on each dimension for a quarter of the pixels) at 240 fps, or does the overhead for each render outstrip the efficiency from generating fewer pixels? That is, how much of a fixed overhead for a frame is there, and can it be spread over 4 frames, with slightly offset camera?
3D graphics images aren't calculated left to right, top to bottom. They are calculated by drawing a bunch of triangles all over the image to represent 3D geometry. Triangles are often drawn on top of other triangles. Many modern games also use multipass rendering to achieve certain lighting and special effects. Only after a whole image is computed can it be transferred to the monitor if you want the image to make sense. If you stop rendering half way through, the end result would be objects full of holes with entire features missing or distorted. The time needed for the actual transfer of the image to the display is generally a drop in the bucket by comparison.
Well, while you are right for the way games and GPUs currently work, there is more than one 3d rendering algorithm- Zbuffer is not the end all be all. The question is what can be rendered efficiently with the hardware we have. At some point we could collectively decide that having a rock solid framerate is more important than more detail, decide to use a scanline progressive renderer, and there you go. It is possible, but would we do it?
But on the other hand I was confused- I forgot that zbuffer was the algorithm still in use in most game rendering engines. and you cleared that up. thanks.
But on the other hand I was confused- I forgot that zbuffer was the algorithm still in use in most game rendering engines. and you cleared that up. thanks.
This also looks really good for video playback (as in movies). It would avoid stuttering from pulldown. I am quite hyped about this.
This holds some potential for media playback as well, since matching display frame-rate to source material is usually necessary to avoid jerky playback (Some people are very sensitive to any kind of micro-stutter while others don't seem to notice). Of course, playback software will have to support it (easy) and it may be a while before HDTV's or projectors include Nvidia's chip, if that even happens at all! However, if it does happen, this will effectively signal the end of adjusting frame-rate on displays. Even simply allowing higher refresh rates than 60 Hz could clear up a lot of problems.
I don't think Displayport has sufficient bandwidth for 4K video at 144 Hz unfortunately. An upgrade on Displayport may be necessary. Given that HDCP has proven to be utterly pointless, one would hope DRM gets the boot. Encrypting and decrypting data at over 20 Gbit/s is probably a PITA when you're aiming for low latency and low cost.
I don't think Displayport has sufficient bandwidth for 4K video at 144 Hz unfortunately. An upgrade on Displayport may be necessary. Given that HDCP has proven to be utterly pointless, one would hope DRM gets the boot. Encrypting and decrypting data at over 20 Gbit/s is probably a PITA when you're aiming for low latency and low cost.
What exactly does the extension card do? 1920x1080 24-bit pixels is 6MB, so why does it need 768MB of on-board memory?
There are games that can use up 2GB of on-board GPU memory right now, for textures/buffers/etc. It won't be difficult to use up as much as 8 in the near future when panel resolutions get higher - higher panel resolution means a demand for higher resolution textures.
Given this, I'm sure there are things they can do with that onboard memory in the display. Maybe buffer up a half dozen video frames with timing data for smooth high res video playback?
Given this, I'm sure there are things they can do with that onboard memory in the display. Maybe buffer up a half dozen video frames with timing data for smooth high res video playback?
But we aren't moving the gpu to the screen.
With it's currently advertised feature set, there is no need for that much ram.
Assuming future-proofing support for 4k monitors with framebuffers in 16bit floating point format, That's still only 48mb per buffer and there is no reason to have more than two buffers in a screen.
With it's currently advertised feature set, there is no need for that much ram.
Assuming future-proofing support for 4k monitors with framebuffers in 16bit floating point format, That's still only 48mb per buffer and there is no reason to have more than two buffers in a screen.
Maybe they were 256 megabit chips.
The resolution of textures has absolutely nothing to do with the fact that the size of the framebuffer is only 1920x1080x24 bits large,so 6075KB(5.93MB). Unless you are trying to imply that the memory in the monitor will be used as an extension of the GPUs memory, which I really don't see how it would work over existing connections and deal with the latency and relatively low bandwidth involved.
But since the primary market for this is gaming, you can't buffer up anything or you kill latency, which is even more important than smoothness.
So is the idea here that the buffer is actually on the monitor now and no longer on the gpu? Is that why there is 3x256MB memory on the controller? Perhaps the gpu just pushes bits across the wire as soon as its ready now? And the monitor is now responsible for maintaining a localize triple buffer mechanism?
The idea is that the monitor will only refresh after it is given a frame to render from the graphics card, which makes the monitor sync to the framerate of the graphics card, not the other way around. From what I can tell, this basically means instead of having a fixed refresh rate, the modified monitor now has a variable refresh rate, slaved to the output of the GPU.
Right, but a traditional 1920x1080 monitor only needs ~50MB of memory to store a frame of data to flush to the screen. So why introduce 3x256MB of memory on the controller? Unless the idea is to move away from managing frame buffers on the gpu.
Could this be a bits versus bytes issue? a 256MB ram chip typically is 256 megabits (and a typical memory module would have some multiple of 8 of them). 3x256Megabits is 96 Megabytes, which still seems like a lot, but might (1920 x 1080 x 32-bit = a little less than 8 megabytes, unless my math is way wrong)
Am I missing something, or this would make a real difference only if you monitor has a low top refresh rate? If the refresh rate is 144 Hz, like the one they talk about in the article, I wouldn't expect that waiting for vsync will have big impact on the fps.
If you running at 144 the frame rate drops to 72 whenever the game can't maintain it. Carmack says when this happens in VR it's like being kicked in the head but they can't disable Vsync because tearing is even worse.
Unless you triple buffer but which adds its own problems, including more latency. But this tech is also about the feel -- even at the 'same' frame rate the subjective impression is of a much smoother game. For obvious reason this can not be captured in a video so it's tricky to sell, they have to show it you in person.
Their slide showing vsync lag:
http://images.anandtech.com/doci/7432/NVMontreal-097.jpg
Anand has some of his personal impressions posted already:
http://www.anandtech.com/show/7436/nvidias-gsync-attempting-...
"I can't stress enough just how smooth the G-Sync experience was, it's a game changer."
Unless you triple buffer but which adds its own problems, including more latency. But this tech is also about the feel -- even at the 'same' frame rate the subjective impression is of a much smoother game. For obvious reason this can not be captured in a video so it's tricky to sell, they have to show it you in person.
Their slide showing vsync lag:
http://images.anandtech.com/doci/7432/NVMontreal-097.jpg
Anand has some of his personal impressions posted already:
http://www.anandtech.com/show/7436/nvidias-gsync-attempting-...
"I can't stress enough just how smooth the G-Sync experience was, it's a game changer."
72 FPS looks very high to me, but I never tried VR. Would be curious to see the difference there.
The first ingredient of the answer is that for all sorts of video (including games) to look smooth, it is important to know exactly when a frame is going to be displayed on the screen.
The second ingredient of the answer is that, with a fixed monitor refresh rate, you're basically forced to run at a divisor of that refresh rate.
So, if your game usually runs at 72Hz (on a 144Hz monitor) and you get to a busier section, the framerate has to drop down to 36Hz, even when 60Hz would still be possible in terms of CPU and GPU power.
The second ingredient of the answer is that, with a fixed monitor refresh rate, you're basically forced to run at a divisor of that refresh rate.
So, if your game usually runs at 72Hz (on a 144Hz monitor) and you get to a busier section, the framerate has to drop down to 36Hz, even when 60Hz would still be possible in terms of CPU and GPU power.
I get the divisor thing (if you're not doing triple buffering), but I don't get why it should be divided by a power of two. After 72Hz it should be 48Hz (still very good) and only then 36Hz (still not bad I think).
Anyway, I would be curious to try to see if I could notice a difference. Math can take us just so far :)
Anyway, I would be curious to try to see if I could notice a difference. Math can take us just so far :)
G-Sync syncs the monitors refresh rate with the FPS your graphics card is feeding the monitor - thus removing screen tearing and other anomalies that happen when the refresh rates are out of sync.
It doesn't make a difference if your card is renderring at 60 fps with out a hitch, but you can see the difference when the fps drop to 50 or 40. Check out this video: http://www.engadget.com/2013/10/18/nvidia-g-sync/
It doesn't make a difference if your card is renderring at 60 fps with out a hitch, but you can see the difference when the fps drop to 50 or 40. Check out this video: http://www.engadget.com/2013/10/18/nvidia-g-sync/
the point is that refresh rate is kept equal to gpu frame rate, so there's no need for vsync. this results in less display lag without mid-frame tearing, basically best of both worlds.
I understand that, but if you have vsync there is already no tearing, and with a 144Hz refresh rate your lag is at most 1/144th of a second, or 7ms, and on average 3.5ms - I don't think that's something noticeable.
This would be useful for low-end monitors, but, if I understood correctly, it's not going to work on that kind of monitors anyway.
This would be useful for low-end monitors, but, if I understood correctly, it's not going to work on that kind of monitors anyway.
The problem with vsync is that if your FPS drops below the refresh rate, the next frame is skipped, straight out halving your FPS. So if you have a 60hz display, and you're capable of rendering at 59fps smoothly, you actually end up rendering at 30fps.
Worse, if you're alternating between 55-65fps due to subtle changes in scene complexity, the FPS will flip between 30fps and 60fps erratically, which is absolutely horrific (worse than just sticking at 30fps).
To get smooth vsync 60fps, you probably want to be capable of running at 100fps on average, so there's a margin of safety and your worst case doesn't drop below 60fps.
If this technology can make 55-65fps rendering seem as smooth as a 100fps capable machine with vsync enabled, it just nearly halved the system requirements.
If it sounds like a mundane way to progress, that's because it is. All this is doing is replacing a system of bad design due to legacy reasons. Pushing data when it's done is almost always more sensible than an awkward fixed interval polling loop.
Worse, if you're alternating between 55-65fps due to subtle changes in scene complexity, the FPS will flip between 30fps and 60fps erratically, which is absolutely horrific (worse than just sticking at 30fps).
To get smooth vsync 60fps, you probably want to be capable of running at 100fps on average, so there's a margin of safety and your worst case doesn't drop below 60fps.
If this technology can make 55-65fps rendering seem as smooth as a 100fps capable machine with vsync enabled, it just nearly halved the system requirements.
If it sounds like a mundane way to progress, that's because it is. All this is doing is replacing a system of bad design due to legacy reasons. Pushing data when it's done is almost always more sensible than an awkward fixed interval polling loop.
>I understand that, but if you have vsync there is already no tearing, and with a 144Hz refresh rate your lag is at most 1/144th of a second, or 7ms, and on average 3.5ms - I don't think that's something noticeable.
If the game never drops below 144 then it's not going to be very noticeable, still get extra performance of having vsync off and lower latency though. Even if you could run something that fast, though, it would likely be better overall if you could run it slightly less fast but with higher quality lighting model enabled or whatnot.
And so another way of looking at is: this lets you run at high quality than before, resulting in lower framerates of 30 - 60, because the negative effect of those lower frame rates between 30 and 60 is so minimized.
If the game never drops below 144 then it's not going to be very noticeable, still get extra performance of having vsync off and lower latency though. Even if you could run something that fast, though, it would likely be better overall if you could run it slightly less fast but with higher quality lighting model enabled or whatnot.
And so another way of looking at is: this lets you run at high quality than before, resulting in lower framerates of 30 - 60, because the negative effect of those lower frame rates between 30 and 60 is so minimized.
Here is a great read about V-Sync and how it tries to fix the disparities in refresh rates:
http://hardforum.com/showthread.php?t=928593
Contrast it with what G-Sync is trying to achieve.
http://hardforum.com/showthread.php?t=928593
Contrast it with what G-Sync is trying to achieve.
Reading that, it sounds like triple buffering is already a better solution to this problem. It sounds simpler than requiring a specially embedded DSP from Nvidia in your monitor to talk with your Nvidia card (and only with your Nvidia card).
Triple buffering does not solve the problem. If your monitor is locked at 60Hz and your rendering can't produce more than, say, 55 frames per second then you're going to have to display the same frame for multiple monitor refresh cycles. The monitor will typically show every rendered frame for 2 refresh cycles, with triple buffering you may be able to occasionally show a frame for just one cycle.
With something like G-Sync you can instead show 55 frames per second.
With something like G-Sync you can instead show 55 frames per second.
My understanding from nivla's link is that triple buffering does solve the problem. Rather than experiencing the Frequency/N fps drop when the card can't push out enough frames, triple buffering lets the display's framerate stay at the framerate the card is pushing, with the price being VRAM overhead.
But perhaps I'm misunderstanding the problem statement. The link talks merely about fps drops and how triple buffering can permit the display to be more efficient than Frequency/N fps by essentially pipelining the frames over 3 buffers to keep the monitor's framerate at the graphic card's framerate. However, it accomplishes this by having some frames stay for 2 cycles and others for 1. I don't know how noticeable this is. Perhaps it's a perceptible problem, in which case being able to dynamically manipulate the display's refresh rate is a suitable solution. Granted, at the current price range it sounds like a cheaper solution would be to simply buy a newer model graphics card that can maintain an FPS greater than or equal to your monitor's refresh rate. Though of course I'm sure the goal is to drive the price down and it appears they already have display partners lined up to integrate their embedded DSP.
But perhaps I'm misunderstanding the problem statement. The link talks merely about fps drops and how triple buffering can permit the display to be more efficient than Frequency/N fps by essentially pipelining the frames over 3 buffers to keep the monitor's framerate at the graphic card's framerate. However, it accomplishes this by having some frames stay for 2 cycles and others for 1. I don't know how noticeable this is. Perhaps it's a perceptible problem, in which case being able to dynamically manipulate the display's refresh rate is a suitable solution. Granted, at the current price range it sounds like a cheaper solution would be to simply buy a newer model graphics card that can maintain an FPS greater than or equal to your monitor's refresh rate. Though of course I'm sure the goal is to drive the price down and it appears they already have display partners lined up to integrate their embedded DSP.
Triple-buffering doesn't help that much, and it adds an extra frame of latency, which is really bad.
For anyone excited about this, check out http://www.blurbusters.com/. It's a blog dedicated to everything-refresh-rate.
One thought on this is that this technology, or something related, would be beneficial to Oculus. Carmack already had a comment on it:
G-Sync won't work on any of the display panels Oculus is considering right now, but it will probably be influencing many future panels.
Any benefit if your game is rendering 120fps and your monitor is doing 120hz? Even though they're the same speed they're not necessarily in sync?
Almost no game can render at exactly a target frame rate, and if so it means the game isn't pushing the GPU to the limit. For any graphically awesome game the frame rate will vary and G-Sync should help in that case.
Nah, there's plenty of (mostly a bit older) games that my fairly powerful gaming PC can max out on FPS at very high rates.
Yeah, a computer can easily render frames at the same rate that the monitor accepts them, while being horribly out of phase with the timing that the monitor expects to receive the frames. Imagine you and your friend playing on a swingset, where both of you are swinging at the same speed, but you are always at the top when your friend is at the bottom. That's likely the most common source of tearing: the frame rendered within enough time, but due to poorly configured graphics card drivers, the game process being briefly swapped out for something else at the wrong moment, naive timing in the game's rendering loop, or something else, the framebuffer is swapped out too late anyway.
The traditional way to solve this is with v-sync, which works great on realtime systems like game consoles, but on multitasking operating systems, it's rather difficult to synchronize an application perfectly to the display's refresh rate. As such, every graphics card I've ever used adds a frame or two of buffer when v-sync is enabled, resulting in a buttery-smooth but very noticeably laggy display. It doesn't matter how intensive a scene you are rendering, either: I was playing a Quake source port the other day, a game released over 17 years ago, and even that was unplayable for me with v-sync enabled.
I'm very excited for this technology, because allowing programs to control both the frequency and phase of display updates has the potential to eliminate the vast majority of display artifacts I've experienced.
The traditional way to solve this is with v-sync, which works great on realtime systems like game consoles, but on multitasking operating systems, it's rather difficult to synchronize an application perfectly to the display's refresh rate. As such, every graphics card I've ever used adds a frame or two of buffer when v-sync is enabled, resulting in a buttery-smooth but very noticeably laggy display. It doesn't matter how intensive a scene you are rendering, either: I was playing a Quake source port the other day, a game released over 17 years ago, and even that was unplayable for me with v-sync enabled.
I'm very excited for this technology, because allowing programs to control both the frequency and phase of display updates has the potential to eliminate the vast majority of display artifacts I've experienced.
If they're both able to maintain 120hz/120FPS why can't they just sync up once? Or once every N minutes if they drift apart?
I thought I was pretty clear about that. Synchronization is relatively straightforward in a realtime system, where your program has total control over the timing of execution. On a desktop operating system, however, the graphics card and its drivers can lie, the operating system can lie, the operating system can swap your process out for something else whenever it wants, there is a delay associated with accessing a PC's high accuracy timer (so even the clock lies!), and on top of that, everyone's PC is different. It's still possible to get some rough degree of synchronization going, but it's very difficult and imperfect.
If you still don't think I'm being serious about the timing stuff, consider that some folks still keep around old machines running DOS for real-time communication with microcontrollers.
If you still don't think I'm being serious about the timing stuff, consider that some folks still keep around old machines running DOS for real-time communication with microcontrollers.
Now this is interesting. What can operating systems, drivers, and GPU manufacturers do to restore a DOS like real time sync of frame generation/transfer/display.
[deleted]
Nothing. Otherwise the os would have to stop calling itself preemptive.
Now that every gamer has a multi-core processor, couldn't you allocate n-1 of them to the game with guaranteed non-preemption, and have one core where pre-emption can occur?
it's kind of a cool idea. I wonder if OS's would ever implement something like this. You would probably need to put some iOS like restrictions on it- only one app at a time, it must be running full screen in focus, and cannot be given that kind of priority in the background. Like having a dedicated built in console system.
[deleted]
If the game ever drops below 120 there will be a benefit.
Another way of putting it -- you can crank up graphics settings so that you don't need to stay way above 120 just to maintain a minimum 120, because dipping to 90 for a second no longer produce a huge drop in quality.
Another way of putting it -- you can crank up graphics settings so that you don't need to stay way above 120 just to maintain a minimum 120, because dipping to 90 for a second no longer produce a huge drop in quality.
I have been wondering for a while what reason there is for LCD displays to have refresh rates at all. There is nothing to refresh per se. Shouldn't the display just tell the videocard it can handle frames as long as they're at least some ms apart, then the videocard pushes the frames when they are ready, whenever that may be?
It makes way more sense to me than carrying over the limitations of old (CRT) technology.
It makes way more sense to me than carrying over the limitations of old (CRT) technology.
You are correct. The whole reason things are set up this way is historical, compatibility with CRTs.
With this, ShadowPlay, and Gamestream, I'm very excited for the near future of gaming. NVIDIA is really streamlining the experience for PC gamers providing amenities like TwitchTV streaming via hardware acceleration.
They're going to make hardware manufacturers that strictly focused on streaming capabilities go out of business.
They're going to make hardware manufacturers that strictly focused on streaming capabilities go out of business.
This and Wayland and hopefully I will never ever see a teared or mistimed(stuttering) frame anymore. I can't wait.
Will these work with other GPU's? How hard is it to replicate by the competitors? And will all displays have to support 10 different GPU makers for this feature, if there's no standard set?
I don't think I like the LCD refresh rate to go anywhere below 60hz. 30 seems like really quite bad, duplicated frame or not.
Are my eyes just that bad or is this solving a problem I didn't ever notice exists?
Where is my Optimus driver for Linux?
That said, I read the article and yet remain confused as to where exactly the G-sync module integrates with the monitor. From what I understand, it the G-sync hardware/firmware will run on a packet level, analyzing in realtime the incoming feed of DisplayPort packets and deciding how much of what goes where and when. Very neat.
The most important question, I believe, is what monitors can this be used with? The text makes it clear that users will be able to mod their own ASUS VG248QE monitors to add on the G-sync card, but that's a very, very specific make and model. Is this technology going to require hardware manufacturers to cooperate with nVidia, or will their cooperation simply make things nicer/easier?
Also, some of us have (in business environments) $1k+ S-IPS 30"+ monitors — the quality of these monitors is way above that of consumer models like the VG248QE and others. If there is no way to generically mod monitors without onboard DSPs, I could see that hindering adoption.