Adaptive-Sync for mobile devices

One of the most exciting display features in recent years has been variable refresh. While gaming monitors have long offered higher refresh rates, NVIDIA pioneered variable refresh PC monitors with G-SYNC in 2013. By controlling refresh rates from the display’s end of the display chain, G-SYNC significantly improved perceptual smoothness by removing stutter within a certain frame rate range, eliminating tearing entirely, and reducing input lag. Here is a video of the original G-SYNC demo explaining its benefits.

AMD soon followed with its competing FreeSync, and VESA then added a free implementation called Adaptive-Sync to the DisplayPort 1.2a specification. Originally, Adaptive-Sync was actually introduced in 2009 for displays using internal video signaling, in the eDP specification which mobile devices use for their integrated displays.

Following the rollout of panel self-refresh, I have been eagerly anticipating the adoption of Adaptive-Sync in mobile display stacks. There hasn’t been a real reason to include it, however, because everything in mobile OSes targets 60fps.

The reason for this is simple: going past 60Hz increases display energy consumption. Driving higher frame rates also requires greater compute and thus further increased power draw. Additionally, there is an important technological distinction between IGZO and LTPS displays, the former generally being limited to larger mobile panels, while the latter’s greater efficiency has led it dominate the market for small panels for smartphones and tablets.

I won’t explain display backplanes here, but IGZO transistors do provide certain advantages, such as faster switching speed due to higher subthreshold swing. IGZO displays thus make higher refresh rates more easily achievable than they are for LTPS panels in mobile and laptops. And although a display stack running at higher refresh rates will burn more power, variable refresh can significantly mitigate this increase.

You may have noticed that the iPad Pros and 2016 MacBook Pros are advertised as supporting variable refresh rates. (This will be a feature of the iPhone 8, too.) This is actually a different implementation than that of PCs and desktop monitors. What Apple is instead doing is driving display refresh down to a constant 30fps when the screen is displaying static content. This provides a significant improvement to battery life.

Over the past several years mobile displays have been produced with increasingly higher peak brightness and wider color gamuts. Where do you go from here? Higher refresh rates.

Ordinarily, whether you are targeting 90 or 120Hz refresh, this would require quite a improvement in system-wide performance. Android and iOS both target 60fps, but neither actually manages to run at a consistent 60fps (and with smooth frame-pacing) pretty much ever, excepting small numbers of extremely performant apps.

iOS used to run quite consistently at 60fps across all of its system apps prior to iOS 7, but then pervasive Gaussian blur, increasingly complex UI layouts, and other factors led to a persistent and significant decline in performance. The iPhone 5 running iOS 6 was the last mobile device to truly run at 60fps, in other words.

Thus, without a major focus on improving system performance, don’t hold you breath for iOS to achieve a consistent 60Hz frame delivery anytime soon. That would require a serious OS-wide code review effort, and it’s not a given that Apple cares enough.

Even if that were to happen, some things are basically impossible. Assume that Apple wants to target a 120fps device refresh rate, a nice even multiple of 60 and 24fps. There is absolutely no chance that it can magically get every first and third party iOS app to render within the 8.3ms render window (1/120 seconds).

As you might have guessed, there is another way — Adaptive-Sync.

To support the feature, the display controller (part of the SoC) must support the feature, and the display driver (DDIC) and the panel itself must be capable of higher refresh rates. I believe now is the time for mobile implementations to finally arrive. A device like a large tablet particularly has the energy capacity to spare, so much so that battery cell volume in the iPad Pros was replaced with larger speaker cavities partially to save weight.

Note that using Adaptive-Sync to avoid dropping frames is not as good as hitting a target frame rate in the first place. Rendering smoothly at 55fps is still inferior to hitting a steady 90fps delivery target, for example, because you are simply seeing fewer frames. But you do achieve the same benefits as with PC implementations: stutter is eliminated, and input lag is reduced. (There is already no tearing on mobile.)

For Android, I think Google could even in theory eventually disable triple buffering should Adaptive-Sync ever become ubiquitous in mobile. Only a frame of latency would be gained back, but it would be a clean win. There is no reason for this to ever happen, though.

In conclusion, I hope to soon see tablets and other devices that support Adaptive-Sync. It’s a killer feature that would make a huge difference for both input responsiveness and motion image quality.

 

Update

I somehow missed that Qualcomm has its own variable refresh implementation called Q-Sync for Snapdragon 835's display controller. Qualcomm didn't mention it to me at CES, so I have no idea about the details. I will try to find out more, but it sounds like it may be a proprietary implementation since it seemingly requires Q-Sync-specific support from display driver vendors. I'm a bit skeptical about adoption. Thanks to Naseer for the heads-up!