Making high quality video efficient

YouTube works hard to provide the best looking video at the lowest bandwidth. One way we're doing that is by optimizing videos with bandwidth in mind. We recently made videos stream better -- giving you higher-quality video by improving our videos so they are more likely to fit into your available bandwidth.

When you watch a video the YouTube player measures the bandwidth on the client and adaptively chooses chunks of video that can be downloaded fast enough, up to the limits of the device’s viewport, decoding, and processing capability. YouTube makes multiple versions of each video at different resolutions, with bigger resolutions having higher encoding bitrates.

Figure 1: HTTP-based Adaptive Video Streaming.

YouTube chooses how many bits are used to encode a particular resolution (within the limits that the codecs provide). A higher bitrate generally leads to better video quality for a given resolution but only up to a point. After that, a higher bitrate just makes the chunk bigger even though it doesn’t look better. When we choose the encoding bitrate for a resolution, we select the sweet spot on the corresponding bitrate-quality curve (see Figure 2) at the point where adding more data rate stops making the picture look meaningfully better.

Figure 2: Rate-quality curves of a video chunk for a given video codec at different encoding resolutions.

We found these sweet spots, but observing how people watch videos made us realize we could deliver great looking video even more efficiently.

These sweet spots assume that viewers are not bandwidth limited but if we set our encoding bitrates based only on those sweet spots for best looking video, we see that in practice video quality is often constrained by viewers’ bandwidth limitations. However, if we consider an operating point (other than the sweet spot) given our users’ bandwidth distribution (what we call streaming bandwidth), we end up providing better looking video (what we call delivered video quality).

A way to think about this is to imagine the bandwidth available to a user, as a pipe shown in Figure 3. Given the pipe’s capacity fits a 360p chunk but not a 480p chunk, we could tweak the 480p chunk size to be more likely to fit within that pipe by estimating the streaming bandwidth, thereby increasing the resolution users see. We solved the resulting constrained optimization problem to make sure there was no perceivable impact to video quality. In short, by analyzing aggregated playback statistics, and correspondingly altering the bitrates for various resolutions, we worked out how to stream higher quality video to more users.¹

Figure 3: Efficient streaming scenario before and after our proposal

To understand how streaming bandwidth is different from an individual viewer’s bandwidth, consider the example in Figure 4 below. Given the measured distribution of viewers’ available bandwidth, the playback distribution can be estimated using the areas between the encoding bitrates of neighboring resolutions.

Using playback statistics, we are able to model the behavior of the player as it switches between resolutions. This allows us in effect to predict when an increased bitrate would be more likely to cause a player to switch to a lower resolution and thereby cancel the effect of bitrate increase in any one resolution. With this model, we are able to find better operating points for each video in the real world.¹

Figure 4: For a given resolution 720p for example, the playback distribution across resolutions can be estimated from the probability density function of bandwidth. Partitioning the bandwidth using encoding bitrates of the different representations, the probability of watching a representation can then be estimated with the corresponding area under the bandwidth curve.

Another complication here is that the operating points provide an estimate of delivered quality, which is different from encoded quality. If the available bandwidth of a viewer decreases, then the viewer is more likely to switch down to a lower resolution, and therefore land on a different operating point. This doesn’t influence the encoded quality per resolution, but changes the delivered quality.

Fig.5 Our system for encoder optimization

In Figure 5, the Rate-quality analyzer takes the video to be encoded and generates rate-quality curves for each resolution. The Performance Estimator takes these curves and the distributions of viewer resolutions and streaming bandwidth to estimate possible operation points, so the Non-linear optimizer can choose the best possible set.

The output is a set of optimized operation points, one for each resolution. The optimization algorithm can be configured to minimize average streaming bandwidth subject to a constraint of delivered video quality or to maximize delivered video quality subject to a streaming bandwidth budget.

When we used this system to process HD videos, we delivered a reduction of 14 percent in the streaming bandwidth in YouTube playbacks. This reduction in bandwidth is expected to help the viewers to lower their data consumption when watching YouTube videos, which is especially helpful for those on limited data plans. We also saw watch time for the HD resolution increase by more than 6 percent as more people were able to stream higher-resolution videos on both fixed and mobile networks.

Another big benefit of this method is improved viewer experience. In addition to very low impact on delivered quality, these videos loaded up to 5 percent faster with 12 percent fewer rebuffering events.

We have made progress towards better video streaming efficiency. But we still want to do more.

Our optimization approach is currently based on global distribution of viewers’ bandwidth and player resolutions. But videos sometimes are viewed regionally. For example, a popular Indian music video may be less likely to be as popular in Brazil or a Spanish sporting event may not be played many times in Vietnam. Bandwidth and player resolution distributions vary from country to country. If we can accurately predict the geographic regions in which a video will become popular, then we could integrate the local bandwidth statistics to do a better job with those videos. We're looking into this now to try to make your video playback experience even better!

-- Balu Adsumilli, Steve Benting, Chao Chen, Anil Kokaram, and Yao-Chung Lin

¹Chao Chen, Yao-Chung Lin, Anil Kokaram and Steve Benting, "Encoding Bitrate Optimization Using Playback Statistics for HTTP-based Adaptive Video Streaming," Arxiv, 2017