Of NeRDs and Silverlight

My last post, Instant Netflix on PS3 and TiVo, was about how you can stream movies and TV shows directly from Netflix, via the Internet, to a home TV or computer screen. You don't even need to wait for a DVD in the mail.

Here I'd like to begin going into more depth about how Netflix accomplishes this magic. I'll discuss various topics, plus I'll give links to where you can find out more. Fair warning: I don't guarantee that all of my information is perfectly accurate or complete. I'll try to do the best I can.


You can stream Netflix content onto your computer via a Web browser, or you can stream it to a TV screen through any of several home entertainment devices: certain recent-model TVs that allow Internet access; certain set-top boxes such as TiVo DVRs; certain Blu-ray players that can get online using BD-Live connectivity; and certain game consoles like the Microsoft Xbox 360 and Sony PlayStation 3. These non-computer devices that can stream Netflix are what Netflix calls Netflix-Ready Devices; I call them "NeRDs."

If you're using a computer, the magic formula enlists either Firefox or Safari, web browsers compatible with Silverlight, a plugin from Microsoft that plays video content from Netflix right in a browser window. Let's say you want to watch WALL-E, the 2008 Disney-Pixar hit. You encounter a WALL-E thumbnail at Netflix.com that has a white-on-blue "Play" button below it, or you pull up the dedicated page for the film and see a similar "Play on Computer" button. You click once and watch as a video player appears in your browser window.

But instead of seeing the movie, you get a warning that you need to download and install the software called Silverlight. Your browser offers to do the download, after which you use standard procedures to install it as a browser plugin. It all happens quickly and painlessly. Then you restart your browser and try WALL-E again, and after a few seconds of preliminaries — "Connecting to the Netflix movie server" / "Downloading movie information" / "Determining your video quality" / "Buffering" / "Acquiring content license" — the movie begins, all thanks to Silverlight!

What about those non-computer NeRDs? Home entertainment devices that can stream Netflix content don't use web browsers. They incorporate firmware (aka "system software") that, in turn, incorporates the same functionality as Silverlight. (For all I know, it is Silverlight in a different form ... but it isn't in the form of a browser plugin.)

An exception is the Sony PlayStation game console-cum-Blu-ray player, which won't get firmware that incorporates Silverlight functionality until late 2010. For now, that functionality comes on a Blu-ray disc that you slide into the PS3 each time you want to stream Netflix.


Silverlight (or its NeRD clones) enables Netflix's movie servers to select a bitrate that your network connection to the Netflix server can handle. Each movie is encoded several times, at various bitrates. The higher the bitrate, the better the picture and sound. But if the Internet connection can't keep up with the bitrate that is originally selected, the server won't get "return receipts" (technical term: acknowledgments) back from Silverlight in time. The server can then switch seamlessly to a lower bitrate, and Silverlight is capable of hiding what is going on from the user's eyes and ears.

So, during the "Determining your video quality" preliminary stage, the server cooperates with Silverlight to measure your network speed. Then it selects a bitrate that fits that speed.

Even so, sometimes acknowledgments begin arriving back late, or not at all ... in which case the server assumes some of the information it sends out in the form of message packets is getting lost. It arranges with Silverlight to start using smaller packets. The smaller the packet, the less the information that has to be retransmitted when a packet is lost. Cutting down the packet size saves retransmission overhead and allows the server to avoid dropping down to a lower bitrate. Ideally, playback quality remains the same.

If necessary, though, the server can go so far as to lower the bitrate. If the video is standard definition, 1,500 kbps (the highest SD bitrate that Netflix uses) can drop to 1,000 kbps, 500 kbps, or even as low as 375 kbps. (I'm getting this, by the way, from an article in The Netflix Blog called "Encoding for Streaming.")

When I watch an SD movie using my PlayStation 3, the PS3 lets me display the instantaneous bitrate on the TV screen. I find that the instantaneous bitrate varies widely. Frequently it goes well above 2,000 kbps. When there is little motion in the scene though, the bitrate drops well below 1,000 kbps. In other words, the instantaneous bitrate is variable. This is called "variable bitrate encoding," or VBR. The average bitrate, I assume, is the advertised 1,500 kbps.


Sometimes the speed of the network connection drops enough to cause problems if the original average bitrate isn't likewise dropped down to a lower bitrate.

In the VC-1 Advanced Profile (VC1AP) encoding that Netflix uses, according to the blog post, "each GOP [group-of-pictures; a "picture" is an individual "video frame"] header includes frame size and resolution, which allows [Netflix servers] to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates."

This happens without the user being aware of it, because Silverlight recognizes on-the-fly changes in bitrate by inspecting the parameters in the GOP headers, and it accommodates those changes automatically.

The result can, however, be a drop in perceived video quality. Lower average bitrates can mean more digital compression artifacts, hence messier-looking images. Or there can be fewer video frames per second in the lower-bitrate encode, so motion gets choppy.

However, the swings in instantaneous bitrate due to VBR encoding are a different story. They happen in response to the changing complexity of the moving image, just as they do on a DVD. In order for the viewer to perceive unchanging video quality, more bits are needed to encode highly complex images having a lot of fast motion, while fewer bits are needed for simple, static images.

When Netflix uses a 1,500-kbps encode, it is implicit that the instantaneous bitrate will sometimes be much higher than 1,500 kbps — and so the network connection has to be able to deliver more kilobits per second than the average bitrate implies. In other words, the speed of the connection must be fast enough to allow for the occasionally high overhead of VBR encoding.


What about high-definition encodes? At the time the Netflix blog article was written (Nov. 6, 2008) there were some 400 HD streams available. The article says HD is encoded in VC1AP at 3,800 kbps and again at 2,600 kbps. The video resolution in both cases is 720p, which means each video frame, with its 16:9 widescreen aspect ratio, is made up of 720 rows of 1,280 pixels per row.

This is not "Full HD" 1080p encoding, such as is found on Blu-ray, or even 1080i, each of which would require each frame to contain 1,080 rows x 1,920 pixels.

There are 24 frames of 720p video per second for material transferred from film, mimicking the frame rate of film itself. Shot-to-video HD material is at 30 frames per second (25 fps for video material shot in the British PAL standard). Again, this is not as good as Blu-ray, which uses 60 fps for HD video material shot in the U.S. This type of encoding is referred to as 1080p60. Netflix uses 720p30 (or 720p24 for film-based material).

Netflix says it believes "using 1080p60 would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years."


Another article from the Netflix blog, "Netflix Trying for Consistent Excellence on Streaming," gives more information about what goes on behind the scenes. Netflix streams use servers scattered around the country to avoid congested Internet "backbone" lines. The servers are organized into "content delivery networks" (CDNs). The CDN approach groups servers in regions that serve nearby users. The nodes of the CDNs cooperate with each other to satisfy requests for content by nearby users.

If several users are watching, say, WALL-E at 1,500 kbps, one particular server of one particular CDN is active for all of them. If that server encounters major congestion between itself and any particular user, that one user may develop problems with playback, or lowered video quality. Other WALL-E watchers may see no problems whatever.

In fact, Netflix says, the combination of server and network path may vary depending on what type of device the user is using. "Accordingly," the article says, "[individual] customers may see better performance on [an] Xbox than [on] their PC, or vice-versa."


Netflix's strategy is to keep as many users satisfied as possible, as much of the time as possible, while giving them the best possible video and audio quality. In addition, Netflix wants playback to begin as quickly as possible after the user has initiated it.

This sounds good ... but some users have complained that they typically have slow network connections and never get top-quality results. These users have asked to be able to pre-buffer content using the best-quality encode at the highest available bitrate. They say they don't mind if the start of playback is delayed until a hefty portion of the content — or all of it — has already been buffered.

So far, though, Netflix has not built such a pre-buffering option into Silverlight.