Buyers' Guide to DRM 2017

How it works, the leading technologies, licensing options, business models, and pricing: This guide includes everything content owners need to know to secure their valuable assets.
By Jan Ozer

What Is DRM?

The move away from plugins like Flash and Silverlight has made video delivery easier, but it's also made DRM more complicated. This article describes what DRM looks like today, along with a discussion of the leading DRM technologies and DRM service providers.

By Jan Ozer, StreamingMedia.com

Buyer's Guide to DRM 2016

A simple guide through the complex landscape of multiple DRM technologies. Learn what DRM is, and how to choose and deploy the best solution for each platform.

Reconciling Mozilla’s Mission and W3C EME

With most competing browsers and the content industry embracing the W3C EME specification, Mozilla has little choice but to implement EME as well so our users can continue to access all content they want to enjoy. Read on for some background on how we got here, and details of our implementation.

Digital Rights Management (DRM) is a tricky issue. On the one hand content owners argue that they should have the technical ability to control how users share content in order to enforce copyright restrictions. On the other hand, the current generation of DRM is often overly burdensome for users and restricts users from lawful and reasonable use cases such as buying content on one device and trying to consume it on another.

DRM and the Web are no strangers. Most desktop users have plugins such as Adobe Flash and Microsoft Silverlight installed. Both have contained DRM for many years, and websites traditionally use plugins to play restricted content.

In 2013 Google and Microsoft partnered with a number of content providers including Netflix to propose a “built-in” DRM extension for the Web: the W3C Encrypted Media Extensions (EME).

The W3C EME specification defines how to play back such content using the HTML5 video element, utilizing a Content Decryption Module (CDM) that implements DRM functionality directly in the Web stack. The W3C EME specification only describes the JavaScript APIs to access the CDM. The CDM itself is proprietary and is not specified in detail in the EME specification, which has been widely criticized by many, including Mozilla.

Mozilla believes in an open Web that centers around the user and puts them in control of their online experience. Many traditional DRM schemes are challenging because they go against this principle and remove control from the user and yield it to the content industry.

Instead of DRM schemes that limit how users can access content they purchased across devices we have long advocated for more modern approaches to managing content distribution such as watermarking. Watermarking works by tagging the media stream with the user’s identity. This discourages copyright infringement without interfering with lawful sharing of content, for example between different devices of the same user.

Mozilla would have preferred to see the content industry move away from locking content to a specific device (so called node-locking), and worked to provide alternatives.

Instead, this approach has now been enshrined in the W3C EME specification. With Google and Microsoft shipping W3C EME and content providers moving over their content from plugins to W3C EME Firefox users are at risk of not being able to access DRM restricted content (e.g. Netflix, Amazon Video, Hulu), which can make up more than 30% of the downstream traffic in North America.

We have come to the point where Mozilla not implementing the W3C EME specification means that Firefox users have to switch to other browsers to watch content restricted by DRM.

This makes it difficult for Mozilla to ignore the ongoing changes in the DRM landscape. Firefox should help users get access to the content they want to enjoy, even if Mozilla philosophically opposes the restrictions certain content owners attach to their content.

As a result we have decided to implement the W3C EME specification in our products, starting with Firefox for Desktop. This is a difficult and uncomfortable step for us given our vision of a completely open Web, but it also gives us the opportunity to actually shape the DRM space and be an advocate for our users and their rights in this debate. The existing W3C EME systems Google and Microsoft are shipping are not open source and lack transparency for the user, two traits which we believe are essential to creating a trustworthy Web.

The W3C EME specification uses a Content Decryption Module (CDM) to facilitate the playback of restricted content. Since the purpose of the CDM is to defy scrutiny and modification by the user, the CDM cannot be open source by design in the EME architecture. For security, privacy and transparency reasons this is deeply concerning.

From the security perspective, for Mozilla it is essential that all code in the browser is open so that users and security researchers can see and audit the code. DRM systems explicitly rely on the source code not being available. In addition, DRM systems also often have unfavorable privacy properties. To lock content to the device DRM systems commonly use “fingerprinting” (collecting identifiable information about the user’s device) and with the poor transparency of proprietary native code it’s often hard to tell how much of this fingerprinting information is leaked to the server.

We have designed an implementation of the W3C EME specification that satisfies the requirements of the content industry while attempting to give users as much control and transparency as possible. Due to the architecture of the W3C EME specification we are forced to utilize a proprietary closed-source CDM as well. Mozilla selected Adobe to supply this CDM for Firefox because Adobe has contracts with major content providers that will allow Firefox to play restricted content via the Adobe CDM.

Firefox does not load this module directly. Instead, we wrap it into an open-source sandbox. In our implementation, the CDM will have no access to the user’s hard drive or the network. Instead, the sandbox will provide the CDM only with communication mechanism with Firefox for receiving encrypted data and for displaying the results.

Traditionally, to implement node-locking DRM systems collect identifiable information about the user’s device and will refuse to play back the content if the content or the CDM are moved to a different device.

By contrast, in Firefox the sandbox prohibits the CDM from fingerprinting the user’s device. Instead, the CDM asks the sandbox to supply a per-device unique identifier. This sandbox-generated unique identifier allows the CDM to bind content to a single device as the content industry insists on, but it does so without revealing additional information about the user or the user’s device. In addition, we vary this unique identifier per site (each site is presented a different device identifier) to make it more difficult to track users across sites with this identifier.

Adobe and the content industry can audit our sandbox (as it is open source) to assure themselves that we respect the restrictions they are imposing on us and users, which includes the handling of unique identifiers, limiting the output to streaming and preventing users from saving the content. Mozilla will distribute the sandbox alongside Firefox, and we are working on deterministic builds that will allow developers to use a sandbox compiled on their own machine with the CDM as an alternative. As plugins today, the CDM itself will be distributed by Adobe and will not be included in Firefox. The browser will download the CDM from Adobe and activate it based on user consent.

While we would much prefer a world and a Web without DRM, our users need it to access the content they want. Our integration with the Adobe CDM will let Firefox users access this content while trying to maximize transparency and user control within the limits of the restrictions imposed by the content industry.

There is also a silver lining to the W3C EME specification becoming ubiquitous. With direct support for DRM we are eliminating a major use case of plugins on the Web, and in the near future this should allow us to retire plugins altogether. The Web has evolved to a comprehensive and performant technology platform and no longer depends on native code extensions through plugins.

While the W3C EME-based DRM world is likely to stay with us for a while, we believe that eventually better systems such as watermarking will prevail, because they offer more convenience for the user, which is good for the user, but in the end also good for business. Mozilla will continue to advance technology and standards to help bring about this change.

By Andreas Gal, Mozilla

Interoperability, Digital Rights Management and the Web

We are on the verge of an important inflection point for the Web. In the next few years commercial web video delivery utilizing new, international standards (DASH Media Ecosystem) will become commonplace. These standards will enable cross-platform, interoperable media applications and will transform the media entertainment industry, delight consumers and expand the nature of the Web.

Although all of the standards outlined below are necessary, the most significant change was the introduction of interoperable digital rights management technologies which enable the distribution of digital media on the open web while respecting the rights of content producers.

Download the white paper

Via Video Breakthroughs

The Current State of Android and Video

In the OS landscape, Android is by far the most widely used for tablets and mobile devices. With over 900 million current activated users and approximately 70% market share in the space, the platform is not only the most popular, but it’s also the most fragmented in terms of OEM’s and OS versions out there.

Android History and Origins
Android is a Java-Based operating system used for mobile phones and tablet computers. It was introduced and developed initially by Android, with financial backing from Google. Google acquired Android in 2005. Google announced their mobile plans for Android in 2007 and the first iteration of Android hit the shelves in 2008. Android is the world’s most widely distributed and installed mobile OS. However, in terms of application usage and video consumption, iOS devices lead the way. A consistent user experience and standardized video playback are two of the reasons for this.

Performance of Video on Android
When running video on Android devices, the experience varies from OEM to OS version to media source. Because of this lack of standardization with video, we wanted to give an overall look at the top mobile devices running Android to determine how video is delivered and how it performs across a few key sites and platforms.

We tested the following top devices running the most used versions of Android (2.3, 4.0, 4.1 or 4.2):

• Google Nexus 7
• Google Nexus 4
• Samsung Galaxy 4
• HTC One
• Samsung Galaxy II
• HTC EVO 4G

In summary, the newer versions had overall better video capabilities and quality. On devices running 4.1 or higher, the video players were generally built in, and most were shown in one-third of the screen and ran with little interruption.

On the devices running older versions of Android, the experience was inconsistent across sites and the video performance wasn’t as strong. Some of the top video sites showed the option for either video display on a player or in the web browser, and some had very poor viewing capabilities for video.

A sample look at the different variations of video transfer and display on the Android devices is below:



Click to enlarge

Open Source on Android
Because Android is open source, Google releases the code under the Apache License. For this reason, every OEM modifies the open source code for their devices.

OEM’s create their own coding and specifications for Android for each device. This makes any standardization very difficult. When testing different versions of Android on different target devices, there are a lot of inconsistencies.

Google regularly releases updates for Android which further confuses things. End users often do not upgrade, either because they don’t know how, or because their device does not support the new release. The scattered consistency of updates further confuses any efforts at standardization.

Two of the largest and most widely used Android OEM’s both released their latest open source codes earlier this year. For the Samsung Galaxy codes please click here. For the HTC One click here.


Versions of Android
In 2008 Android v1.0 was released to consumers. Starting in 2009, Android started using dessert and confection code names which were released in alphabetical order: Cupcake, Donut, Eclair, Froyo, Gingerbread, Honeycomb, Ice Cream Sandwich, and the latest, Jelly Bean.

A historical look at the Android devices is below:




In terms of market share, the Gingerbread remains the most popular.





Top Android Devices per OS version
The top devices running Android in terms of both sales and popularity come from various OEM’s, with the majority from Samsung, HTC, LG and Asus. A few of the top devices from the most widely used Android OS versions are as follows:



Click to enlarge

DRM Content on Android
Android offers a DRM framework for all devices running their 3.0 OS and higher. Along with their DRM framework, they offer consistent DRM for all devices using Google’s Widevine DRM (free on all compatible Android devices) which is built on top of their framework. On all devices running 3.0 and higher, the Widevine plugin is integrated with the Android DRM framework to protect content and credentials. However, the content protection depends on the OEM device capabilities. The plug in provides licensing, safe distribution and protected playback of media content.

The image below shows how the framework and Widevine work together.



Click to enlarge

Closed Captions on Android
As developers know, closed captioning is not a simple “feature” of video that can be simply activated. There are a number of formats, standards, and approaches and it’s especially challenging for multiscreen publishers. On Android devices, closed captioning varies from app to app. However, any device using Jelly Bean 4.1 or higher can use their media player which supports internal and external subtitles. Click here for more information.

For any device using the Gingerbread version or lower which do not have any support for rendering subtitle, you can either add subtitle support yourself or integrate a third party solution.

Most larger broadcasters pushing content to OTT devices now serve closed captioning on Android (Hulu Plus, HBO GO, and Max Go to name a few).


Does Android support HLS?
Android has limited support for HLS (Apple’s HTTP Live streaming protocol), and device support is not the same from one version or one device to the next. Android devices before 4.x (Gingerbread or Honeycomb), do not support HLS. Android tried to support HLS with Android 3.0, but excessive buffering often caused streams to crash. Devices running Android 4.x and above will support HLS, but there are still inconsistencies and problems.





Best Practices for Video on Android
For deploying video on Android, there are several suggested specifications to follow. Below is a list of files supported by Android devices. Developers can also use media codecs either provided by any Android-powered device, or additional media codecs developed by third-party companies. If you want to play videos on Android, find a multi-format video player or convert videos to Android compatible formats using an encoding company.





Video Specifications for Android
Below are the recommended encoding parameters for Android video from the Android developer homepage. Any video with these parameters are playable on Android phones.





Video Encoding Recommendations
This table below lists examples of video encoding profiles and parameters that the Android media framework supports for playback. In addition to these encoding parameter recommendations, a device’s available video recording profiles can be used as a proxy for media playback capabilities. These profiles can be inspected using the CamcorderProfile class, which is available since API level 8.




For video content that is streamed over HTTP or RTSP, there are additional requirements:

• For 3GPP and MPEG-4 containers, the moov atom must precede any mdat atoms, but must succeed the ftypatom.
• For 3GPP, MPEG-4, and WebM containers, audio and video samples corresponding to the same time offset may be no more than 500 KB apart. To minimize this audio/video drift, consider interleaving audio and video in smaller chunk sizes.

For information about how to target your application to devices based on platform version, read Supporting Different Platform Versions.

Source: Encoding.com

HTML5 Video in IE 11 on Windows 8.1

We've previously discussed our plans to use HTML5 video with the proposed "Premium Video Extensions" in any browser which implements them. These extensions are the future of premium video on the web, since they allow playback of premium video directly in the browser without the need to install plugins.

Today, we're excited to announce that we've been working closely with Microsoft to implement these extensions in Internet Explorer 11 on Windows 8.1. If you install the Windows 8.1 Preview from Microsoft, you can visit Netflix.com today in Internet Explorer 11 and watch your favorite movies and TV shows using HTML5!

Microsoft implemented the Media Source Extensions (MSE) using the Media Foundation APIs within Windows. Since Media Foundation supports hardware acceleration using the GPU, this means that we can achieve high quality 1080p video playback with minimal CPU and battery utilization. Now a single charge gets you more of your favorite movies and TV shows!

Microsoft also has an implementation of the Encrypted Media Extensions (EME) using Microsoft PlayReady DRM. This provides the content protection needed for premium video services like Netflix.

Finally, Microsoft implemented the Web Cryptography API (WebCrypto) in Internet Explorer, which allows us to encrypt and decrypt communication between our JavaScript application and the Netflix servers.

We expect premium video on the web to continue to shift away from using proprietary plugin technologies to using these new Premium Video Extensions. We are thrilled to work so closely with the Microsoft team on advancing the HTML5 platform, which gets a big boost today with Internet Explorer’s cutting edge support for premium video. We look forward to these APIs being available on all browsers.

By Anthony Park and Mark Watson, The Netflix Tech Blog

HTML5 Video at Netflix

Today, we’re excited to talk about proposed extensions to HTML5 video that enable playback of premium video content on the web. We currently use Microsoft Silverlight to deliver streaming video to web browsers on the PC and Mac. It provides a high-quality streaming experience and lets us easily experiment with improvements to our adaptive streaming algorithms.

But since Microsoft announced the end of life of Silverlight 5 in 2021, we need to find a replacement some time within the next 8 years. We'd like to share some progress we've made towards our goal of moving to HTML5 video.

Silverlight and Browser Plugins
Silverlight is a browser plugin which allows our customers to simply click "Play" on the Netflix website and watch their favorite movies or TV shows, but browser plugins have a few disadvantages. First, customers need to install the browser plugin on their computer prior to streaming video. For some customers, Netflix might be the only service they use which requires the Silverlight browser plugin. Second, some view browser plugins as a security and privacy risk and choose not to install them or use tools to disable them. Third, not all browsers support plugins (eg: Safari on iOS, Internet Explorer in Metro mode on Windows 8), so the ability to use them across a wide range of devices and browsers is becoming increasingly limited. We're interested to solve these problems as we move to our next generation of video playback on the web.

HTML5 Premium Video Extensions
Over the last year, we've been collaborating with other industry leaders on three W3C initiatives which are positioned to solve this problem of playing premium video content directly in the browser without the need for browser plugins such as Silverlight. We call these, collectively, the "HTML5 Premium Video Extensions":

Media Source Extensions (MSE)
The W3C Media Source Extensions specification "extends HTMLMediaElement to allow JavaScript to generate media streams for playback." This makes it possible for Netflix to download audio and video content from our content delivery networks and feed it into the video tag for playback. Since we can control how to download the audio/video content in our JavaScript code, we can choose the best HTTP server to use for content delivery based on real-time information, and we can implement critical behavior like failing over to alternate servers in the event of an interruption in content delivery. In addition, this allows us to implement our industry-leading adaptive streaming algorithms (real-time selection of audio/video bitrates based on available bandwidth and other factors) in our JavaScript code. Perhaps best of all, we can iterate on and improve our content delivery and adaptive streaming algorithms in JavaScript as our business needs change and as we continue to experiment.

Encrypted Media Extensions (EME)
The W3C Encrypted Media Extensions specification "extends HTMLMediaElement providing APIs to control playback of protected content." The video content we stream to customers is protected with Digital Rights Management (DRM). This is a requirement for any premium subscription video service. The Encrypted Media Extensions allow us to play protected video content in the browser by providing a standardized way for DRM systems to be used with the media element. For example, the specification identifies an encrypted stream format (Common Encryption for the ISO file format, using AES-128 counter mode) and defines how the DRM license challenge/response is handled, both in ways that are independent of any particular DRM. We need to continue to use DRM whether we use a browser plugin or the HTML5 media element, and these extensions make it possible for us to integrate with a variety of DRM systems that may be used by the browser.

Web Cryptography API (WebCrypto)
The W3C Web Cryptography API specification defines an API for "basic cryptographic operations in web applications, such as hashing, signature generation and verification, and encryption and decryption." This API allows Netflix to encrypt and decrypt communication between our JavaScript and the Netflix servers. This is required to protect user data from inspection and tampering, and allows us to provide our subscription video service on the web.

First Implementation in Chrome OS
We've been working with Google to implement support for the HTML5 Premium Video Extensions in the Chrome browser, and we've just started using this technology on the Samsung ARM-Based Chromebook. Our player on this Chromebook device uses the Media Source Extensions and Encrypted Media Extensions to adaptively stream protected content. WebCrypto hasn't been implemented in Chrome yet, so we're using a Netflix-developed PPAPI (Pepper Plugin API) plugin which provides these cryptographic operations for now. We will remove this last remaining browser plugin as soon as WebCrypto is available directly in the Chrome browser. At that point, we can begin testing our new HTML5 video player on Windows and OS X.

We're excited about the future of premium video playback on the web, and we look forward to the day that these Premium Video Extensions are implemented in all browsers!

By Anthony Park and Mark Watson, The Netflix Tech Blog

Google, Microsoft, Netflix Want Encrypted HTML5 Media Spec

Google, Microsoft, and Netflix have put forward a proposal that could add a level of copy protection to HTML5 audio and video. Encrypted Media Extensions would let apps on the web and elsewhere use keys to control who has access to a given media stream. It would allow any format that would work in HTML5 as long as the format itself can support some kind of key or bit.

If implemented, it could eliminate objections raised by Adobe as well as content providers regarding HTML5's limits. Paid movie services and others that want a secure stream have either limited HTML5 to native mobile apps, where users can't easily rip the feed, or avoided it altogether.

Netflix would have the option of dropping Microsoft's Silverlight plugin when on the web, while Google could skip Flash for its YouTube-based movie service in Android Market. Microsoft has been deprecating Silverlight and could use HTML5 even on the Xbox 360.

Concerns exist both on technical and philosophical levels. It may not necessarily provide the promised experience in at least the initial spec and could require revision. Despite promising to keep digital rights management out of HTML5, the trio have also raised alerts from Mozilla and others concerned it would break standards and force a less than ideal concern that might hurt some HTML5 supporters.

The World Wide Web Consortium, which has to consider the proposal, hasn't said when it might decide. As an unofficial draft, it could spend some time being completed before the question of approval and possible implementation comes up.

Source: Electronista

The Case for a Common DRM Framework

It is possible to overstate the complexity of multi-screen video, but the absolute number and types of display devices are indeed increasing, which means that efforts to promote standards and greater simplicity address a live concern. A current initiative, playing out within the MPEG-DASH Industry Forum, among other places, to enable Digital Rights Management (DRM) interoperability is a case in point.

First, however, a few words about complexity. As underscored in several recent stories, Apple’s iPad continues to dominate the second-screen. Pay TV operators can meet much of the multi-screen demand simply by delivering to that one device, which stands atop a sort of multi-screen pecking order.

“In order to deploy Over-The-Top (OTT) services quickly, operators have strategically prioritized the CE devices that they wish to support, based on consumer popularity and market adoption,” writes Steve Tranter, VP at NDS (now part of Cisco) in a paper delivered during a TV Everywhere session at the SCTE Cable-Tec Expo in Orlando. “The most popular being the Apple iPad, followed by PCs, Android tablets and smart phones, game consoles and connected TVs.”

But even delivering to a single device would not eliminate complications. “Not only do you have iOS and Android fragmentation,” said Albert Lai, Innovation Architect for Media and Entertainment at Brightcove, during another session in Orlando, “but within iOS, there is fragmentation within the devices, and within Android, much more.” Thus the countervailing efforts to promote common platforms: HTML5 in the case of Brightcove; and a common DRM framework, in the case of NDS (Cisco) and others.

Building his case for a common downloadable DRM framework that is independent of but compatible with CE devices of all shapes and sizes, Tranter names three standards that could play a foundational role, namely:

• Simulcrypt — the long-standing DVB protocol published by ETSI used to enable multiple key management systems;
• MPEG-DASH — Dynamic Adaptive Streaming over HTTP (DASH), which became an ISO standard in late 2011;
• UltraViolet — an authentication and cloud-based rights system deployed over the past few years by a consortium of studios, manufacturers and service providers.

Enhancements to Simulcrypt that Tranter believes would advance this cause include multiple encoder algorithms; in-band delivery that would move beyond proprietary manifest mechanisms; forensic watermark and key fingerprinting insertion; periodic key rotation on linear channels; and separate encryption keys for different bitrates. As for MPEG-DASH, he notes that while it provides a standard way for DRM systems to exchange encryption keys, it does not define how a system acquires decryption keys or distributes licenses. Finally, while Tranter leans toward local rather than centralized storage of licenses, which UltraViolet promotes, he sees value in the consortium’s common license format.

The end goal is giving Pay TV operators more control. “This is the key thing: You’re not relying on Apple or other devices to upgrade their security systems,” Tranter said during the session. “You can drive your own service portfolio.”

Reached for comment, Robin Wilson, VP Business Development at Nagra, seconded this effort to build bridges within the DRM ecosystem. “There is work well underway in the MPEG-DASH Industry Forum to come up with something analogous to Simulcrypt,” he said. The idea, proposed in one instance by Nagra and being discussed in one of the Forum’s sub-groups, is to refresh the aging standard with updates and extensions to enable that the lowest level of key that works on any given stream is shared between DRM systems.

Wilson said that UltraViolet is not trying to invent anything, but is looking at MPEG-DASH as the underlying standard, which could bring with it this ongoing work in interoperability. “But all is not totally rosy (with MPEG-DASH), because it may become too challenging to make all DRM or all file formats work together,” Wilson said.

Given enduring differences in implementation, it is likely that two branches—or what Wilson called two “half” or “partner” standards—will emerge from the overarching MPEG-DASH project. In terms of DRM, he said that even though the schemes falling within MPEG-DASH use AES encryption and 128-bit keys, they use different AES modes, which would make it hard to share keys and re-purpose streams.

“But at least the licensing and key servers will work together,” Wilson said.

So where does the 800-pound gorilla stand on these efforts? “Apple is helping around the sidelines on MPEG-DASH Industry Forum; they’re not driving it,” Wilson said. “They have their own vertically integrated ecosystem, so there’s no strong desire for them to go outside for a different DRM.”

“On the other hand, they are trying to help make HLS (HTTP Live Streaming) compatible with MPEG-DASH.” he said. “There is quite a lot of commonality between MPEG-DASH and how HLS works.”

By Jonathan Tombes, Videonet

HD Forum Releases TNT 2.0 Specifications and Conformance Tests

HD Forum is aimed at helping and promoting the development of innovative technologies for the French Digital TV environment. Among various topics, HD Forum has worked during the last 2 years on the TNT2.0 project.

The goal was to coordinate the launch of HbbTV interactive services in France. This effort is done in partnership with the HbbTV consortium and led to an upgrade version of HbbTV standard, version 1.5 and a first recommendation for DRM's.

In this respect:

• The HD Forum board is making official the TNT 2.0 specifications to be applied on the French market;


• All together, manufacturers, broadcasters and distributors representatives committed around the technical choices made so far, expecting that products and services will ramp up from Q4’2012 to April 2013 and beyond. Of course, products and service launch plans details remain the respective choices of broadcasters, manufacturers and distributors;


• In order to support this goal, the “TNT 2 0 - Terminal Specification - V1.1 Final” including last improvement has been made available on the HD FORUM web site for a wider distribution to all interested parties, including non HD FORUM members;


• The HD FORUM is also pleased to announce the full release of the TNT 2.0 test suite, developed in cooperation with companies Farncombe and Digital TV labs in the UK. This test suite has already been licensed to multiple manufacturers, and the HD FORUM restates the requirement for consumer electronic manufacturers to self-validate compliance of their products, by passing the TNT 2.0 test suite. The test suite is licensed under a fair and non-discriminatory terms and conditions approved by the HD FORUM;


• In parallel, work continues on the interoperability of services;


• In due course, HD FORUM will contribute to prepare the launch of products and services complying with the TNT 2.0 specifications.

Source: HD FORUM

Online Video Services and DRM Technology

Major contemporary trends in online video services:

• Multiscreen: anytime access from any device.
  
  
• A variety of business models: many options to access video content (online viewing, pay-per-view, download, in-stream ads).
  
  
• Technological complexity – a variety of video platform modules and features (content load, processing, protection and delivery; video load balancing, video playback, etc.).
  
  
• Providing customers with access to the premium content and the related need in security tools: the major movie studios impose heavy demand on security as they offer very expensive content (multi-million-dollar budgets are common for today’s movies).

A simplified design of a modern video platform for Internet broadcast involves content preparation, protection and distribution to the end user devices. Usually, the source content first enters a transcoding system (ingestion) that generates a set of videos for different end-user devices and bitrates for each device (to enable dynamic bit rate). The prepared content is sent to the operator’s in-house video delivery network or the CDN provider external infrastructure.


A simplified online video platform design

There are several most popular options for delivery of content to the user:

• Downloading a video file to be viewed in a special application (HTTP Download).
  
  
• Live streaming using ad-hoc protocols and playback of the video stream (RTSP, RTMP, MPEG2-TS).
  
  
• Downloading a file in fragments, gluing of the fragments together in the video player and viewing of the video content in the process of download (Adaptive Streaming over HTTP).

If the content needs high-quality protection, prior to transferring it to the delivery network it has to be encrypted with a DRM solution. On the attempt of incoming content playback, the video player can detect the encrypted data and request authentication and a decryption key from the license server. At the level of a video player run on a client device, video analytics is very often implemented. It includes collection of detailed viewing data, monitoring of quality parameters, etc.

Specifics of Content Protection
Almost all DRM solutions are built on a unified architecture and consist of two parts: the server part (business logic) and the client part (player). The server part also consists of two modules: the encoder which prepares (encrypts) the content and the license server which issues content playback licenses to the users (players).

In terms of content delivery networks, there is no difference between the secure and open content. DRM solutions usually use asymmetric encryption algorithms, in this case, the content is encrypted with the master key, but users have to apply their unique session-specific keys.

In the case of LIVE streaming, to enable a more robust content protection, key rotation mechanisms are typically used. A key is changed at a predefined time interval. In case of VOD, different keys are used for different content units and, if the copyright holder has taken care of this, license caching is also performed. That is, the key is stored on the local machine for a certain amount of time or a given number of views, so that the application does not have to contact the license server at each viewing.

All these parameters usually mentioned as a Content Protection Policy which could be different for different major copyright holders and even for different content units.

Major DRM Vendors
There are many Digital Rights Management systems on the market. Each of them has its strengths and weaknesses. The situation is exacerbated by a high degree of segmentation on the end-user device market: Windows (with multiple browsers), MacOS, iOS, Android, TV sets and set-top boxes with their relevant operating systems and technologies.

With such a variety of market players, it is pretty difficult to agree on common standards and technologies. Therefore, multiple groups of technical solutions emerge, deferring creation of the universal technology for a certain while.

The most popular content protection tools used by the online video services are the following:

• Adobe Flash Access. Due to its popularity (most videos on the Web are based on Adobe Flash). This is the perfect solution to protect content in Web browsers, on PCs, and on the Android devices.
  
  
• Microsoft PlayReady is a successor of Windows Media DRM, one of the first content protection solutions. That’s why the Microsoft PlayReady SDK has got installed on most of Connected TV devices (LG, Samsung, etc.) and, of course, on the Windows operating systems (Media Player).
  
  
• Widevine, now part of Google, is a developer of content protection solutions. Its popularity is due to the support of a broad range of consumer devices (TVs, game consoles, iOS devices).
  
  
• Marlin is one of the few DRM solutions supported in Philips TVs, Sony PS3 and PSP devices. It is notable by its reliance on open standards and SaaS model support (paying per transaction for obtaining licenses).

Adobe Protected Streaming deserves special attention. It implements RTMPE (RTMP Encrypted) to enable secure streaming. This is an alternative to DRM. In contrast to DRM, security is implemented at the transport level rather than at the content level. It includes encryption of multimedia content between Flash Media Server and Flash Player.

However this is not the most reliable solution, but still it is approved by most major copyright holders. RTMPE involves additional mechanisms for content protection at the application level, such as: tokenization of links to content (i.e., preventing of direct content access bypassing the video service logic), and user authentication.

Compared to full-scale DRM, this approach combines a several times lower cost and ease of launch (a minimum configuring complexity). In addition, no user actions to install the DRM component are required. However, as new distribution technologies are spreading (such as the Adaptive Streaming ove HTTP), operators use to give up this approach in favor of DRM that is more universal. In addition, there are certain tools capable of cracking RTMPE.

DRM Solution Architecture
Almost any DRM solution consists of three parts:

• The service of content encryption (server) pre-encrypts the content for distribution via the open Web channels in the protected format only. Most of solutions use AES-128. Many vendors have supported HSM modules to optimize encryption and offload the main CPU.
  
  
• Licensing service (server) is a solution making a decision whether to issue a video content decryption key, in accordance with the business logic (content distribution rules). This includes identification of a pending payment or checking of viewer balance sufficiency. In most cases, this is the application server (for instance, it can be Java-based).
  
  
• SDK (i.e., a set of software libraries) to handle protected content in the video player. It integrates the logic of interacting with the licensing service into the video player, and decodes the video before playback. Also, the SDK tools implement additional security features: control of access to the video player memory, protection of analog and digital outputs, and detection of hacker attacks. SDK validates all technical and logical restrictions set on the server for the content and a particular user, such as the ability to record to a disk or mobile device.

So, at the DRM input, the content is received and access restriction business rules are applied to it. The output video can then be played back on a wide range of consumer devices.

As it has already been noted above, DRM content delivery is independent of its protection method and is determined primarily by the business model. Hence, the same protected content can be made either downloadable (e.g., via the P2P networks, to minimize the cost) or presented for online streaming or viewing. However, when selecting a DRM system, you should check which delivery models and access restriction models are enabled by a given solution.


DRM algorithm for streaming video


DRM algorithm for downloadable video

The above figures show that, in terms of DRM there is no difference between streaming and video on demand. As a rule, it all comes down to using of different alternative authentication algorithms (identifying whether the download or just online viewing has been paid).

DRM Vendor Selection
A critical issue in choosing a content protection solution is the range of consumer devices supported. Each online video service has its own priority of consumer platforms. Here we would like to discuss just the very basics of running content security solutions on various platforms:

• Web browsers on Windows / MacOS / Linux
  
• Desktop applications
  
• Web browsers on Android
  
• Android applications
  
• Web browsers on iOS
  
• iOS-applications
  
• Connected TV

The key differences between the various content protection solutions consist in supporting the above technological platforms. For instance, Adobe Flash Access offers user-friendly tools of content protection that do not require installation of additional plugins, as Flash Player is already installed almost anywhere (according to Adobe, on 99% of computers and 80% of mobile devices).

The need to install additional plugins can filter off those potential service users that possess no administrator rights or have a low level of computer literacy. In addition, there is a great community of Flash developers leveraging the Adobe technology. They have multiple years of experience in creating of cross-platform applications with rich interactive interfaces (which is pretty important for online video services).

Other technologies rely on less abundant players and feel an acute deficit of technical experts. Still, it is worth noting that the Widevine based solutions provide efficient content protection for Samsung and LG TV sets and iOS devices. Historically, the solutions based on Windows Media DRM have been supported on an even greater number of Connected TVs. Also, Verimatrix is unequaled among the STB vendors.

Recently, Adobe has shook the industry by announcing discontinuation of Flash plugin in mobile browsers (in fact, this means Android mostly), explaining it by their focus on mobile AIR application development and HTML5. To clarify its stand, just the other day Adobe announced that it is now working on HTML5 DRM.

It looks like we are in for something fascinating to come. The rapid spread of the new HTML5 video format is prevented by absence of uniform standards of video codecs (such as H.264, WebM, Ogg Theora) across a variety of browsers and lack of a single reliable DRM system to meet the copyright holder requirements. Different DRM solutions are used to support various video containers (mp4, wmv, ogg, mkv). Now, most of video on the Web is using mp4, so all the DRM vendors are concerned with supporting mp4. But with HTML5, mp4 is far from ubiquitous support.

In the case of Desktop applications the situation is much simpler: the vendors offer various SDKs to integrate DRM into video applications.

Content protection for iOS devices is a separate big topic. The issue is that Apple went its own way and preferred Flash to HTML5 with H.264 video. For video service developers, only one method of content protection is available: blockwise stream encryption with AES-128. This scheme is supported, for example, in the latest version of Adobe Flash Media Server 4.5.

However, the developers have to implement their own license management mechanisms or use an SDK from Widevine or other vendors. In case of browser playback, license management has to be implemented in JavaScript, which posits a substantial vulnerability. In case of playback from an application, Objective C SDK has to be used to enable a much higher level of protection.

In the world of music, DRM solutions showed no viability, especially after Apple iTunes abandoned FairPlay DRM. In our opinion, this is due to a significantly lower cost of audio content as compared to Hollywood blockbusters. As a result, copyright holders take a substantially harder stand with videos. The well-known western movie studios present a long list of technical requirements to companies seeking to distribute their digital content. Practically all solutions of IPTV operators and the major Runet websites with premium content are the examples of content protection systems.

All DRM solutions are scalable and consume roughly the same resources, as the encryption algorithms used are uniform. When choosing the DRM solutions, we recommend you to make a list of user devices based on their priorities. Supported devices (as well as containers and codecs), price, user experience, business models and technologies for content delivery: all these make up the applicable selection criteria.

Source: DENIVIP Media

The Trials and Tribulations of HTML Video in the Post-Flash Era

Adobe reversed course on its Flash strategy after a recent round of layoffs and restructuring, concluding that HTML5 is the future of rich Internet content on mobile devices. Adobe now says it doesn’t intend to develop new mobile ports of its Flash player browser plugin, though existing implementations will continue to be maintained.

Adobe’s withdrawal from the mobile browser space means that HTML5 is now the path forward for developers who want to reach everyone and deliver an experience that works across all screens. The strengths and limitations of existing standards will now have significant implications for content creators who want to deliver video content on the post-flash Web.

Apple’s decision to block third-party browser plugins like Flash on its iOS devices played a major role in compelling Web developers to build standards-based fallbacks for their existing Flash content. This trend will be strengthened when Microsoft launches Windows 8 with a version of Internet Explorer that doesn’t support plugins in the platform’s new standard Metro environment.

Flash still has a significant presence on the Internet, but it's arguably a legacy technology that will decline in relevance as mobile experiences become increasingly important. The faster pace of development and shorter release cycles in the browser market will allow open standards to mature faster and gain critical mass more quickly than before. In an environment where standards-based technologies are competitive for providing rich experiences, proprietary vendor-specific plugins like Flash will be relegated to playing a niche role.

Our use of the phrase “post-Flash” isn’t intended to mean that Flash is dead or going to die soon. We simply mean that it’s no longer essential to experiencing the full Web. The HTML5 fallback experiences on many Flash-heavy sites still don’t provide feature parity with the Flash versions, but the gap is arguably shrinking—and will continue to shrink even more rapidly in the future.

Strengths and Weaknesses of HTML5 Video
HTML5 has much to offer for video delivery, as the HTML5 video element seamlessly meshes with the rest of the page DOM and is easy to manipulate through JavaScript. This means that HTML5 video offers significantly better native integration with page content than it has ever been possible to achieve with Flash. The open and inclusive nature of the standards process will also make it possible for additional parties to contribute to expanding the feature set.

A single company no longer dictates what can be achieved with video, and your video content is no longer isolated to a rectangle embedded in a page. HTML5 breaks down the barriers between video content and the rest of the Web, opening the door for more innovation in content presentation. Three are some really compelling demonstrations out there that showcase the use of video in conjunction with WebGL and other modern Web standards. For example, the video shader demo from the 3 Dreams of Black interactive film gives you a taste of what’s possible.

Click to watch the video

Of course, transitioning video delivery in the browser from Flash to HTML5 will also pose some major challenges for content creators. The standards aren’t fully mature yet and there are still a number of features that aren’t supported or widely available across browsers.

For an illustration of how deep the problems run, you need only look at Mozilla’s Firefox Live promotional website, which touts the organization’s commitment to the open Web and shows live streaming videos of Red Panda cubs from the Knoxville Zoo. The video is streamed with Flash instead of using standards-based open Web technologies.

In an FAQ attached to the site, Mozilla says that it simply couldn’t find a high-volume live streaming solution based on open codecs and open standards. If Mozilla can’t figure out how to stream its cuddly mascot with open standards, it means there is still work to do.


Flash is required to see the Red Panda cubs on Mozilla's website

Two of the major technical issues faced by HTML5 video adopters are the lack of adequate support for adaptive streaming and the lack of consensus surrounding codecs. There is currently an impasse between backers of the popular H.264 codec and Google’s royalty-free VP8 codec. There’s no question that a royalty-free video format is ideal for the Web, but the matter of whether VP8 is truly unencumbered by patents—and also meets the rest of the industry’s technical requirements—is still in dispute.

There is another major issue that hasn’t been addressed yet by open Web standards that could prove even more challenging: content protection. The vast majority of Flash video content on the Internet doesn’t use any kind of DRM and is trivially easy to download. Flash does, however, provide DRM capabilities and there are major video sites that rely on that technology in order to protect the content they distribute.

Can DRM Be Made to Play Nice with Open Standards?
DRM is almost always bad for regular end users and its desirability is highly debatable, but browser vendors will have to support it in some capacity in order to make HTML5 video a success. Many of the content creators who license video material to companies like Netflix and Hulu contractually stipulate a certain degree of content protection.

Mozilla’s Robert O’Callahan raised the issue of HTML5 video DRM in a recent blog entry shortly after Adobe’s announcement regarding mobile Flash. He expressed some concern that browser vendors will look for a solution that is expedient rather than inclusive, to the detriment of the open Web.

“The problem is that some big content providers insist on onerous DRM that necessarily violates some of our open Web principles (such as Web content being equally usable on any platform, based on royalty-free standards, and those standards being implementable in free software),” O'Callahan wrote. “We will probably get into a situation where Web video distributors will be desperate for an in-browser strong DRM solution ASAP, and most browser vendors (who don’t care all that much about those principles) will step up to give them whatever they want, leaving Mozilla in another difficult position. I wish I could see a reasonable solution, but right now I can’t. It seems even harder than the codec problem.”

O'Callahan also pointed out in his blog entry that the upcoming release of Windows 8, which will not support browser plugins in its Metro environment, means that the lack of DRM support in standards-based Web video is no longer just a theoretical concern. Microsoft may need to furnish a solution soon, or risk frustrating users who want to watch commercial video content on the Web in Windows 8 without installing additional apps or leaving the Metro shell.

Netflix Stands Behind DASH
Flash evangelists may feel that the limitations of HTML5 video and the problems that content creators are sure to face during the transition are a vindication of the proprietary plugin model. But the advantages of a truly open, vendor-neutral, and standards-based video solution that can span every screen really dwarf the challenges. That is why major stakeholders are going to be willing to gather around the table to try find a way to make it work.

Netflix already uses HTML5 to build the user interfaces of some of its embedded applications, including the one on the PS3. The company has soundly praised the strengths of a standards-based Web technology stack and has found that there are many advantages. But the DRM issue and the lack of suitably robust support for adaptive streaming have prevented Netflix from dropping its Silverlight-based player in regular Web browsers.

The company has committed to participating in the effort to make HTML5 a viable choice for all video streaming. Netflix believes that the new Dynamic Adaptive Streaming over HTTP (DASH) standard being devised by the Motion Picture Experts Group (MPEG) will address many of the existing challenges and pave the way for ubiquitous adoption of HTML5 for streaming Internet video.

DASH, which is expected to be ratified as an official standard soon, has critical buy-in from many key industry players besides Netflix, including Microsoft and Apple. An early DASH playback implementations is already available as a plugin for the popular VLC video application.

The DASH standard makes video streaming practical over HTTP and addresses the many technical requirements of high-volume streaming companies like Netflix, but it doesn’t directly address the issue of DRM by itself. DASH can be implemented in a manner that is conducive to supporting DRM, however.

DASH and DRM
Ericsson Research, which is involved in the DASH standardization effort, has done some worthwhile preliminary research to evaluate the viability of DRM on DASH. Ericsson produced a proof-of-concept implementation that uses DRM based on the Marlin rights management framework.

Marlin, which was originally created by a coalition of consumer electronics vendors, is relatively open compared to alternate DRM technologies and makes use of many existing open standards. But Marlin is still fundamentally DRM and suffers from many of the same drawbacks, and adopters have to obtain a license from the Marlin Trust Management Organization, which holds the keys.


The architecture of the Marlin rights management framework

Ericsson explains in its research that it chose to experiment with Marlin for their proof-of-concept implementation because it’s available and mature—other similar DRM schemes could also easily be adopted. Existing mainstream DRM schemes would all likely pose the same challenges, however, and it’s unlikely that such solutions will be viewed as acceptable by Mozilla. More significantly, an implementation of HTML5 video that relies on this kind of DRM would undermine some of the key values and advantages of openness that are intrinsic to the open Web.

The ease with which solutions like Marlin can be implemented on top of HTML5 will create pressure for mainstream browser vendors to adopt them quickly. This could result in the same kind of fragmentation that exists today surrounding codecs. As O’Callahan said, it’s easy to see this issue becoming far more contentious and challenging to overcome than the codec issue.

What Next?
The transition to HTML5 and standards-based technology for video delivery will bring many advantages to the Web. There are some great examples that show what can be achieved when developers really capitalize on the strengths of the whole open Web stack. The inclusiveness of the standards process will also give a voice to additional contributors who want to expand the scope of what can be achieved with video on the Web.

There are still some major obstacles that must be overcome in order for the profound potential of standards-based Web video to be fully realized in the post-Flash era. Open standards still don’t deliver all of the functionality that content creators and distributors will require in order to drop their existing dependence on proprietary plugins. Supplying acceptable content protection mechanisms will prove to be a particularly bitter challenge.

Despite the barriers ahead, major video companies like Netflix recognize the significant advantages of HTML5 and are willing to collaborate with other stakeholders to make HTML5 video a success. The big question that remains unanswered is whether that goal can be achieved without compromising the critically important values of the open Web.

By Ryan Paul, Ars Technica

Multi-Screen IP Video Delivery

Online and mobile viewing of widely-available, high-quality video content including TV programming, movies, sports events, and news is now poised to go mainstream. Driven by the recent availability of low-cost, high-resolution desktop/laptop/tablet PCs, smart phones, set-top boxes and now Ethernet-enabled TV sets, consumers have rapidly moved through the ‘novelty’ phase of acceptance into expectation that any media should be available essentially on any device over any network connection. Whether regarded as a disruption for cable TV, telco or satellite TV providers, or an opportunity for service providers to extend TV services onto the web for on-demand, time-shifted and place-shifted programming environments – often referred to as ‘three screen delivery’ or ‘TV Anywhere’ – this new video delivery model is here to stay.

While tremendous advancements in core and last mile bandwidth have been achieved in the last decade around the world – primarily driven by web-based data consumption – video traffic represents a quantum leap in bandwidth requirements. Coupled with the fact that the Internet at large is not a managed quality-of-service environment, requires that new methods of video transport be considered to provide the quality of video experience across any device and network that we have come to expect from managed TV-delivery networks.

The evolution of video delivery transport has led to a new set of de facto standard adaptive delivery protocols from Apple, Microsoft, Adobe that are now positioned for broad adoption. Consequently, networks must now be equipped with servers that can take high-quality video content from its source live or file format and ‘package’ it for transport to devices ready to accept these new delivery protocols.

Video Delivery Background

The Era of Stateful Protocols

For many years, stateful protocols including Real Time Streaming Protocol (RTSP), Adobe’s Real Time Messaging Protocol (RTMP), and Real Networks' RTSP over Real Data Transport (RDT) protocol were utilized to stream video content to desktop and mobile clients. Stateful protocols require that from the time a client connects to a streaming server until the time it disconnects, the server tracks client state. If the client needs to perform any video session control commands like start, stop, pause or fast-forward it must do so by communicating state information back to the streaming server.

Once a session between the client and the server has been established, the server sends media as a stream of small packets typically representing a few milliseconds of video. These packets can be transmitted over UDP or TCP. TCP overcomes firewall blocking of UDP packets, but may also incur increased latency as packets are sent, and resent if not acknowledged, until received at the far end.

These protocols served the market well, particularly during the era where desktop and mobile device experiences were limited by frequency, quality, duration, screen/window size/resolution, constrained processor, memory and storage capabilities of mobile devices, etc.

However, the above experience factors have all changed dramatically in the last few years. And that has exposed a number of stateful protocol implementation weaknesses:

• Stateful media protocols have difficulty getting through firewalls and routers.
  
  
• Stateful media protocols require special proxies/caches.
  
  
• Stateful media protocols cannot react quickly or gracefully to rapidly fluctuating network conditions.
  
  
• Stateful media client server implementations are vendor-specific, and thus require the purchase of vendor-specific servers and licensing arrangements – which are also more expensive to operate and maintain.

The Era of the Stateless Protocol – HTTP Progressive Download.

A newer type of media delivery is HTTP progressive download. Progressive download (as opposed to ‘traditional’ file download) pulls a file from a web server using HTTP and allows the video file to start playing before the entire file has been downloaded. Most media players including Adobe Flash, Windows Media Player, Apple Quicktime, etc., support progressive download. Further, most video hosting websites use progressive download extensively, if not exclusively.

HTTP progressive download differs from traditional file download in one important respect. Traditional files have audio and video data separated in the file. At the end of the file, a record of the location and structure of the audio and video tracks (track data) is provided. Progressively downloadable files have track data at the beginning of the file and interleave the audio and video data. A player downloading a traditional file must wait until the end of the file is reached in order to understand track data. A player downloading a progressively downloadable file gets track data immediately and can, therefore, play back audio/video as it is received.

Unfortunately, it isn’t possible to efficiently store the audio and video, and create progressive download files from live streams. Audio and video track data needs to be 1) computed after the entire file is created and, then 2) written to the front of the file. Thus, it isn’t possible to deliver a live stream using progressive download, because the track data can never be available until after the entire file has been created.

Even so, HTTP Progressive Download greatly improves upon its stateful protocol predecessors as a result of the following:

• No issue getting through firewalls and routers as HTTP traffic is passed through Port 80 unfettered.
  
  
• Utilizes the same web download infrastructure utilized by CDNs and hosting providers to provide web data content – making it much easier and less expensive to deliver rich media content.
  
  
• Takes advantage of newer desktop and mobile clients’ formidable processing, memory and storage capabilities to get start video playback quickly, maintain flow, and preserve a highquality experience.

The Modern Era – Adaptive HTTP Streaming

Adaptive HTTP Streaming takes HTTP video delivery several steps further. In this case, the source video, whether a file or a live stream, is encoded into segments – sometimes referred to as "chunks" – using a desired delivery format, which includes a container, video codec, audio codec, encryption protocol, etc. Segments typically represent two to ten seconds of video. Each segment is sliced at video Group of Pictures (GOP) boundaries beginning with a key frame, giving the segment complete independence from previous and successive segments. Encoded segments are subsequently hosted on a regular HTTP web server.

Clients request segments from the web server, downloading them via HTTP. As the segments are downloaded to the client, the client plays back the segments in the order received. Since the segments are sliced along GOP boundaries with no gaps between, video playback is seamless – even though it is actually just a file download via a series of HTTP GET requests.

Adaptive delivery enables a client to ‘adapt’ to fluctuating network conditions by selecting video file segments encoded to different bit rates. As an example, suppose a video file had been encoded to 11 different bit rates from 500 Kbps to 1 Mbps in 50 Kbps increments, i.e., 500 Kbps, 550 Kbps, 600 Kbps, etc. The client then observes the effective bandwidth throughout the playback period by evaluating its buffer fill/depletion rate. If a higher quality stream is available, and network bandwidth appears able to support it, the client will switch to the higher-quality bit rate segment. If a lower quality stream is available, and network bandwidth appears too limited to support the currently used bit rate segment flow, the client will switch to the lower quality bit rate segment flow. The client can choose between segments encoded at different bit rates every few seconds.

This delivery model works for both live- and file-based content. In either case, a manifest file is provided to the client, which defines the parameters of each segment. In the case of an on-demand file request, the manifest is sent at the beginning of the session. In the case of a live feed, updated ‘rolling window’ manifest files are sent as new segments are created.

Since the web server can typically send data as fast as its network connection will allow, the client can evaluate its buffer conditions and make forward-looking decisions on whether future segment requests should be at a higher or lower bit rate to avoid buffer overrun or starvation. Each client will make this decision based on trying to select the highest possible bit rate for maximum quality of playback experience, but not so great that it starves its own buffer of the next needed segments.

A number of advantages accrue with this delivery protocol approach:

• Lower infrastructure costs for content providers by eliminating specialty streaming servers in lieu of generic HTTP caches/proxies already in place for HTTP data serving.
  
  
• Content delivery is dynamically adapted to the weakest link in the end-to end-delivery chain, including highly varying last mile conditions.
  
  
• Subscribers no longer need to statically select a bit rate on their own, as the client can now perform that function dynamically and automatically.
  
  
• Subscribers enjoy fast start-up and seek times as playback control functions can be initiated via the lowest bit rate and subsequently ratcheted up to a higher bit rate.
  
  
• Annoying user experience shortcomings including long initial buffer time, disconnects, and playback start/stop are virtually eliminated.
  
  
• Client can control bit rate switching – with no intelligence in the server – taking into account CPU load, available bandwidth, resolution, codec, and other local conditions.
  
  
• Simplified ad insertion accomplished by file substitution.

Encoding/Transcoding

The transcoder (or encoder, if the input is not already compressed) is responsible for ingesting the content, encoding to all necessary outputs, and preparing each output for advertising readiness and delivery to the packager for segmentation. The transcoder must perform the following functions for multi-screen adaptive delivery – and at high concurrency, in real time and with high video quality output.

Video Transcoding

• Transcode the output video to a progressive format, which requires the transcoder to support input de-interlacing.
  
  
• Transcode the input to each required output profile – where a given profile will have its own resolution and bit rate parameters – including scaling to resolutions suitable for each client device. Because the quality of experience of the client depends on having a number of different profiles, it is necessary to encode a significant number of output profiles for each input. Deployments may use anywhere from 4 to 16 output profiles per input. The table below shows a typical use case for the different output profiles:
  
  
  
• GOP-align each output profile such that client playback (shifting between different bit rate ‘chunks’ created for each profile) is continuous and smooth.

Audio Transcoding

Transcode audio into AAC – the codec used by adaptive delivery protocols from Apple, Microsoft and Adobe.

Ad Insertion

Add IDR frames at ad insertion points, so that the video is ready for SCTE 35 ad insertion. It is also potentially possible to align chunk boundaries with ad insertion points so that ad insertion can be done via chunk-substitution rather than traditional stream splicing.

Ingest Fault Tolerance

The transcoding system needs to allow two different transcoders that ingest the same input to create identically IDR-aligned output – contributing to strong fault tolerance. This can be used to create a redundant backup of encoded content in such a way that any failure of the primary transcoder is seamlessly backed up by the secondary transcoder.

Packaging

To realize the benefits of HTTP adaptive streaming, a ‘packager’ function – sometimes referred to a ‘segmenter’, ‘fragmenter’ or ‘encapsulator’ – must take each encoded video output from the transcoder and ‘package’ the video for each delivery protocol. To perform this function, the packager must be able to:

Ingest

Ingest live streams or files, depending on whether the work flow is live or on-demand.

Segmentation

Segment chunks according to the proprietary delivery protocols specified by Microsoft Smooth Streaming, Apple HTTP Live Streaming (HLS), and Adobe HTTP Dynamic Streaming.

Encryption

• Encrypt segments on a per delivery protocol basis (in a format compatible with each delivery protocol) as they are packaged, enabling content rights to be managed on an individual session basis. For HLS, this is file-based AES-128 encryption. For Smooth Streaming, it is also AES-128, but with PlayReady compatible signaling. Adobe HTTP Dynamic Streaming uses Adobe Flash Access for encryption.
  
  
• Integrate with third party key management systems to retrieve necessary encryption information.

Note: Third party key management servers manage and distribute the keys to clients. If the client is authorized, it can retrieve decryption keys from a location designated in the manifest file. Alternatively, depending on the protocol used, key location can be specified within each segment. Either way, the client is responsible for retrieving decryption keys, which are normally served after the client request is authenticated. Once the keys are received, the client is able to decrypt the video and display it.

Delivery

The final step in this process is the actual delivery of segments to end clients – the aforementioned desktop/laptop/tablet PCs, smart phones, IP-based set-top boxes and now Internet-enabled television sets. Optimal delivery network design must take into consideration several content type, device type, delivery protocol type and DRM options.

Live vs. File Delivery

In the case of live delivery, it is possible to serve segments directly from the packager when the number of clients is relatively small. However, the typical use case involves feeding the segments to a CDN, either via a reverse proxy ‘pull’ or via a ‘push’ mechanism, such as HTTP POST. The CDN is then responsible for delivering the chunks and playlist files to clients.

The same delivery model can also be utilized in video-on-demand (VOD), but VOD also offers the alternative of delivering directly from the packager, even to a large number of users. However, with VOD delivery, it is sometimes desirable to distribute one file (or a small number of files) that contains all the chunks together; referred to as an aggregate format for the content. Distributing one file allows service providers to easily preposition content in the CDN without having to distribute and manage thousands of individual chunks per piece of content. When a client makes a request, the aggregate file is segmented ‘on the fly’ for that client, using the client’s requested format. The tradeoff is that while the CDN and file management is simpler, more packagers are required – ‘centralized’ packagers that create and aggregate the chunks and ‘distributed, edge-located’ packagers that segment the aggregation format (on demand) into actual chunks delivered to clients.

Output Profile Selection

The optimal number of profiles, bit rates and resolutions to use are very service-specific. However, there are a number of generally applicable guidelines. First, what are the end devices and what is the last-mile network? The end devices drive the output resolutions. It is desirable to have one or two profiles to service the high-quality video service for the device, and these would be encoded at the full resolution of the target device. For PCs, that’s typically 720p30.

Looking at the delivery network, for mobile distribution, it is typical to use very low bandwidth profiles. Even 3G mobile networks, which have relatively high peak bandwidths of several hundred kbps may fall back to much lower sustained bandwidths required for video streaming. WiFi networks have higher capacity, but also suffer from potential degradation depending on the distance to the base station or composition of walls between the transmitter and receiver. DSL distribution to PCs also varies widely in bandwidth capacity. And almost all last-mile networks suffer bandwidth reduction caused by aggregation, for example at a cable node or at the DSLAM. The table below suggests the number of output profiles in different scenarios:



Protocol Selection

Which of Apple HLS, Microsoft Silverlight Smooth Streaming or Adobe HTTP Dynamic Streaming is the optimal choice for a service provider? Each protocol has its own appeal, and so service providers must carefully consider the following in making a delivery protocol selection:

• Adobe has a huge installed client base on PCs. For service operators that want to serve PCs and do not want to distribute a client, this is a big benefit. The availability of Adobe’s server infrastructure, including backwards compatibility with RTMP and Adobe Access, may also be appealing to service operators.
  
  
• Apple HLS uses MPEG-2 transport stream files as chunks. The existing infrastructure for testing and analyzing TS files makes this protocol easy to deploy and debug. It also allows for the type of signaling that TS streams already carry, such as SCTE 35 cues for ad insertion points, multiple audio streams, EBIF data, etc.
  
  
• Microsoft Smooth Streaming has a very convenient aggregate format and provides an excellent user experience that can adapt to changes in bandwidth rapidly, as it makes use of short chunks and doesn’t require repeated downloads of a playlist. Smooth Streaming is also an obvious choice when content owners require the use of PlayReady DRM.

Redundancy and Failover

Transcoder redundancy is typically managed using an N:M redundancy scheme in which a management system loads one of M standby transcoders with the configuration of a failed transcoder in a pool of N transcoders. The packager component can be managed similarly, but it can also be managed in a 1:1 scheme by having time-outs in the CDN root failover to the secondary packager. Avoiding outages in these scenarios involves making sure that the primary and backup packagers are synchronized, that is, they create identical chunks.

DRM Integration

DRM integration remains challenging. Broadly, there are two approaches:

• The first uses unique encryption keys for every client stream. In this case, CDN caching provides no value. Every user view is a unicast connection back to the center of the network; network load is high; but the content is as secure as possible.
  
  
• The second approach uses shared keys for all content, but keys are only distributed to authenticated clients. CDN caching can then lead to significant bandwidth savings, but key management still requires a unique connection to the core for each client. Fortunately, these connections are far lower bandwidth than the video streams.

Different DRM vendors provide different solutions, and interoperability between vendors for client authentication doesn’t exist:

• Adobe uses Adobe Access to restrict access to streams, giving a unified, single vendor work flow.
  
  
• Apple HLS provides a description of the encryption mechanism, but leaves client authentication as an implementation decision.
  
• Microsoft’s PlayReady is a middle ground. Client authentication is well specified, but the interfaces between the key management server and the packager component is not. This means some integration is typically required to create a fully deployed system.

Apple HTTP Live Streaming (HLS)

HTTP Live Streaming (HLS) allows you to stream live and on-demand video and audio to an iPhone, iPad, or iPod Touch. HLS is similar to Smooth Streaming in Microsoft’s Silverlight platform architecture, and can be thought of as a successor to both RTSP and HTTP Progressive Download (HTTP PD), although both of those video options serve a purpose and likely won’t be going away anytime soon.

HLS was originally unveiled by Apple with the introduction of the iPhone 3.0 in mid-2009. Prior to the iPhone 3, no streaming protocols were supported natively on the iPhone, leaving developers to wonder what Apple had in mind for native streaming support. Apple proposed HLS as a standard to the IETF, and the draft is now in its sixth iteration.

As an adaptive streaming protocol, HLS has several advantages including multiple bit rate encoding for different devices, HTTP delivery, and segmented stream chunks suitable for delivery of live streams over widely available HTTP CDN infrastructure.

How HLS Works

HLS lets you send streaming video and audio to any supported Apple product, including Macs with a Safari browser. HLS works by segmenting video streams into 10-second chunks; the chunks are stored using a standard MPEG-2 Transport Stream file format. Optionally, chunks are created using several bit rates, allowing a client to dynamically switch between different bit rates depending on network conditions.

How does a stream get into HLS format? There are three main steps:

• An input stream is encoded/transcoded. The input can be a satellite feed or any other typical input. The video and audio source is encoded (or transcoded) to an MPEG-2 Transport Stream container, with H.264 video and AAC audio, which are the codecs Apple devices currently support.
  
  
• Output profiles are created. Typically a single input stream will be transcoded to several output resolutions/bit rates, depending on the types of client devices that the stream is destined for. For example, an input stream of H.264/AAC at 7 Mbps could be transcoded to four different profiles with bit rates of 1.5Mbps, 750K, 500K, and 200K. These would be suitable for devices and network conditions ranging from high-end to low-end, such as an iPad, iPhone 4, iPhone 3, and a low bit rate version for bad network conditions.
  
  
• The streams are segmented. The streams contained within the profiles all need to be segmented and made available for delivery to an origin web server or directly to a client device over HTTP. The software or hardware device that does the segmenting (the segmenter) also creates an index file which is used to keep track of the individual video/audio segments.

Optionally, the segmenter might also encrypt the stream (each individual chunk) and create a key file.



What Does the Client Do?

The client downloads the index file via a URL that identifies the stream. The index file tells the client where to get the stream chunks (each with its own URL). For a given stream, the client then fetches each stream chunk in order. Once the client has enough of the stream downloaded and buffered, it displays it to the user. If encryption is used, the URLs for the decryption keys are also given in the index file.

If multiple profiles (that is, bit rates and resolutions) are available, the index file is different in that it contains a specially tagged list of variant index files for the different stream profiles. In that case, the client downloads the primary index file first, and then downloads the index file for the bit rate it wants to play back. The bit rates and resolutions of the variant streams are specified in the main index file, but precise handling of the variants offered is left up to the client implementation.

Typical playback latency for HLS is around 30 seconds. This is caused by the size of the chunks (10 seconds) and the need for the client to buffer a number of chunks before it starts displaying the video.

An odd curiosity about HLS is that it doesn’t make use of Apple’s Quicktime MOV file format, which is the basis for the ISO MPEG file format. Apple thought that the TS format was more widely used and better understood by the broadcasters who would ultimately use HLS, and so they focused on the MPEG-2 TS format. Ironically, Microsoft’s Smooth Streaming protocol does make use of ISO MPEG files (MPEG-4 Part 12).

For more information on how HLS works, good resources include Apple’s Live Streaming overview documentation.

Microsoft Smooth Streaming (SS)

Smooth Streaming was announced by Microsoft in October 2008 as part of the Silverlight architecture. That year they demonstrated a prototype version of Smooth Streaming by delivering live and on-demand streaming content such as the Beijing Olympics and Democratic National Convention.

Smooth Streaming has all of the typical characteristics of adaptive streaming. The video content is segmented into small chunks, it is delivered over HTTP, and usually multiple bit rates are encoded so that the client can choose the best video bit rate to deliver an optimal viewing experience based on network conditions.

Adaptive streaming is valuable for many reasons including low web-based infrastructure costs, firewall compatibility and bit rate switching. Microsoft is definitely a believer in those benefits as it is making a strong push to adaptive streaming technology with Microsoft Silverlight, Smooth Streaming and Mediaroom.

Smooth Streaming vs. Apple HLS

Microsoft has chosen to implement adaptive streaming in unique ways, however. There are several key differences between Silverlight Smooth Streaming and HLS:

• HLS makes use of a regularly updated “moving window” metadata index file that tells the client which chunks are available for download. Smooth Streaming uses time codes in the chunk requests and thus the client doesn’t have to repeatedly download an index file.
  
  
• Because HLS requires a download of an index file every time a new chunk is available, it is desirable to run HLS with longer duration chunks, thus minimizing the number of index file downloads. Thus, the recommended chunk duration with HLS is 10 seconds, while with Smooth Streaming it is 2 seconds.
  
  
• The “wire format” of the chunks is different. Both formats use H.264 video encoding and AAC audio encoding, but HLS makes use of MPEG-2 Transport Stream files, while Smooth Streaming makes use of “fragmented” ISO MPEG-4 files. The “fragmented” MP4 file is a variant in which not all the data in a regular MP4 file is included in the file. Each of these formats has some advantages and disadvantages. MPEG-2 TS files have a large installed analysis toolset and have pre-defined signaling mechanisms for things like data signals (e.g. specification of ad insertion points). Fragmented MP4 files are very flexible and can easily accommodate all kinds of data, for example decryption information, that MPEG-2 TS files don’t have defined slots to carry.

For a more in-depth overview of how Microsoft Smooth Streaming works, a good resource is Microsoft’s Smooth Streaming Technical overview whitepaper.

Additionally, for a slightly biased but still accurate representation of some of the key differences between Smooth Streaming, HLS, and Adobe Flash Dynamic Streaming, check out this adaptive streaming comparison matrix.

Adobe HTTP Dynamic Streaming (HDS)

HTTP Dynamic Streaming was announced by Adobe in late 2009 as “project Zeri” and was delivered in June 2010. HDS is more similar to Microsoft Smooth Streaming than it is to Apple HLS. Primarily this is because it uses a single aggregate file from which MPEG-file container fragments are extracted and delivered rather than HLS-like individual chunks, and consequently there are certain implications of that design, which will be discussed in detail.

Characteristics of Adobe HDS

HTTP Dynamic Streaming supports both live and on-demand content using a standard MP4 fragment format (F4F). Video/audio codec support includes VP6/MP3 and H.264/AAC, however as with HLS and SS, the predominant video/audio codecs are H.264/AAC.

Similar to other adaptive streaming protocols, at the start of the stream, the client or CDN/origin server downloads the manifest file (in this case F4M file) which provides all the information needed to play back the content, including fragment format, available bitrates, Flash Access license server location, and metadata information.

Files representing either live or VOD workflows are sent to an HTTP origin server. The origin server is responsible for receiving segment requests from the client over HTTP and returning the appropriate segment from the file. Standard origin servers like Apache can leverage Adobe’s Open Source Media Framework (OSMF) to serve the content.

Differences Between Adobe HDS and Apple HLS

There are several key differences between Adobe HDS and Apple HLS:

• HLS makes use of a regularly updated “moving window” metadata index (manifest) file that tells the client which chunks are available for download. Adobe HDS uses sequence numbers in the chunk requests and thus the client doesn’t have to repeatedly download a manifest file.
  
  
• In addition to the manifest, there is a bootstrap file, which in the live case gives the updated sequence numbers and is equivalent to the repeatedly downloaded HLS playlist.
  
  
• Because HLS requires a download of a manifest file as often as every time a new chunk is available, it is desirable to run HLS with longer duration chunks, thus minimizing the number of manifest file downloads. More recent Apple client versions appear to now check how many segments are in the playlist and only re-fetch the manifest when the client runs out of segments. Nevertheless, the recommended chunk duration with HLS is 10 seconds, while with Adobe HDS it is usually 2-5 seconds.
  
  
• The “wire format” of the chunks is different. Both formats use H.264 video encoding and AAC audio encoding, but HLS makes use of MPEG-2 Transport Stream files, while Adobe HDS (and Microsoft SS) make use of “fragmented” ISO MPEG-4 files.

As with HLS, Adobe Flash clients first request a manifest file. The manifest contains information about what streams are available, bit rates, codecs, etc. and the streams are represented by a URL. Using a contiguous file creates two significant changes in the client/server architecture:

• The client reads the manifest and can request chunks by a URL with a sequence number rather than a specific chunk name.
  
  
• The server must calculate exact byte range offsets within the aggregate file by translating URL requests and delivers the appropriate chunk.

For a more in-depth overview of how Adobe HDS works, a good resource is Adobe’s HTTP Dynamic Streaming technical whitepaper.

By Andy Salo, RGB Networks