The State of Streaming Media Protocols 2013

Protocols are a geeky thing, almost up there with metadata or IP address schemes. Without protocols, there would be no concept of a webpage, email delivery, or even VoIP (voice or video over IP). In fact, there would be no streaming of any flavor.

With that in mind, let's take a quick look at the latest developments in a few key protocols, some of which are just coming into widespread use for streaming.

On Time or On Target? The UDP Question
You may have heard the truism that "time is money." We often use that concept when we talk about hardware- or software-based transcoding: If a job needs to be done quickly, even faster than real time, go for the hardware; if time is less critical, use software.

The same concept, slightly morphed, can be applied to the world of streaming protocols: If low latency is key, go with UDP (User Datagram Protocol), but if delivery guarantee is more critical, go with TCP (Transmission Control Protocol). The latter provides control over guaranteed packet delivery -- the control in the transmission control protocol name -- while the former does not.

Every other protocol that we discuss in this article will hinge on TCP, but recent advances on the UDP front bear mention.

I'm not going to retrace step by step the key points that Dom Robinson travels in his recent article titled Reliable UDP (RUDP): The Next Big Streaming Protocol?, but I do want to point out several highlights.

First, in and of itself UDP is unreliable, but it's fast and efficient. The protocol itself has no mechanism to guarantee delivery or request missing packets, as does TCP. A properly constructed application, though, can act as the first line of defense in detecting loss or packet corruption with subsequent request for dropped packets.

"It can take TCP upward of 3 seconds to renegotiate for the sequence to restart from the missing point," wrote Robinson, "discarding all the subsequent data, which must be requeued to be sent again. Just one lost packet can cause an entire ‘window' of TCP data to be re-sent."

Second, UDP can work hand-in-hand with error-correction techniques. Many legacy intermittent networks, including those based on asynchronous transfer protocols such as ATM, used Forward Error Correction (FEC) to anticipate intermittent outages. FEC provides "packet flooding" at a certain percentage above 100% of packets -- be it 15%, 20%, 30% -- to then allow the client application to reconstruct the missing or corrupt packets without the need to request that packets be retransmitted.

Third, the concept of reliable UDP has been around for quite some time and can be accomplished with a number of open source tools, but there are also a number of commercial licensees that offer tools based on RUDP.

Your mileage may vary if you want to tinker under the hood of a freeware application such as UDP Data Transfer, or you may just want to reach out to a vendor in the RUDP space. Regardless of your choice, there are enough RUDP options out there to warrant research, especially if very low latencies and FEC are part of your workflow.

HTTP Über Alles
The fanfare around Dynamic Adaptive Streaming over HTTP, or DASH for short, continues to build at a rate we'd only ever seen back in the early streaming days of Real versus Microsoft. DASH was ratified back in late 2011, but its adoption has moved at a rapid pace.

Let's take a look at the HTTP flavors out there in current circulation: HDS, HLS, MPEG DASH, and Smooth Streaming. We'll look first at DASH followed, in alphabetical order, by the Adobe, Apple, and Microsoft proprietary offerings.

DASH It All, or Just 264?
Due to DASH's ability to deliver any of several types of files -- from the fragmented MP4 version of ISO Base Media File Format (ISOBMFF) to the Apple-modified MPEG2 Transport Stream (M2TS) -- the DASH specification reads like an encyclopedia of encoding, encryption, and delivery technologies.

To offset the potential problem that plagued MPEG 4 system -- a wide-ranging specification, based on Apple QuickTime, that was too complex to easily implement -- the industry has begun looking at options and commonalities of all the major HTTP delivery solutions. It has, as of the time of this writing, begun drafting a subset specification that centers on H.264 served as fragmented MP4. This use of ISOBMFF and H.264 is being dubbed as DASH 264.

What impact the DASH 264 specification has on the potential of bringing Apple into the DASH fold -- it was a contributor to the DASH specification, but is the only M2TS-based HTTP solution -- remains to be seen. But several industry players we spoke to stated the strong need to get solid DASH implementations into the field in early 2013.

Adobe's Flash vs. DASH Conundrum
While Adobe wouldn't publicly come out in support of DASH, despite co-sponsoring an ISOBMFF white paper in late 2011, the company did put its weight behind DASH in late February 2012.

"I am excited to announce that Adobe's video solutions will adopt the emerging video standard, MPEG-DASH across our video streaming, playback, protection and monetization technologies," wrote Kevin Towes in an Adobe blog post. "Adobe will support MPEG-DASH ISOBFF on demand and live profiles which are recommended in DASH-264 recommendation."

Towes anticipated the question many would ask: What about Flash and the Adobe HTTP Dynamic Streaming (HDS) protocol? He noted that Adobe will continue to push forward with HDS, even as Adobe supports DASH.

"Adobe will continue developing its HDS format used to deliver high quality, protected video experiences across multiple devices," wrote Towes, noting that the DASH profile for ISOBMFF "is similar to Adobe's HDS format and supports many of the performance objectives of the HDS format."

The continued development of HDS makes sense, as it offers Adobe the chance to try out new functionality within the confines of the Flash Player, but as we move into 2013, we wonder whether this is a sustainable model.

Adobe MAX 2013, to be held in May, may shed some light on HDS -- and perhaps even RTMP -- but meanwhile Adobe continues to show DASH functionality in the Flash Player.

An Apple (Protocol) a Day
Much has been made of the idea that HTTP Live Streaming (HLS) is a standard in the marketplace, but as one panelist at a recent DASH event quipped, "The IETF drafts of the Pantos spec are more a suggestion than a standard."

The Pantos spec, as it is known in the industry, is a series of working drafts for HLS submitted by two Apple employees as an information draft for the Internet Engineering Task Force. As of the time of this article, the Pantos spec is currently at informational version 10.

Much has changed between the early versions and the most recent v10 draft, but one constant remains: HLS is based on the MPEG-2 Transport Streams (M2TS), has been in use for almost 2 decades, and is deployed widely for varied broadcast and physical media delivery solutions.

In that time frame, however, little has changed for basic M2TS transport stream capabilities. For instance, M2TS still lacks an integrated solution for Digital Rights Management (DRM). As such, all HLS versions cannot use "plain vanilla" M2TS, and even the modified M2TS used by Apple lacks timed-text or closed-captioning features found in more recent fragmented elementary stream streaming formats.

Yet Apple has been making strides in addressing the shortcomings of both M2TS and the early versions of HLS: In recent drafts, the HLS informational draft allows for the use of elementary streams, which are segmented at the time of demand rather than beforehand. This use of elementary streams means that one Achilles' heel of HLS -- the need to store thousands, tens of thousands, or hundreds of thousands of small segments of long-form streaming content -- is now eliminated.

Google, with its Android mobile operating system platform, has adopted HLS for Android OS 4. Some enterprising companies have even gone back and created HLS playback for earlier versions of Android OS-based devices.

Smooth Streaming Ahead?
No discussion of HTTP protocol-based streaming delivery would be complete without a mention of Microsoft Smooth Streaming. After all, the Protected Interoperable File Format (PIFF) is the basis for the Common File Format (CFF) that is being used for UltraViolet, and the Common Encryption Scheme (CES) is based partly on Microsoft's 2008 idea that common encryptions and encoding could be implemented for use around the fragmented MP4 standard.

As we enter 2013, rationalization of PIFF-CFF-CES and the upcoming Common Streaming Format (CSF) will continue to pare down the number of options, which is a good thing if we are to get back to the business of creating content and delivering it to anyone who wants to view it.

Yet Microsoft isn't resting on its laurels, as the company announced in late 2012 that it would be supporting Smooth Streaming via the Open Source Media Framework (OSMF) that is part of Adobe's Strobe initiative for Flash. Yes, you heard that right: Not only does Flash Player support DASH in beta (thanks to Adobe), but it now supports Smooth Streaming (thanks to Microsoft).

Is RTSP Dead?
One of the most venerable streaming protocols, the Real-Time Streaming Protocol, has been implemented natively into every type of device: set-top boxes, smartphones, tablets, and PCs. Yet these implementations are often fraught with buggy code, limited support, and a number of oddities.

In testing performed on a number of Android OS devices, we have been surprised to find that RTSP-based video playback -- served from a standards-based server -- could not be played with built-in applications and services, despite the requirement for the base Android OS to be able to play this content. Content that would play consistently on numerous RTSP implementations would stop dead in its tracks on other devices.

This was true of any standards-based streaming protocol in the late 1990s, but these days, consumers just expect their content to stream with limited buffering and at varying data rates. RTSP offers neither of these as certainty, but is quite inexpensive to implement, so we suspect that it will be with us for at least a few more years before retiring to greener pastures to make way for DASH and more recent streaming protocols.

Where Does RTMP Fit?
At a recent informational meeting I attended with a major software company, a slide that was shown caught my attention, more for the lack of what was shown than for what the slide contained.

This particular slide listed the typical agnosticism -- codec, protocol, player -- of their soon-to-be-released update, and it had the regular litany of compatibilities. Yet what caught my attention was that the sparsest, by far, was the protocol list.

Only two protocols were noted -- HTTP and RTMP -- so I asked why RTMP was listed when other non-HTTP protocols were not. To summarize their response, they said RTSP and other non-HTTP protocols weren't being requested at all, but RTMP was still a valid industry solution.

Part of the reason lies in the fact that RTMP is "true" streaming with very low latencies and session "statefulness" that can't yet be found in HTTP-based delivery. In addition, RTMP is firmly entrenched on the vast majority of devices -- with the exception of the iOS devices -- thanks to the inclusion of the Flash Player on handsets, tablets, mobile devices, and PCs.

Yet, for all that entrenchment, as we've noted previously, Adobe continues to lean toward the HTTP delivery model in all the important ways including monetization functionality. We're not ruling out RTMP, but we do understand that the scalability and interoperability of HTTP solutions such as MPEG DASH and HLS offer compelling reasons to surf the fine streaming waves coming out of Apache servers everywhere.

Conclusion
So what does 2013 hold? We think the year is DASH's to lose.

Once DASH is officially supported in Flash, we see the possibility that DASH will be firmly enough entrenched to begin "hockey stick" growth for online video delivery. If DASH 264 can be implemented as quickly as it appears it will be ratified, and if there is some form of rationalization between HLS and DASH, including the ability to include Apple's DRM scheme in the Common Encryption Scheme, we might just note 2013 not only as the beginning of true online video delivery growth but also as the point at which cable and satellite providers begin to pay attention to delivery to all devices -- including set-top boxes -- for a true TV Everywhere experience.

By Tim Siglin, StreamingMedia