What is HLS (HTTP Live Streaming)?
HTTP Live Streaming (or HLS) is an adaptive streaming protocol created by Apple to communicate with iOS and Apple TV devices and Macs running OSX in Snow Leopard or later. HLS can distribute both live and on-demand files and is the sole technology available for adaptively streaming to Apple devices, which is an increasingly important target segment to streaming publishers.
HLS is widely supported in streaming servers from vendors like Adobe, Microsoft, RealNetworks, and Wowza, as well as real time transmuxing functions in distribution platforms like those from Akamai. The popularity of iOS devices and this distribution-related technology support has also led to increased support on the player side, most notably from Google in Android 3.0.
In the Apple App Store, if you produce an app that delivers video longer than ten minutes or greater than 5MB of data, you must use HTTP Live Streaming, and provide at least one stream at 64Kbps or lower bandwidth. Any streaming publisher targeting iOS devices via a website or app should know the basics of HLS and how it’s implemented.
How HLS Works
At a high level, HLS works like all adaptive streaming technologies; you create multiple files for distribution to the player, which can adaptively change streams to optimize the playback experience. As an HTTP-based technology, no streaming server is required, so all the switching logic resides on the player.
To distribute to HLS clients, you encode the source into multiple files at different data rates and divide them into short chunks, usually between 5-10 seconds long. These are loaded onto an HTTP server along with a text-based manifest file with a .M3U8 extension that directs the player to additional manifest files for each of the encoded streams.
The player monitors changing bandwidth conditions. If these dictate a stream change, the player checks the original manifest file for the location of additional streams, and then the stream-specific manifest file for the URL of the next chunk of video data. Stream switching is generally seamless to the viewer.
of audio/video data within those streams.
HLS File Preparation
HLS currently supports H.264 video using the Baseline profile up to Level 3.0 for iPhone and iPod Touch clients and the Main profile Level 3.1 for the iPad 1 and 2. Audio can be HE-AAC or AAC-LC up to 48 KHz, stereo. The individual manifest files detail the profile used during encoding so the player will only select and retrieve compatible streams. This allows producers to create a single set of HLS files that will serve iPhone/iPod touch devices with Baseline streams and iPads with streams encoded using the Main profile.
Though encoded using the H.264 video codec and AAC audio codec, audio/video streams must be segmented into chunks in an MPEG-2 Transport Stream with a .ts extension. All files are then uploaded to an HTTP server for deployment. In a live scenario, the .ts chunks are continuously added and the .M3U8 manifest files continually updated with the locations of alternative streams and file chunks.
Before producing files for HLS, you should read through Apple’s Tech Note TN2224 which contains detailed recommended configurations (resolution, data rate, keyframe interval) for distributing both 4:3 and 16:9 video to all compatible iDevice and Apple TV players.
Content Protection and Closed Captions in HLS
HLS doesn’t natively support digital rights management (DRM) though you can encrypt the data and provide key access using HTTPS authentication. There are several third-party DRM solutions becoming available, including from AuthenTec, SecureMedia, and WideVine.
HLS can support closed captions included in the MPEG-2 Transport Stream.
Deploying HLS Streams
Delivery via HTTP has several advantages; no streaming server is required and the audio/video chunks should leverage HTTP caching servers located in the premises of internet service providers, cellular providers, and other organizations, which should improve video quality for viewers served from these caches. HTTP content should also pass through most firewalls.
Apple recommends using the HTML5 video tag for deploying HLS video on a website.
On the Playback Side
On computers and iPad devices, the Safari browser can play HLS streams within a web page, with Safari launching a full-screen media player on iPhones and iPod touch devices. Starting with version 2, all Apple TV devices include an HTTP Live Streaming client.
Producing HLS
As discussed, the HLS experience has two components: a set of chunked files in .ts format and the required manifest files. In an on-demand environment, you can encode the alternative files using any standalone H.264 encoding tool, with the latest version of Sorenson Squeeze offering a multiple file HLS encoding template. More recently, Telestream updated Episode to include command line HLS multiple file creation. Cloud encoding services like those provided by Encoding.com can also typically produce HLS-compatible files.
Once you have the encoded streams, you can use Apple tools to create the chunked files and playlists:
- Media Stream Segmenter - Inputs an MPEG-2 Transport Stream and produces chunked .ts files and index files. It can also encrypt the media and produced encryption keys.
- Media File Segmenter - Inputs H.264 files and produces chunked .ts files and index files. It can also encrypt the media and produced encryption keys.
- Variant Playlist Creator - Compiles the individual index files created by the Media Stream or Media File Segmenter into a master .M3U8 file that identifies the alternate streams.
- Metadata Tag Generator - Creates ID3 metadata tags that can either be written to a file or inserted into outgoing stream segments.
- Media Stream Validator - Examines index files, stream alternates, and chunked .ts files to validate HLS compatibility.
For live HLS distribution, you need an encoding tool that can encode the files into H.264 format, create the MPEG-2 Transport Stream chunks and create and update the manifest files. When Apple first announced HLS in 2009, only two live encoders were available; one each from Inlet (now Cisco) and Envivio. Now most vendors of encoding hardware also offer live HLS-compatible products, including Digital Rapids, Elemental Technologies, Haivision, Seawell Networks, and ViewCast.
Real Time Transmuxing
The other approach to live or on-demand streaming to HLS-compatible players is via transmuxing, which is offered by multiple streaming server vendors and CDNs. Specifically, these servers input an H.264-stream originally compatible with Flash or Silverlight (or other formats) and then dynamically re-wrap the file into the required MPEG-2 Transport Stream chunks and create the required manifest files.
Server-based implementations include:
- Adobe Flash Media Server 4.5
- Wowza Media Server
- Microsoft IIS Media Services
- RealNetworks Helix Universal Server
Akamai also offers “in the network” repackaging of H.264 input files for HLS deployment.
In these applications, any live encoding tool that can deliver multiple streams of input to the server, like the Adobe Flash Live Media Encoder, Haivision, Microsoft Expression Encoder Pro, or Telestream Wirecast, can serve as the encoding front end for multiple-platform adaptive distribution including HLS.
Not surprisingly given the level of technology support, many of the larger online video platforms are now starting to support HLS distribution, including Brightcove, Kaltura, and Ooyala.
Conclusion
The iOS platform is a critical target for virtually all streaming publishers, and HLS can deliver the best possible experience to that platform, and others that support HLS playback. Fortunately, the streaming industry had embraced HLS with tools and technologies that make this very simple and affordable.
HLS Resources
One of the reasons that HLS has been so successful is that Apple has created multiple documents that comprehensively address the creation and deployment of HLS files.
You can also watch this video tutorial.
By Jan Ozer, StreamingMedia
