An Introduction to LTFS for Digital Media

We live in an age where content is king. For the entertainment and media industry, the majority of this content is now produced in digital form, and virtually all of this content has digital distribution; that content is now digital data. Protecting that content, the lifeblood of this industry, with the right data storage system is more important than ever.

Tape is already firmly established in the media production environment whether to secure on-set content or for long-term archiving. LTFS broadens LTO technology usefulness by being easier to use and more robust. The open and self-contained LTFS format is useful if the tapes are to be sent offsite, archived or shared with a variety of recipients.

The LTFS standard was adopted by the LTO Program in April 2010. It is an open format and software specification that supports simpler and new ways to access data on tape. Although the tape model hasn’t changed dramatically over the years, the speed, storage density and features of data tape have improved significantly, ultimately providing reliable and inexpensive storage as a sequential storage medium. With LTFS, accessing files stored on the LTFS-formatted media is similar to accessing files stored on other forms of storage media, such as disks or removable USB flash drives. It has no application software dependencies, offers support for large and numerous files, and often can have a lower total cost than traditional managed tape storage.

How LTFS Works
LTFS consists of a software driver and the format specification. Drivers, some free and open source, are available for various operating and tape systems from the tape hardware vendor websites.

The format is a self-describing tape format and defines the organization of data and metadata on tape. Within the partitions, the tape contents are still stored as usual, as blocks of data and file marks. Files are mapped by LTFS to a hierarchical directory structure.

LTFS uses media partitioning, where tape is logically divided “lengthwise” into two partitions:

• Index partition: Contains file system info, index, metadata.
• Content partition: Contains the files and content bodies.

The standard LTO-5 tape cartridge is segmented into two partitions, one for the index and one for the data, so the index partition can be modified as needed without affecting the append-only data partition. This creates a self-describing tape where a user can see the tape cartridge and its contents in the operating system directory tree browser and can copy, paste or drag and drop files/folders to and from the tape. Similarly, applications can access the data on tape directly, unaware they are using tape, though there may be differences in latency inherent to tape.

When the tape is inserted and mounted in a tape drive, the information in the index partition is read and cached in the workstation’s memory. From that point on, the index is accessed and updated in the workstation memory for fast performance and so the tape head can stay positioned in the content portion of the tape to more quickly access files or write new ones.

As the tape is used, the index is updated in the workstation memory for fast performance, eliminating the need to go back to the beginning of the tape. To protect the index, LTFS periodically copies the index from the workstation’s memory to the data partition. When the tape is done being used (unmounted), the index is copied one more time at the end of the data partition, the tape is rewound and then the index is written twice to the index partition. Essentially, there are multiple copies of the index on tape for restoring in the unlikely event the index partition index is not usable or if a user wants to roll back the tape to a previous version.

For instance, you may want to restore it to the way the tape appeared last Monday by choosing an index from that date. In some tape library implementations, the tape cartridge’s indexes are cached to a server disk for fast searching without having to remount the tape cartridges. LTFS also supports extended attributes, which enable custom file metadata.


Workflow
LTFS opens up new opportunities for media and entertainment storage and distribution, providing support to media workflows. LTFS directly affects two major trends having significant impact on media and entertainment companies and digital media producers: the shift to file-based workflows and increasing storage demands.

With the advent of digital technologies, moving image (video and film) content producers and distributors are transitioning from analog/linear workflows based on film or videotape technology to digital/nonlinear workflows based on the manipulation of data files. Many organizations have already completed this transition. This has been commonly referred to as “moving to a tapeless workflow,” though it is more accurate to call it moving to a “videotapeless” workflow.

The other fact of life in media and entertainment is the demand of ever-increasing visual resolution and complexity (HD, 4K, 3-D, etc.) creating more and larger files that must be managed. Keeping hours of such media online on disk quickly becomes cost prohibitive. LTO tape, especially with LTFS, can address the challenge. LTFS can work with LTO hardware-based lossless compression, which can provide bandwidth and capacity benefits depending on the data content.


Production
LTO tape is already firmly established in the media production environment. For production, LTFS efficiently supports several important requirements.

• Camera media reuse: Digital cameras encode motion images directly to SSDs or removable disks in the camera. These media are quite expensive (three to more than 350 times the equivalent media cost on LTO). Fast transfer of their contents to tape with LTFS enables reuse of this expensive media, reducing the number of SSDs or disks that must be purchased or rented.


• Backup: Backup of daily footage to LTO tape is a common requirement as the loss of a day’s worth of production is costly. LTFS facilitates backup by enabling small portable independent systems to easily write daily content to tape.


• Transport: The density and cost of LTO-5 tapes with the self-describing capabilities of LTFS combine to create an effective transport medium. Large amounts of data can be sent more quickly and economically than network-based transmission methods. This is especially compelling for digital productions, which can produce terabytes of data for every day of shooting. The encryption features of LTO tape help secure the data in transit.


• Economic direct access to data: For any file-based production workflow, an LTFS-enabled tape drive can feed workstations or networks with content directly and relatively quickly, similar to a disk and unlike most traditional tape systems. An application via the operating system always has a direct and persistent view of a mounted LTFS tape and the files it contains. Consequently, in a workflow where access to a file is expected to be fast but not instantaneous, such as a stock footage collection or archive footage of an ongoing news story, an LTFS tape is an effective and economical choice for storage.


• Archive: LTFS-formatted tapes can be easily imported into an LTFS-compatible archive by simply reading the index and adding the file metadata to an archive manager’s catalog.

Conversely, traditional systems that use separate media for transport and archive require all the data be recopied. With LTFS, there is no need to read the much larger data partition or transfer the data to other storage media. The transport media and the archive storage media are one and the same under this scenario. The “import bandwidth” of tapes being added directly to a library en masse far exceeds any system that requires movement of the actual data.

By Rainer Richter, Broadcast Engineering

Reliable UDP (RUDP): The Next Big Streaming Protocol?

Those who have had a little experience will probably have heard of TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). They are transport protocols that run over IP links, and they define two different ways to send data from one point to another over an IP network path. TCP running over IP is written TCP/IP; UDP in the same format is UDP/IP.

TCP has a set of instructions that ensures that each packet of data gets to its recipient. It is comparable to recorded delivery in its most basic form. However, while it seems obvious at first that "making sure the message gets there" is paramount when sending something to someone else, there are a few extra considerations that must be noted. If a network link using TCP/IP notices that a packet has arrived out of sequence, then TCP stops the transmission, discards anything from the out-of-sequence packet forward, sends a "go back to where it went wrong" message, and starts the transmission again.

If you have all the time in the world, this is fine. So for transferring my salary information from my company to me, I frankly don't care if this takes a microsecond or an hour, I want it done right. TCP is fantastic for that.

In a video-centric service model, however, there is simply so much data that if a few packets don't make it over the link there are situations where I would rather skip those packets and carry on with the overall flow of the video than get every detail of the original source. Our brain can imagine the skipped bits of the video for us as long as it's not distracted by jerky audio and stop-motion video. In these circumstances, having an option to just send as much data from one end of the link to the other in a timely fashion, regardless of how much gets through accurately, is clearly desirable.

It is for this type of application that UDP is optimal. If a packet seems not to have arrived, then the recipient waits a few moments to see if it does arrive -- potentially right up to the moment when the viewer needs to see that block of video -- and if the buffer gets to the point where the missing packet should be, then it simply carries on, and the application skips the point where the missing data is, carrying on to the next packet and maintaining the time base of the video. You may see a flicker or some artifacting, but the moment passes almost instantly and more than likely your brain will fill the gap.

If this error happens under TCP then it can take TCP upward of 3 seconds to renegotiate for the sequence to restart from the missing point, discarding all the subsequent data, which must be requeued to be sent again. Just one lost packet can cause an entire "window" of TCP data to be re-sent. That can be a considerable amount of data, particularly when the link is known as a Long Fat Network link (LFN or eLeFaNt; it's true -- Google it!).

All this adds overhead to the network and to the operations of both computers using that link, as the CPU and network card's processing units have to manage all the retransmission and sync between the applications and these components.

For this reason HTTP (which is always a TCP transfer) generally introduces startup delays and playback latency, as the media players need to buffer more than 3 seconds of playback to manage any lost packets.

Indeed, TCP is very sensitive to something called window size, and knowing that very few of you ever will have adjusted the window size of your contribution feeds as you set up for your live Flash Streaming encode, I can estimate that all but those same very few have been wasting available capacity in your network links. You may not care. The links you use are good enough to do whatever it is you are trying to do.

In today's disposable culture of "use and discard" and "don't fix and reuse," it's no surprise that most streaming engineers just shrug and assume that the ability to get more bang for your buck out of your internet connection is beyond your control.

For example, did you know that if you set your Maximum Transmission Unit (MTU) -- ultimately your video packet size -- too large then the network has to break it in two in a process called fragmentation? Packet fragmentation has a negative impact on network performance for several reasons. First, a router has to perform the fragmentation -- an expensive operation. Second, all the routers in the path between the router performing the fragmentation and the destination have to carry additional packets with the requisite additional headers.

Also, in the event of a retransmission, larger packets increase the amount of data you need to resend if a retransmission occurs.

Alternatively, if you set the MTU too small then the amount of data you can transfer in any one packet is reduced and relatively increases the amount of signaling overhead (the data about the sending of the data, equivalent to the addresses and parcel tracking services in real post). If you set the MTU as small as you can for an Ethernet connection, you could find that the overhead nears 50% of all traffic.

UDP offers some advantages over TCP. But UDP is not a panacea for all video transmissions.

Where you are trying to do large-video file transfer, UDP should be a great help, but its lossy nature is rarely acceptable for stages in the workflow that require absolute file integrity. Imagine studios transferring master encodes to LOVEFiLM or Netflix for distribution. If that transfer to the LOVEFiLM or Netflix playout lost packets then every single subscriber of those services would have to accept that degraded master copy as the best possible copy. In fact, if UDP was used in these back-end workflows, the content would degrade the user's experience in the same way that historically tape-to-tape and other dubbed and analog replication processes used to. Digital media would lose that perfect replica quality that has been central to its success.

Getting back to the focus on who may want to reduce their network capacity inefficiencies: Studios, playouts, news desks, broadcast centers, and editing suites all want their video content intact/lossless, but naturally they want to manipulate that data between machines as fast as possible. Having video editors drinking coffee while videos transfer from one place to another is inefficient (even if the coffee is good).

Given they cannot operate in a lossy way, are these production facilities stuck with TCP and all the inherent inefficiencies that come with the reliable transfer? Because TCP ensures all the data gets from point to point, it is called a "reliable" protocol. In UDP's case, that reliability is "left to the user," so UDP in its native form is known as an "unreliable" protocol.

The good news is that there are indeed options out there in the form of a variety of "reliable UDP" protocols, and we'll be looking at those in the rest of this article. One thing worth noting at the outset, though, is that if you want to optimize links in your workflow, you can either do it the little-bit-hard way and pay very little, or you can do it the easy way and pay a considerable amount to have a solution fitted for you.

Reliable UDP transports can offer the ideal situation for enterprise workflows -- one that has the benefit of high-capacity throughput, minimal overhead, and the highest possible "goodput" (a rarely used but useful term that refers to the part of the throughput that you can actually use for your application's data, excluding other overheads such as signaling). In the Internet Engineering Task Force (IETF) world, from which the IP standards arise, for nearly 30 years there has been considerable work in developing reliable data transfer protocols. RFC-908, dating from way back in 1984, is a good example.

Essentially, RDP (Reliable Data Protocol) was proposed as a transport layer protocol; it was positioned in the stack as a peer to UDP and TCP. It was proposed as an RFC (Request For Comment) but did not mature in its own right to become a standard. Indeed, RDP appears to have been eclipsed in the late 1990s by the Reliable UDP Protocol (RUDP), and both Cisco and Microsoft have released RUDP versions of their own within their stacks for specific tasks. Probably because of the "task-specific" nature of RUDP implementations, though, RUDP hasn't become a formal standard, never progressing beyond "draft" status.

One way to think about how RUDP types of transport work is to use a basic model where all the data is sent in UDP format, and each missing packet is indexed. Once the main body of the transfer is done, the recipient sends the sender the index list and the sender resends only those packets on the list. As you can see, because it avoids the retransmission of any windows of data that have already been sent that immediately follow a missed packet, this simple model is much more efficient. However, it couldn't work for live data, and even for archives a protocol must be agreed upon for sending the index. It responds to that rerequest in a structured way (which could result in a lot of random seek disc access, for example, if it was badly done).

There are many reasons the major vendor implementations are task-specific. For example, where one may use UDP to avoid TCP retransmission after errors, if the entire data must be faultlessly delivered to the application, one needs to actually understand the application.

If the application requires control data to be sent, it is important for the application to have all the data required to make that decision at any point. If the RUDP system (for example) only looked for and re-requested all the missing packets every 5 minutes (!) then the logical operations that lacked the data could be held up waiting for that re-request to complete. This could break the key function of the application if the control decision needed to be made sooner than within 5 minutes.

On the other hand, if the data is a large archive of videos being sent overnight for precaching at CDN edges, then it may be that the retransmission requests could be managed during the morning. So the retransmission could be delayed until the entire archive has been sent, following up with just the missing packets on a few iterations until all the data is delivered. So the flow, in this case, has to have some user-determined and application-specific control.

TCP is easy because it works in all cases, but it is less efficient because of that. On the other hand, UDP either needs its applications to be resilient to loss or the application developer needs to write in a system for ensuring that missing/corrupted packets are retransmitted. And such systems are in effect proprietary RUDP protocols.

There is an abundance of these, both free and open source, and I am going to look at several of each option (Table 1). Most of you who use existing streaming servers will be tied to the streaming protocols that your chosen vendor offers in its application. However, for those of you developing your own streaming applications, or bespoke aspects of workflows yourselves, this list should be a good start to some of the protocols you could consider. It will also be useful for those of you who are currently using FTP for nonlinear workflows, since the swap out is likely to be relatively straightforward given than most nonlinear systems do not have the same stage-to-stage interdependence that linear or live streaming infrastructures do.

Let's zip (and I do mean zip) through this list. Note that it is not meant to be a comprehensive selection but purely a sampler.

The first ones to explore in my mind are UDP-Lite and Datagram Congestion Control Protocol. These two have essentially become IETF standards, which means that inter-vendor operation is possible (so you won't get locked into a particular vendor).

Table 1: A Selection of Reliable UDP Transport

DCCP
Let's look at DCCP first. DCCP provides initial code implementations for those inclined. From the point of view of a broadcast video engineer, this is really deeply technical stuff for low-level software coders. However, if you happen to be (or simply have access to) engineers of this skill level then DCCP is freely available.

DCCP is a protocol worth considering if you are using shared network infrastructure (as opposed to private or leased line connectivity) and want to ensure you get as much throughput as UDP can enable, while also ensuring that you "play fair" with other users. It is worth commenting that "just turning on UDP" and filling the wire up with UDP data with no consideration of any other user on the wire can saturate the link and effectively make it unusable for others. This is congestion, but DCCP manages to fill the pipe as much as possible, while still inherently enabling other users to use the wire too.

Some of the key DCCP features include the following:

• Adding a reliability layer to UDP
• Discovery of the right MTU size is part of the protocol design (so you fill the pipe while avoiding fragmentation)
• Congestion control

Indeed, to quote the RFC: "DCCP is intended for applications such as streaming media that can benefit from control over the tradeoffs between delay and reliable in-order delivery."


UDP-Lite
The next of these protocols is UDP-Lite. Also an IETF standard, this nearly-identical-to-UDP protocol differs in one key way: It has a checksum (a number that is the result of a logical operation performed on all the data, which if it differs after a transfer indicates that the data is corrupt) and a checksum coverage range that that checksum applies to, whereas vanilla UDP -- optionally in IPv4, and always in IPv6 -- has just a simple checksum on the whole datagram and if present the checksum covers the entire payload.

Let's simplify that a little: What this means is that in UDP-Lite you can define part of the UDP datagram as something that must arrive with "integrity," i.e., a part that must be error-free. But another part of the datagram, for example the much bigger payload of video data itself, can contain errors (remain unchecked against a checksum) since it could be assumed that the application (for example, the H.264 codec) has error handling or tolerance in it.

This UDP-Lite method is very pragmatic. In a noisy network link, the video data may be subject to errors but could be the larger part of the payload, where the important sequence number may only be a smaller part of the data (statistically less prone to errors). If it fails, the application can use UDP-Lite to request a resend of that packet. Note that it is up to the application to request the resend; the UDP-Lite protocol simply flags the failure up and the software can prioritize a resend request, or it can simply plan to work around a "discard" of the failed data. It is also worth noting that most underlying link layer protocols such as Ethernet or similar MAC-based systems may discard damaged frames of data anyway unless something interfaces with those link layer devices. So to work reliably, UDP-Lite needs to interface with the network drivers to "override" these frame discards. This adds complexity to the deployment strategy and certainly most likely takes the opportunity away from being "free." However, it's fundamentally possible.

So I wanted to see what was available "ready to use" for free, or close to free at least. I went looking for a compiled, user-friendly, simple-to-use application with a user-friendly GUI, thinking of the videographers having to learn all this code and deep packet stuff just to upload a video to the office.


UDPXfer
While it's not really a protocol per se, I found UDPXfer, a really simple application with just a UDP "send" and "listener" mode for file transfer.

I set up the software on my laptop and a machine in Amazon EC2, fiddled with the firewall, and sent a file. I got very excited about the prompt 5MB UDP file transfer taking 2 minutes and 27 seconds, and I then set up an FTP of the same file over the same link but was disappointed that the FTP took 1 minute and 50 seconds -- considerably faster. When I looked deeper, however, the UDPXfer sender had a "packets per second" slider. I then nudged the slider to its highest setting, but it was still only essentially 100Kbps maximum, far slower than the effective TCP. So I wrote to the developer, Richard Stanway, about this ceiling. He sent a new version that allowed me to set a 1300 packets-per-second transmission. He commented that it would saturate the IP link from me to the server, and in a shared network environment a better approach would be to the tune the TCP window's size to implement some congestion control. His software was actually geared to resiliency over noisy network links that cause problems for TCP.

Given that I see this technology being used on private wires, the effective saturation that Stanway was concerned about was less of a concern for my enterprise video workflow tests, so I decided to give the new version a try. As expected, I managed to bring the transfer time down to 1 minute and 7 seconds. So while the software I was using is not on general release, it is clearly possible to implement simple software-only UDP transfer applications that can balance reliability with speed to find a maximum goodput.


Commercial Solutions
But what of the commercial vendors? Do they differentiate significantly enough from "free" to cause me to reach into my pocket?

I caught up with Aspera, Inc. and Motama GmbH, and I also reached out to ZiXi. All of this software is complex to procure at the best of times, so sadly I haven't had a chance to play practically with these. Also, the vendors do not publish rate cards, so it's difficult to comment on their pricing and value proposition.

Aspera co-presented at a recent Amazon conference with my company, and we had an opportunity to dig into its technology model a bit. Aspera is indeed essentially providing variations on the RUDP theme. It provides protocols and applications that sit on top of those protocols to enable fast file distribution over controlled network links. In Aspera's case, it was selling in behind Amazon Web Services Direct Connect to offer optimal upload speeds. It has a range of similar arrangements in place targeting enterprises that handle high volumes of latency-sensitive data. You can license the software or, through the Amazon model, pay for the service by the hour as a premium AWS service. This is a nice flexible option for occasional users.

I had a very interesting chat with the CEO of Motama, which has a very appliance-based approach to its products. The RUDP-like protocol (called RelayCaster Streaming Protocol or RCSP) is used internally by the company's appliances to move live video from the TVCaster origination appliances to RelayCaster devices. These then can be hierarchically set up in a traditional hub and spoke or potentially other more complex topologies. The software is available (under license) to run on server platforms of your choice, which is good for data center models. They have also recently started to look at licensing the protocol to a wider range of client devices, and they pride themselves in being available for set-top boxes.

The last player in the sector I wanted to note was ZiXi. While I briefly spoke with ZiXi representatives while writing this, I didn't manage to communicate properly before my deadline, so here is what I know from the company's literature and a few customer comments: ZiXi offers a platform that optimizes video transfer for OTT, internet, and mobile applications. The platform obviously offers a richer range of features than just UDP-optimized streaming, and it has P2P negotiation and transmuxing so you can flip your video from standards such as RTMP out to MPEG-TS, as you can with servers such as Wowza. Internally, within its own ecosystem, the company uses its own hybrid ZiXi protocol, including features such as forward error correction, combining applications layer software in a product called Broadcaster that looks like a server with several common muxes (RTMP, HLS, etc.) and includes ZiXi. If you have an encoder with ZiXi running, then you can contribute directly to the server using the company's RUDP-type transport.


Worth the Cost?
I am aware none of these companies licenses their software trivially. The software packages are their core intellectual properties, and defending them is vital to the companies' success. I also realize that some of the problems that they purport to address may "go away" when you deploy their technology, but in all honesty, that may be a little like replacing the engine of your car because a spark plug is misfiring.

I am left wondering where the customer can find the balance between the productivity gains in accelerating his or her workflow with these techniques (free or commercial) against the cost of a private connection plus either the cost of development time to implement one of the open/free standards or the cost of buying a supported solution.

The pricing indication I have from a few undisclosed sources is that you need to be expecting to spend a few thousand on the commercial vendor's licensing, and then more for applications, appliances, and support. This can quickly rise to a significant number.

This increased cost to improve the productivity of your workflow must be at some considerable scale, since I personally think that a little TCP window sizing, and perhaps paying for slightly "fatter" internet access, may resolve most problems -- particularly in archive transfer and so on -- and is unlikely to cost thousands.

However, at scale, where those optimizations start to make a significant productivity difference, it clearly makes a lot of sense to engage with a commercially supported provider to see if its offering can help.

At the end of the day, regardless of the fact that with a good developer you can do most things for free, there are important drivers in large businesses that will force an operator to choose to pay for a supported, tested, and robust option. For many of the same reasons, Red Hat Linux was a premium product, despite Linux itself being free.

I urge you to explore this space. To misquote James Brown: "Get on the goodput!"

By Dom Robinson, streamingMedia

Comparing JPEG 2000 and MPEG

Back in 1988, the first work began on video compression standards that would ultimately result in MPEG. The Motion Picture Experts Group, in cooperation with the International Organization for Standardization (ISO), created multiple standards for video compression, now known as MPEG-1, MPEG-2 and MPEG-4/AVC. MPEG was devised as a means of compressing broadband video into a small bit stream that could fit in extremely narrow broadcast or satellite transmission channels. As such, it is well-suited for distribution purposes, not for applications that require internal switching and processing.

MPEG
Because MPEG is a motion image compression technology, it works on a sequence of video frames, known as a Group of Pictures (GOP). A processor examines several frames of video and assigns one frame as the reference frame for that group (the I-frame). The GOP also has several predictive frames, or P frames, which use information from the I frame and previous P frames to construct images. Finally, there are bidirectional predictive frames, or B frames, which look at preceding and following I and P frames. Motion is analyzed and the motion vectors, which predict the offset from the current frame to the reference frame, are estimated down to a quarter pixel. The motion vectors and the difference between the actual images are used to reconstruct video that looks good at low bit rates.

  MPEG GOP structure

Problems with MPEG encoding arise when there is a scene change or where a large percentage of the image changes at once. The processor can’t predict the movement, and the system creates a distortion known as macroblocking. Raising the bit rate adds more detail in each 8 x 8 block of pixels, which can reduce the number of P frames. But there will always be P frames — and the potential for macroblocking — in all implementations of MPEG.




  Macroblocking effects

Another major failing of MPEG compression is that it is difficult to edit and switch cleanly. With P and B frames, editing systems and switchers have trouble finding a clean frame on which to edit. MPEG also doesn’t react well to being repeatedly encoded and decoded. It works best when the video is encoded once and decoded at the viewing location.


JPEG 2000
Meanwhile, JPEG — short for Joint Photographic Experts Group — was originally developed as a compression standard for still images. With modern processors, however, it is now possible to take a video signal and save each frame as a separate JPEG file, creating a video format known as Motion-JPEG.

Several years after the JPEG team created its first compression scheme, it reconvened to develop an even more efficient and flexible system. Released in 2000, its new JPEG compression was called JPEG 2000. The original JPEG used discrete cosine transform (DCT), the same techniques used in MPEG, to reduce data into blocks of 8 x 8 pixels. JPEG 2000 uses discrete wavelet transform (DWT), which performs simultaneous multiresolution image analysis.

The resulting wavelet coefficients are gathered in sub-bands, each of which is partitioned into small “code blocks” and independently coded by an adaptive binary arithmetic encoder. Finally, the output of the arithmetic encoder is organized as a compressed bit stream that offers a significant degree of flexibility.

Wavelet compression provides the same processing for all pixels, and with each video frame being compressed individually, there is no error propagation from one picture to the next. As a result, there is no macroblocking at low bit rates; instead, there is blurring on the images. JPEG works on a single image, stripping away redundant data and encoding each video frame independently for consistent high-quality images. As a result, editing may be performed on any frame in a sequence of images because each video frame contains its own picture information.

With no dependency on other images, JPEG 2000 has low latency, less than 1.5 frames encode and/or decode, and maintains sync between the video and audio. In contrast, MPEG and other compression schemes that rely on predictive frames and motion estimation algorithms have high latency.

JPEG 2000 and its underlying wavelet compression can allocate 10 bits or even 12 bits at 4:4:4 quality, a level in line with the demands of digital cinema post production. In fact, there are several systems that use wavelet compression techniques similar to JPEG 2000. REDCODE and CineForm, for example, are used successfully for digital cinema production, which demonstrates the quality and robustness of wavelet compression.


Contribution
There are significant differences between MPEG-4 and JPEG 2000 for broadcast contribution. Further development of video and still image compression standards continues, and there will no doubt be future advancements for both standards. For example, High Efficiency Video Coding (HEVC), unofficially called H.265, promises improved video quality and data compression, as well as support for higher resolutions.




  JPEG 2000 and MPEG-4/AVC attributes for broadcast contribution

Broadcast contribution links seek to have the highest video quality because the signal may have to go through post-production editing. The output video quality from the contribution stage is propagated to the distribution link. As a consequence, the quality and efficiency from the distribution link is highly dependent upon the level of quality experienced during contribution. Broadcast contribution signals should be lightly compressed — almost equal to the original uncompressed signal — so there is minimal quality loss under multiple generations of encoding. JPEG 2000 can compress HD video to a significantly lower bit rate, and then extract the video with no loss of information. When Peak Signal to Noise Ratio (PSNR) performance is used to compare original footage with video compressed with JPEG 2000, the degradation is close to visually lossless and deviation from the original is low. The PSNR of footage encoded with MPEG, however, can vary greatly from picture to picture due to different methods of prediction.


Performance Comparison
Recently, the performance of JPEG 2000 and MPEG-4/AVC under multigeneration encoding was evaluated using a single vendor’s encoders and decoders. Video quality was measured in terms of PSNR for different HD interlaced sequences at the operating points of the encoders. The streams were encoded at 90Mb/s for JPEG 2000 and 45Mb/s for MPEG-4/AVC streams. These bit rates were used throughout the testing because they are common operating points.




  PSNR simulation setup

MPEG-4/AVC provided high-quality video at 45Mb/s. Under multigeneration encoding and decoding, the PSNR reduction was sharp — roughly 2.2dB after the fourth generation. JPEG 2000 also delivered high-quality video (at 90Mb/s), but it had less than 0.5dB loss after the fourth generation of encoding and decoding, which was less than one quarter of the reduction experienced by MPEG-4. In addition, the predictive coding of MPEG resulted in an end-to-end latency of approximately 1 second, which is roughly 10 times more than the latency experienced with JPEG 2000.




  Multigeneration encoding

For another round of tests, designed to evaluate the influence of contribution encoding on end-user quality, a distribution encoder was used to encode the incoming signal using 4:2:0 8-bit at 6Mb/s, 8Mb/s and 10Mb/s (common settings for IPTV applications). The tests showed that because of JPEG 2000’s high video quality, it is possible to increase the compression ratio for the final compression to broadcast without quality loss, saving from 5 percent to 15 percent of the total bit rate, depending on the final bit rate used for distribution. By spending more money on contribution, it is possible to save on broadcasting to the home, because more content can be placed in the available bandwidth.

To illustrate the bandwidth savings that can be achieved when employing JPEG 2000 video compression, consider an uncompressed HD-SDI signal, which has a bit rate of about 1.5Gb/s. A JPEG 2000 encoder/decoder can maintain the quality of the original signal with a bit stream of 500Mb/s to 800Mb/s, depending on the content. At 600Mb/s, virtually any type of high-quality HD content will be transported mathematically losslessly with a JPEG 2000 system. Reducing the bit rate from there will increase the possibility of mathematical image loss, but any errors will be visually imperceptible even at far lower bit rates.




  Bandwidth savings

Stereo 3-D
Bandwidth-intensive formats like 3-D TV raise the bar even higher for quality and precision. The creation of 3-D video and particularly the need for the best possible picture quality has ramifications for how content is captured, produced and broadcast. In the contribution phase, left- and right-eye information must be transported along with other metadata required to generate the highest quality 3-D images and must be compatible with all 3-D compression and display systems. To ensure a high-quality viewing experience, it is critical that all the data required to display the left- and right-eye images correctly is conveyed throughout the transport chain with minimal visual impairment.

JPEG 2000 ensures that the horizontal resolution, critical to the reconstruction of 3-D images in our brains, is preserved. A JPEG 2000 encoder can transport two 2K or HD-SDI channels, or the two synchronous left-eye/right-eye video streams that comprise stereo 3-D. Even after the JPEG 2000 video is compressed or decompressed, and throughout real-time transport over Metro Ethernet (metropolitan-area Ethernet), IP-MPLS (MultiProtocol Label Switching) or SDH/SONET (Synchronous Digital Hierarchy or Synchronous Optical NETwork), the video quality stays true to the original imagery.


Applications
JPEG 2000 video compression is ideal for venues where high-quality programming regularly originates, such as stadiums, arenas and remote studios. While prices vary between service providers, commonly available IP links that provide high bit rates are universally less expensive than legacy systems, which translates to significant long-term operational cost savings. Before connecting an encoder, however, it is critical that the service provider understands the Quality of Service (QoS) requirements of high-quality video and can ensure sufficient bandwidth.

JPEG 2000 requires a high-bit-rate IP network and usually works best with both a primary and redundant link. If you need to do a remote shoot from a non-fixed location, i.e., in front of a burning house, then you probably need to use a microwave or satellite truck with MPEG compression.

Final signal distribution to viewers is a perfect application for MPEG. JPEG 2000 excels as a contribution technology, but is not intended as a distribution technology. That said, stadiums, city halls, remote studios, OB trucks and fixed locations where you need the highest quality video are good choices to consider using JPEG 2000 to maintain optimal video quality. As the cost of dedicated IP networks comes down and bit rates increase, JPEG 2000 video will help control costs when compared to traditional high-end video transport systems. Careful consideration of your outside broadcast requirements, access to high-speed IP networks and JPEG 2000 capabilities provide another choice in your video transport toolkit.

By Helge Stephansen, Broadcast Engineering

MPEG DASH Use is Blossoming, Says Akamai

Click to watch the video
Source: Beet.tv

A Guide to Closed Captioning for Web, Mobile, and Connected TV

Captioning is coming to Internet video. Legislation goes into effect in the US during 2012 and 2013 that mandates closed captioning on certain categories of online content – see Zencoder's post for details on the legislation. But even apart from this legislation, closed captioning is a good thing for accessibility and usability, and is yet another milestone as Internet video marches towards maturity.

Unfortunately, closed captioning is not a single technology or “feature” of video that can be “turned on”. There are a number of formats, standards, and approaches, ranging from good to bad to ugly. Closed captioning is kind of a mess, just like the rest of digital video, and is especially challenging for multiscreen publishers.

How Closed Captions Work
The first thing to understand is how closed captions are delivered, stored, and read. There are two main approaches today:

• Embedded within a video: CEA-608, CEA-708, DVB-T, DVB-S, WST. These caption formats are written directly in a video file, either as a data track or embedded into the video stream itself. Broadcast television uses this approach, as does iOS.


• Stored as a separate file: DFXP, SAMI, SMPTE-TT, TTML, EBU-TT (XML), WebVTT, SRT (text), SCC, EBU-STL (binary). These formats pass caption information to a player alongside of a video, rather than being embedded in the video itself. This approach is usually used by browser-based video playback (Flash, HTML5).

What about subtitles? Are they the same thing as closed captions? It turns out that there are three main differences:

• Goals: Closed captions are an accessibility feature, making video available to the hard of hearing, and may include cues about who is speaking or about what sounds are happening: e.g. “There is a knock at the door”. Subtitles are an internationalization feature, making video available to people who don’t understand the spoken language. In other words, you would use captions to watch a video on mute, and you would use subtitles to watch a video in a language that you don’t understand. (Note that this terminological distinction holds in North America, but much of the world does not distinguish between closed captions and subtitles.)


• Storage: Historically, captions have been embedded within video, and subtitles have been stored externally. This makes sense conceptually, because captions should always be provided along with a video; 100% accessibility for hard-of-hearing is mandated by legislation. Whereas subtitles are only sometimes needed; a German-language video broadcast in Germany doesn’t need to include German subtitles, but that same video broadcast in France would.


• Playback: Since captions are passed along with the video and interpreted/displayed by a TV or other consumer device, viewers can turn them on and off themselves at any time using the TV itself, but rarely have options for selecting a language. In these situations when subtitles are added for translation purposes, they are generally hard subtitles and thus cannot be disabled. However, when viewing DVD/Blu-Ray/VOD video, the playback device controls whether subtitles are displayed, and in which language.

Formats and Standards
There are dozens of formats and standards for closed captioning and subtitles. Here is a rundown of the most important ones for Internet video:

• CEA-608 (also called Line 21) captions are the NTSC standard, used by analog television in the United States and Canada. Line 21 captions are encoded directly into a hidden area of the video stream by broadcast playout devices. If you’ve ever seen white bars and dots at the top of a program, that’s Line 21 captioning (more information.)


• An SCC file contains captions in Scenarist Closed Caption format. The file contains SMTPE timecodes with the corresponding encoded caption data as a representation of CEA-608 data.


• CEA-708 is the standard for closed captioning for ATSC digital television (DTV) streams in the United States and Canada. There is currently no standard file format for storing CEA-708 captions apart from a video stream.


• TTML stands for Timed Text Markup Language. TTML describes the synchronization of text and other media such as audio or video. See the W3C TTML Recommendation for more.
  
  Example:

  <tt xml:lang="" xmlns="http://www.w3.org/ns/ttml">
    <head>
      <styling xmlns:tts="http://www.w3.org/ns/ttml#styling">
        <style xml:id="s1" tts:color="white" />
      </styling>
    </head>
    <body>
      <div>
        <p xml:id="subtitle1" begin="0.76s" end="3.45s">
          Trololololo
        </p>
        <p xml:id="subtitle2" begin="5.0s" end="10.0s">
          lalala
        </p>
        <p xml:id="subtitle3" begin="10.0s" end="16.0s">
          Oh-hahaha-ho
        </p>
      </div>
    </body>
  </tt>



• DFXP is a profile of TTML defined by W3C. DFXP files contain TTML that defines when and how to display caption data. DFXP stands for Distribution Format Exchange Profile. DFXP and TTML are often used synonymously.


• SMPTE-TT (Society of Motion Picture and Television Engineers – Timed Text) is an extension of the DFXP profile that adds support for three extensions found in other captioning formats and informational items but not found in DFXP: #data, #image, and #information. See the SMPTE-TT standard for more.
  
  SMPTE-TT is also the FCC Safe Harbor format – if a video content producer provides captions in this format to a distributor, they have satisfied their obligation to provide captions in an accessible format. However, video content producers and distributors are free to agree upon a different format.


• SAMI (Synchronized Accessible Media Interchange) is based on HTML and was developed by Microsoft for products such as Microsoft Encarta Encyclopedia and Windows Media Player. SAMI is supported by a number of desktop video players.


• EBU-STL is a binary format used by the EBU standard, stored in separate .STL files. See the EBU-STL specification for more.


• EBU-TT is a newer format supported by the EBU, based on TTML. EBU-TT is a strict subset of TTML, which means that EBU-TT documents are valid TTML documents, but some TTML documents are not valid EBU-TT documents because they include features not supported by EBU-TT. See the EBU-TT specification for more.


• SRT is a format created by SubRip, a Windows-based open source tool for extracting captions or subtitles from a video. SRT is widely supported by desktop video players.


• WebVTT is a text format that is similar to SRT. The Web Hypertext Application Technology Working Group (WHATWG) has proposed WebVTT as the standard for HTML5 video closed captioning.
  
  Example:
  
  

  WEBVTT

  00:00.76 --> 00:03.45
<v Eduard Khil>Trololololo

  00:5.000 --> 00:10.000
lalala


  00:10.000 --> 00:16.000
Oh-hahaha-ho

  
  
• Hard subtitles (hardsubs) are, by definition, not closed captioning. Hard subtitles are overlaid text that is encoded into the video itself, so that they cannot be turned on or off, unlike closed captions or soft subtitles. Whenever possible, soft subtitles or closed captions are generally be preferred, but hard subtitles can be useful when targeting a device or player that does not support closed captioning.

Captioning for Every Device
What formats get used by what devices and players?:

• Flash video players can be written to parse external caption files. For example, JW Player supports captions in SRT and DFXP format.


• HTML5 captions are not yet widely supported by browsers, but that will change over time. There are two competing standards: TTML, proposed by W3C, and WebVTT, proposed by WHATWG. At the moment, Chrome has limited support for WebVTT; Safari, Firefox, and Opera are all working on WebVTT support; and Internet Explorer 10 supports both WebVTT and TTML.
  
  Example:
  

  <video width="1280" height="720" controls>
    <source src="video.mp4" type="video/mp4" />
    <source src="video.webm" type="video/webm" />
    <track src="captions.vtt" kind="captions" srclang="en" label="English" />
  </video>

  
  Until browsers support a format natively, an HTML5 player framework like Video.js can support captions through Javascript, by parsing an external file. (Video.js currently supports WebVTT captions.)


• iOS takes a different approach, and uses CEA-608 captions using a modified version of CEA-708/ATSC legacy encoding. This means that, unlike Flash and HTML5, captions must be added at the time of transcoding. Zencoder can add captions to HTTP Live Streaming videos for iOS.


• Android video player support is still fragmented and problematic. Caption support will obviously depend on the OS version and the player used. Flash playback on Android should support TTML, though very little information is available.


• Some other mobile devices have no support for closed captions at all, and hard subtitles may be the only option.


• Roku supports captions through external SRT files.


• Some other connected TV platforms do not support closed captioning yet. But they will soon enough. Every TV, console, cable box, and Blu-Ray player on the market today wants to stream Internet content, and over the next year and a half, closed captioning will become a requirement. So Sony, Samsung, Vizio, Google TV, et al will eventually make caption support a part of their application development frameworks. Unfortunately, it isn’t yet clear what formats will be used. Most likely, different platforms will continue to support a variety of incompatible formats for many years to come.

Closed Captioning for Internet Video: 2012 Edition
The landscape for closed captioning will change and mature over time, but as of 2012, here are the most common requirements for supporting closed captioning on common devices:

• A web player (Flash, HTML5, or both) with player-side controls for enabling and disabling closed captioning.


• An external file with caption data, probably using a format like WebVTT, TTML, or SRT. More than one file may be required – e.g. SRT for Roku and WebVTT for HTML5.


• A transcoder that supports embedded closed captions for HTTP Live Streaming for iPad/iPhone delivery, like Zencoder. Zencoder can accept caption information in a variety of formats, including TTML, so publishers could use a single TTML file for both web playback and as input to Zencoder for iOS video.

Beyond there, things get difficult. Other input formats may be required for other devices, and hard subtitles are probably necessary for 100% compatibility across legacy devices.

Source: Zencoder

Internet TV Systems and Coding

Today's TV viewers want more content from an increasing number of sources, and that means that Internet delivery is a growing phenomenon. With hybrid technologies emerging, it is reasonable to expect that television broadcast will increasingly use the Internet to expand throughput beyond that afforded by a single RF channel. But there are limitations to the Internet that must be understood in order to capitalize on this commodity, and some of those constraints are being overcome by new technologies.

Streaming Can Now Provide a High Quality of Service
In general, Internet TV is a means to provide streamed video content to a PC, STB or Internet-connected TV, by means of an Internet connection. Internet Protocol Television, or IPTV, refers to a special case where a full-time TV subscriber connection is established by means of a dedicated line (and channel) to the telephone system central office. It is envisioned, however, that many Internet TV viewers will get their content though their Internet connection, and as such, receive OTT video service that shares bandwidth with other Internet traffic.

This sharing of bandwidth creates a QoS challenge for Internet TV service: While a terrestrial channel has a fixed bandwidth (i.e., 19.2Mb/s in the U.S.), an Internet TV service must share the bandwidth, both locally (e.g., within a viewer's household) as well as regionally (e.g., with other subscribers). This means the bandwidth available to a receiver can vary continuously over a wide range, and different subscribers may have different levels of guaranteed service, as well. Lowering the video bit rate to the least common denominator would result in poor video quality to everyone; to deal with this, several technologies are available.

Progressive Download vs. Streaming
The simplest way to deliver video over the Internet is to use progressive download, sometimes called “HTTP streaming.” This is simply a bulk download of a video file to the viewer's terminal (i.e., Internet-connected TV, STB, PC, etc.). A temporary copy of the file is stored on the user's device, typically on a hard drive, and playback can start after a sufficient amount of the file has been downloaded. This means that content will always incur a considerable delay before it is available to be viewed, which makes a live service rather difficult to implement. However, because the files are downloaded using TCP, there can be a nearly 100 percent assurance that every single bit was transferred correctly.

True streaming, on the other hand, opens up a handshaking connection between the server and client using a set of Internet protocols to deliver streams, such as Real Time Streaming Protocol (RTSP), Real Time Messaging Protocol (RTMP) and Microsoft Media Services (MMS). A streaming connection delivers a video stream with minimal buffering, allowing a nearly real-time presentation of the source content. In this respect, streaming has an advantage over progressive download, as continuous delivery is the goal, but the associated downside is that corrupted or missing packets are not detected. The consequence is that audio and video can have ongoing glitches when network congestion is experienced.

Adaptive Bit Rate Streaming
To solve the QoS issue, Adaptive Bit Rate (ABR) streaming has been developed. ABR allows each device to determine the quality of its connection and then use that metric to select the best-coded stream from a number of different quality streams. At the server end, a series of encoders encode a set of multiple streams at different bit rates, and these streams are then sliced up into segments or “chunks.” An ABR client in the viewer terminal detects the incoming stream bandwidth on the fly and uses this, along with a model based on the device's CPU capability, to select a segment among the various streams.

A special manifest file precedes the first segment, providing the client with a list of URLs from which each segment can be accessed. As each segment is received, the client progresses to the next segment in that stream, or it can jump to a parallel segment in one of the other streams if the channel bandwidth changes because of congestion, etc. In principle, a handful of streams will provide enough granularity so that the viewer does not detect a change in picture quality.

Note that ABR provides high transmission bandwidth efficiency when a unicast transmission (i.e., one-to-one) is used, but it can also work well with multicast and broadcast scenarios depending on how well the Internet infrastructure distributes bandwidth to users. ABR has the potential to deliver an audio/visual experience that we have come to expect from linear transmission: low delay, fast start time and a consistent experience across viewers.

Several manufacturers have developed different solutions for ABR streaming. Adobe HTTP Dynamic Streaming (HDS) uses a format called F4F to deliver Flash videos over RTMP and HTTP. Apple HTTP adaptive Live Streaming (HLS) was developed for the iPhone and iPad, and is implemented using HTTP, H.264 and MPEG-2 Transport Streams, with a manifest file called M3U8. Microsoft Internet Information Services (IIS) Smooth Streaming is used within Silverlight on the Windows 7 phone and incorporates fragmented MP4 (fMP4) encapsulation, again with H.264 for video compression.

With these different enterprise systems, an interoperability problem exists because of proprietary protocols and manifest structures. Multiple ABR systems mean that different devices must either pick and choose which systems to support, leading to service-constrained devices, or must include all at increased cost. This situation has motivated companies and experts around the world to propose a single, standard ABR system.

DASH-ing to the Rescue: a Universal ABR System
MPEG-DASH (Dynamic Adaptive Streaming over HTTP) is a newly standardized method for defining Stream Segments and Manifest Files for the purpose of ABR streaming. The specification (ISO-IEC 23009-1) defines a Media Presentation Description (MPD) that formalizes the stream manifest, which includes Segment timing, URLs and media characteristics such as video resolution and bit rates. While compatible Segments can contain any media data — with arbitrary compression — two types of containers are exemplified in the standard: MPEG-4 file format and MPEG-2 Transport Stream.

MPEG-DASH defines a standard set of Media Presentation Description and Segment Formats
that enable adaptive bit rate IP video streaming.

In going to a standard system, MPEG-DASH is quickly deployable with the existing Internet infrastructure, using widely deployed standard HTTP servers/caches for scalable delivery. Generic encoders can be reused, with additional descriptive metadata for better client functionality, and legacy manifest files can be converted easily to MPD format, as well as sent in parallel for backward compatibility with low overhead. In addition, existing content and production equipment supporting legacy ABR streaming systems can be used for MPEG-DASH by means of a set of standard Profiles. Apple HLS content can be used with the DASH M2TS Main profile, and Microsoft IIS Smooth Streaming Content is suitable for DASH ISO-BMFF (Base Media File Format) Live profile.

Vendors are now proposing integrated workflow and delivery systems supporting ABR with multiple source formats, protocols and on multiple devices. While encoding latency can be an issue for live streams, MPEG-DASH includes a profile optimized for live encoding that can achieve a latency of a few seconds by encoding and immediate delivery of short Segments.

In addition to delivery of any multimedia content, MPEG-DASH supports a broad range of use cases, including live, VOD, time shifting (nPVR), ad insertion and dynamic update of program. MPEG-DASH also solves the problem of content repurposing to multiple devices with widely ranging capabilities. In principle, an MPEG-DASH-controlled stream can be targeted simultaneously to both large and small screens, as well as fixed and mobile.

Internet Quickly Becoming Viable for Long-Form Content
The once-exclusive realm of RF transmission as providing the highest quality content consumption experience is being challenged by streaming services. But new technologies and business models are providing broadcasters with the tools to compete with new service entities, and that's where content distribution is headed.

By Aldo Cugnini, Broadcast Engineering

First Live MPEG-DASH Large Scale Demonstration

During the 2012 London Olympics, VRT has been offering its audience the chance to experience the Olympic Games broadcast on their personal devices via MPEG-DASH. The public trial allowed for a maximum of 1000 concurrent viewers to watch their favourite sport events on a laptop, smartphone or tablet.

The commercial deployment of the MPEG-DASH (Dynamic Adaptive Streaming over HTTP) standard is one step closer with the first live public trial, presented by Belgian public broadcaster VRT. This trial has been supported by a number of DASH Promoters Group members: encoding has been provided by Elemental, Harmonic and Media Excel; streaming origins was courtesy of Wowza and CodeShop, who also provided encryption; web clients for PC and Android have been supplied by Adobe; and BuyDRM provided applications for iOS and Android, which incorporate its DRM solution.

This proof of concept was initiated by the European Broadcasting Union, which strongly supports the development of MPEG-DASH as it is a key enabler allowing broadcasters to use a single file and streaming format to deliver content to multiple devices on multiple platforms.

Supported Devices
VRT offered users the following choices for viewing the London Games:

• PCs and MAC running Adobe Flash.


• Web browser for Android provided by Adobe.


• iPhone / iPads from iOS version 4.3, with a special app from the iTunes store. The app is currently pending Apple approval.


• Android smartphones from version 4.0, via Sporza Olympic Games available from the Google Play store.

Used MPEG-DASH Profile
The demonstration featured a live video stream encoded using the MPEG-DASH ISO Base Media File Format Live Profile, delivered through Belgacom’s Content Delivery Network to a range of device categories including tablets, smartphones and PCs running iOS, Android and Windows operating systems. This represents the first large-scale multivendor deployment of MPEG-DASH.

The demonstration was based on an early version of the DASH-264 interoperability guidelines, specifically developed by the DASH Promoters Group for interoperable deployment of the MPEG-DASH standard. DASH-264 provides a general interoperability framework aligned with the HbbTV 1.5 specification and other consortia recommendations. HbbTV 1.5 will be widely used by European broadcasters for interactive services on connected televisions.

DASH details:

• File Format: ISO BM FF
• Profile: Live
• Template: Time based
• Codecs: H.264 Baseline (ABR) / Audio AAC-LC (SBR)

Encoding settings:

The video consists of 6 different streams that can be chosen by the player to adapt the video playout automatically to the available bandwidth. The highest quality is 1500 kbs for the video quality, and there is also an audio-only stream available when there is inadequate Internet speed for video.

The settings of the various adaptive switchable streams in this proof of concept have not been defined for optimal audiovisual quality. Simplicity was the key driver in selecting the Baseline profile for H.264 encoding and 64 kbs audio bitrate limitation. This will ensure that switching between the different streams will run smoothly.



Logic Flow
The logical flow of online distribution based on MPEG-DASH is very similar to what is currently deployed in adaptive streaming systems. One needs only an encoded/packaged stream and an HTTP server to get the job done and play out video to a player that supports MPEG-DASH natively.

The simple part of the workflow is demonstrated by the chain starting with the Elemental Live encoder, which captures the audiovisual content from the SDI feed at the VRT premises. The Apache server located at the CDN of Belgacom picks up the data packages via HTTP GET from the encoder and makes it available by a URL and an MPD file describing how the packages should be interpreted by the player. The Adobe player reads out the MPD, buffers the packages and plays out the video on a device of the end user.

It gets more complicated when one involves dedicated applications that play out video to devices that do not yet support MPEG DASH natively. Additionally, to secure premium content, content protection using DRM technology needs to be implemented in the server and the client. In this trial, a group of equipment suppliers are working together to make this happen.

The Harmonic ProMedia Live encoder encodes the IP feeds. The Wowza server, also located at the VRT premises, acts as an origin server, making the packages available for the Wowza cache server in the CDN. This content is then played out by the BuyDRM Android and iOS players.

In order to also showcase the Common Encryption support for MPEG-DASH with Microsoft’s PlayReady DRM, another distribution chain is set up in which the Media Excel HERO encoder works with CodeShops’s Unified Streaming server acting as an origin to produce live encrypted content. For proof of concept purposes, the BuyDRM Android and iOS applications in this distribution chain switch seamlessly between protected and unprotected content.


Click to enlarge
Source: DASH Promoters Group

New EBU Subtitling Specification Published

The EBU has published EBU-TT part 1, with TT standing for Timed Text. It's a follow-up to the widely used EBU STL specification, which was originally published in 1991. The new format is XML-based, which makes it “human readable” and more suited to modern integrated file-based production methods.

EBU-TT is a simplified version of the W3C Timed Text specification, which means it fits well into the broad family that includes W3C TTML and SMPTE TT, which is more focused on the US environment and on distribution. EBU-TT was developed by the EBU's XML Subtitles group, chaired by Andreas Tai of IRT.

Part 1 (Tech 3350) has been published, defining a structure for the interchange and archiving of subtitles; part 2, now being drafted, will provide mapping guidance for users who want to migrate from EBU STL to EBU-TT. Work has also started towards a specification for live subtitling, with a workshop scheduled for 9 August.

Source: EBU

What is Timed Text?

A nice video introduction to Timed Text by Bruce Devlin.

HD Forum Releases TNT 2.0 Specifications and Conformance Tests

HD Forum is aimed at helping and promoting the development of innovative technologies for the French Digital TV environment. Among various topics, HD Forum has worked during the last 2 years on the TNT2.0 project.

The goal was to coordinate the launch of HbbTV interactive services in France. This effort is done in partnership with the HbbTV consortium and led to an upgrade version of HbbTV standard, version 1.5 and a first recommendation for DRM's.

In this respect:

• The HD Forum board is making official the TNT 2.0 specifications to be applied on the French market;


• All together, manufacturers, broadcasters and distributors representatives committed around the technical choices made so far, expecting that products and services will ramp up from Q4’2012 to April 2013 and beyond. Of course, products and service launch plans details remain the respective choices of broadcasters, manufacturers and distributors;


• In order to support this goal, the “TNT 2 0 - Terminal Specification - V1.1 Final” including last improvement has been made available on the HD FORUM web site for a wider distribution to all interested parties, including non HD FORUM members;


• The HD FORUM is also pleased to announce the full release of the TNT 2.0 test suite, developed in cooperation with companies Farncombe and Digital TV labs in the UK. This test suite has already been licensed to multiple manufacturers, and the HD FORUM restates the requirement for consumer electronic manufacturers to self-validate compliance of their products, by passing the TNT 2.0 test suite. The test suite is licensed under a fair and non-discriminatory terms and conditions approved by the HD FORUM;


• In parallel, work continues on the interoperability of services;


• In due course, HD FORUM will contribute to prepare the launch of products and services complying with the TNT 2.0 specifications.

Source: HD FORUM

What is AS12 and Why Do You Need It?

A nice video introduction to AS12 by Bruce Devlin.

DVB Steering Board Approves Phase 2a of 3DTV Specification

DVB is pleased to announce that at the 71st Meeting of the Steering Board, Phase 2a of the DVB-3DTV specification was approved. The specification will be submitted immediately to the European Telecommunications Standards Institute (ETSI) for formal standardisation. An amended DVB-3DTV BlueBook is to be published to reflect the addition to the specification.

The Phase 2a system, also known as “Service Compatible Mode” is designed to meet the needs of those who need to provide normal HDTV receivers with a 2D version of the 3D programme from the same broadcast channel, and at the same time improve the quality of the 3DTV images. Phase 2a provides a 2D version plus an MPEG MVC top-up signal. Although they are tailored to their different environments, both 3D Blu-ray and Phase 2a use MVC, which will enable receivers to include both capabilities.

The 3DTV Phase 1 specification was published in 2011 for “Frame Compatible” delivery, where the essential requirement was that existing HDTV set-top boxes would allow viewers to watch 3D (if they had a 3D display). This is the most used 3DTV broadcast form in the world today.

Source: DVB

MXF for Program Contribution - AS-11

Ian Wimsett, Senior Technologist at Red Bee Media, describes the Advanced Media Workflow Association (AMWA) specification MXF for Contribution, AS-11. AS-11 is a vendor-neutral subset of the MXF file format to use for delivery of finished programming from program producers and distributors to broadcast stations. AS-11 Files are intended to be complete and ready for playout. AS-11 supports SD video encoded as D-10, 50Mbit/s, and HD as AVC-Intra Class 100. Audio can be PCM, AC-3 or Dolby E.

What is AS11 and Why Do You Need It?

A nice video introduction to AS11 by Bruce Devlin.

What is AS10 and Why Do You Need It?

A nice video introduction to AS10 by Bruce Devlin.

IT Acronyms Explained

As broadcast facilities and post-production houses implement new workflow and media management systems, they deal with an array of new IT-based technologies and are inundated with acronyms associated with those technologies, as well as the best practices concerning their use. Adoption of IT-based technologies continues apace with continued convergence across the media industry.

Despite this rapid shift of operations into the IT realm, however, the industry as a whole has not been as fast in providing engineering staff with an education on the meaning of terms such as XML, WSDL, SOAP, SOA and REST, the technologies underpinning them, and the role they play in present and future media-focused operations. This article will review the most relevant acronyms, their fundamental principles and their typical applications.


XML and XML Schema
Complex control systems depend on a reliable interchange of information in order to make the myriad business decisions that guide material through workflow. This is made simpler if individual systems can swap information in the form of structured data — that is, documents that contain both content and an indication of the content's role in the document.

The concept of structured data is not new; humans have been dealing with information formatted this way for centuries. Examples include published (human-readable) books and magazines, as well as web pages displayed by a browser. In both cases, these systems indicate to the consumer the relevance and importance of any particular item on the page through the use of typographical hints — such as underlining text that hyperlinks to other documents. HTML achieves this by including “tags” in the document, and the browser uses those tags to determine how to emphasize the relative value of the associated data.

As a markup language, HTML represents a fairly limited set of standardized tags intended for the presentation of web content, so its utility beyond that scope likewise is limited.

XML, however, offers flexibility that makes it more suitable for commercial use. This is true because, unlike HTML, XML allows users to define their own tags. Rather than defining the tags themselves (or the semantics of the tags), XML provides a facility for defining tags and using them within a document. As with HTML, data is then bracketed within opening and closing tags, which the receiving device can read and act on as appropriate.

Here is an example of the use of a tag “author” to identify a book's author in a library document:
<author>William Shakespeare</author>

Because XML tags are user-definable, an XML Schema is used to define which tags are valid in a particular document. Such definitions describe, for example, elements that can appear in a document (along with their attributes), whether an element is empty or can include text, and the data types for elements and attributes.

Thus, a document can be compared to an XML Schema — acting much like a blueprint, style guide or template — to ensure the document is valid and contains all relevant information. Again, this idea is not a new concept solely used by XML. Most databases have some sort of schema. Also, textbooks have used schemas for years, generally referring to them as “style guides.”

Here is an example of an XML schema for a memo:

SOAP and SOA
Simple Object Access Protocol (SOAP) is a lightweight protocol for the exchange of information. It describes three main elements: an envelope, the encoding rules, and a convention for the representation of remote procedure calls and responses. SOAP is no longer used; however, the technology remains very much in use.

Though it isn't used, the SOAP acronym is sometimes confused with Service-Oriented Architecture (SOA). Though SOAP standards may be part of an SOA application, the acronyms are not related.

SOA is a design philosophy that separates the core functions of a business into independent modules. These modules are referred to as services — essentially software functions — that are called by one or more other services in the overall workflow. (Media transcoding is an example of a service that could be invoked as part of an overall workflow management system.)

Operating on the principle of loosely coupled systems, SOA abstracts the functions of a service from the low-level substructures of that service, leaving the calling service to use much higher-level language to drive the process. In doing so, SOA can make it easier to choreograph multiple business activities and processes among multiple complex software systems, all working under the control of a central process to achieve the required goal.

Consider the operation of a typical media enterprise: Material comes into a facility on tape, via satellite or as a file transfer, and content from each of these sources must be processed before it can be used. Tapes must be ingested, satellite feeds must be fed through some sort of IRD, and files must be received and error-corrected. Some form of transcoding may then be applied in order to provide the material in the house format. A QC stage may then be applied to ensure technical compliance. In the software world, each of these would be software modules in a SOA-based system.

Without SOA, setting up such systems is a time-consuming process that likely will demand customization so that one vendor's automation/asset management system can talk to another vendor's software module or hardware processor. Any operational or technical changes may require replacement of a module or augmentation with another vendor's module.

For example, in a traditional workflow, the automation system must have deep knowledge of the proprietary API calls required to tell a transcoder to transcode a particular piece of material to another format. Every time a new format is added to the transcoder by its manufacturer, that interface must be modified (or in the worst case, completely rewritten). The custom integration software capable of addressing systems from different vendors thus presents a high-potential investment in time and money.

In a more agile SOA-based architecture, the automation system would simply send an XML message that says, “Transcode file X to format Y, and store the result as Z.” This message remains consistent, even if a new transcoder or format is added. Regardless of the vendor, equally equipped systems will perform the same basic transcode when given the same command. Knowing this, engineers can take a more flexible approach to workflow design. This model depends on the fact that each service describes itself and its capabilities to the rest of the system. Web Services Descriptive Language (WSDL) facilitates this exchange of information.

WSDL is an XML-based language used for describing features and capabilities of a particular service. Provided to the central controlling system, this information ensures all other services and applications are aware of a particular service's capabilities.

Four critical pieces are supplied:

• Interface information describing all publicly available functions;
• Data type information for all message requests and message responses;
• Binding information about the transport protocol to be used in calling the service;
• Address information for locating the specified service.

When several similar services exist in one environment, information delivered via WSDL allows an application to decide which of those is most appropriate for the current task and to engage that service as required.


REST
Representational State Transfer (REST), an architecture for distributed systems such as the World Wide Web, is targeted at HTML rather than XML. It addresses the scalability of component interaction, generality of interfaces and deployment of components to reduce latency and enforce security of transactions. Unlike SOA, which often is considered peer-to-peer architecture, REST is oriented toward client/server interaction where clients initiate requests to servers, and servers process requests and return appropriate responses.

In a RESTful transaction, requests and responses are built around the transfer of representations of resources, which themselves are specific information sources. A representation of a resource is typically a document that captures a resource's current or intended state. Each resource is referenced with a global identifier. To manipulate these resources, components of the network — typically user agents and origin servers — communicate via a standardized interface like HTTP and exchange representations.

In the case of web page information, the raw data on which the page is built is the resource, but the representation of that data could be different for different users. What is representation of a resource? Consider a human using a browser to view a web page in order to retrieve information, and a computer doing the same as part of some larger activity. Both navigate to the same page, but the computer has no interest in the page's layout and styling; it just wants the data. The human does care about layout and styling (which, after all, exists to make the page easier for humans to digest). The two clients (human and computer) want different representations of the resource, both derived from the same raw data source.

While SOA is object-oriented, with the object the service with which other services interact, REST is point-to-point and relies on “GET” commands to a specific URL. It enables code delivery in “applets” as needed, and this capability can reduce basic complexity of any client application. Specialized code (like JavaScript) can be delivered along with data and used to customize how data is presented or obtained.

By Paul Turner, Broadcast Engineering

JPEG 2000, from Master to Archive

Today's broadcasters are looking for the highest image quality, flexible delivery formats, interoperability and standardized profiles for interactive video transport and workflows. They also have a vested interest in a common high-end format to archive, preserve and monetize the avalanche of video footage generated globally.

This is the story behind the rapid adoption of JPEG 2000 compression in the contribution chain. Standardized broadcast profiles were adopted in 2010 to match current industry needs (JPEG 2000 Part 1 Amendment 3 — Profiles for Broadcast Application — ISO/IEC 15444-1:2004/Amd3), ensuring this wavelet-based codec's benchmark position in contribution.

In parallel, these broadcast profiles have also filled the industrywide need for compression standards to archive and create mezzanine formats, allowing transcoding to a variety of media distribution channels. The ongoing standardization process of the Interoperable Master Format (IMF) by SMPTE based on JPEG 2000 profiles brings the adoption full-circle.

The U.S. Library of Congress, the French Institut National de l'Audiovisuel (INA) and several Hollywood studios have selected the codec for the long-term preservation of a century of audio-visual contents.

JPEG 2000 is different from other video codecs. MPEG and other DCT-based codecs have been designed to optimize the compression efficiency to deliver video to viewers via a pipe with limited bandwidth. JPEG 2000, with its wavelet transform algorithm, brings features not only for image compression efficiency, but to give also the user better control and flexibility throughout the image processing chain. The codec provides unique features that are not available in any other compression method.


JPEG 2000 under the Spotlight
JPEG 2000 is based on the discrete wavelet transform (DWT) and uses scalar quantization, context modeling, arithmetic coding and post-compression rate allocation. JPEG 2000 provides random access (i.e. involving a minimal decoding) to the block level in each sub-band, thus making it possible to decode a region, a low-resolution or a low-quality image version of the image without having to decode it as a whole.

JPEG 2000 is a true improvement in functionality, providing lossy and lossless compression, progressive and parseable code streams, error resilience, region of interest (ROI), random access and other features in one integrated algorithm.

Many resolutions and different picture quality files can be derived from a single JPEG 2000 master.

In video applications, JPEG 2000 is used as an intraframe codec, so it closely matches the production workflow in which each frame of a video is treated as a single unit. In Hollywood, its ability to compress frame by frame has made this technology popular for digital intermediate coding. If the purpose of compression is the distribution of essence and no further editing is expected, long-GOP MPEG is typically preferred.


Broadcast Processes
JPEG 2000 brings valuable features to the broadcast process, including ingest, transcoding, captioning, quality control or audio track management. Its inherent properties fully qualify a codec for creating high-quality intermediate masters.

Post-production workflows consist of several encoding/decoding cycles. JPEG 2000 preserves the highest quality throughout this process, and no blocking artifacts are created. Moreover, the technology supports all common bit depths whether 8, 10, 12 bits or higher.

JPEG 2000 enables images to be compressed in lossy and visually or mathematically lossless modes for various applications. Additionally, its scalability allows a “create once, use many times” approach to service a wide range of user platforms.


Support for lossless or lossy compression gives the broadcaster more options.

The technology also enables improved editing: Even at the highest bit rates, its intrinsic flexibility makes it user-friendly on laptop and workstation editing systems, with a limited number of full bit rate real-time video tracks. Improving computing hardware is certain to increase the number of real-time layers.

Since JPEG 2000 is an intraframe codec, this prevents error propagation over multiple frames and allows the video signal to be cut at any point for editing or other purposes.

Easy transcoding appeals to high-end applications where workflows vastly benefit from transcoding to an intermediate version. JPEG 2000 ensures a clean and quick operation when bit rate is at a premium. Professional viewers have labeled correctly transcoded 1080p JPEG 2000 files compressed at 100Mb/s as “visually identical” to the original 2K footage. Furthermore, the wavelet-based JPEG 2000 compression does not interfere with the final — usually DCT-based — broadcast formats.

Last but not least, several standards specify in detail how the JPEG 2000 video stream should be encapsulated in a number of widely adopted containers such as MXF or MPEG-2 TS.


Professional Wireless Video Transmission
Wireless transmission is often challenged to improve its robustness in broadcast. Uncompressed HD wireless transmission is often seen as complex, for even if a 1080p60 transmission (3Gb/s) were possible wirelessly, it would be quite difficult to add the necessary FEC and encryption to the data stream. Of all the compression algorithms available in the market, JPEG2000 is seen as one of the top contenders for the following reasons.

JPEG 2000 is inherently more error resilient than MPEG codecs. The codestream can be configured so the most important data (the lowest frequency data contains the most visually significant information) is located in the front, while successively higher frequency, less important data can be placed in the back. Using appropriate FEC techniques, the lower frequency data can be protected while less protection can be applied to the higher frequency data, as errors in the higher frequency bands have much less effect on the displayed image quality.

Also, similar to the contribution, the low latency of JPEG 2000 is something that would be practically impossible for wireless systems using an MPEG system based on long GOP coding.


Long-term Preservation
The broadcasters and video archivists are looking for long-term digital preservation on disk. In most cases, the source material is not digital, but film that needs to be scanned or high-quality analog videotape. As such, a destination digital format must then be selected.

Key requirements often include reducing the storage costs of uncompressed video while still maintaining indefinite protection from loss or damage. Moreover, the format should preferably enable digitized content to be exploited, which means providing flexibility — workflows again — and security. For these reasons, several studies and user reports claim JPEG 2000 to be the codec for audio-visual archiving.

Several reasons make JPEG 2000 a codec of choice for audio-visual archiving:

• The JPEG 2000 standard can be used with two different wavelet filters: the 9/7 wavelet filter that is irreversible and the 5/3 wavelet filter that is fully reversible. The 5/3 wavelet filter offers a pure mathematically lossless compression that enables a reduction in storage requirement of 50 percent on average while still allowing the exact original image information to be recovered. The 9/7 wavelet filter can encode in lossy or visually lossless modes.


• The scalability allowing proxy extraction, multiple quality layers, is of huge interest to ease client browsing and retrieval or transcoding and streaming.


• JPEG 2000 is an open standard that supports every resolution, color depth, number of components and frame rate.


• JPEG 2000 is license- and royalty-free.

The Future
Several initiatives are pushing the industry beyond today's HD: NHK Super Hi-Vision, also known as 8K and UHDTV, the Higher Frame Rates in Cinema initiative by James Cameron and Peter Jackson (up to 120fps), 16-bit color depth, and the numerous manufacturers that are now offering 4K technology.

The need for efficient codecs has gained significant attraction amongst the industry. The future of JPEG 2000 is bright as it is an open standard that requires less power, consumes less space in hardware implementations and generally delivers greater scalability, flexibility and visual quality than other codecs. An increasing number of manufacturers, broadcasters and producers are using JPEG 2000 implementations to adapt today's industry to these new challenges.

By Jean-Baptiste Lorent and François Macé, Broadcast Engineering

Metadata and MXF

Most video professionals are now quite familiar with metadata. But, just to be sure everyone is on the same page, when we use the term metadata in this tutorial, we are referring to information (data) that typically refers to a program, commercial or other video/audio content. Examples of metadata include: the name of a program, the length of a segment or an ISCII code associated with a commercial.

Collectively, video, audio, subtitles, AFD and other things people consume as part of a program are known as essence. Just remember that essence refers to the stuff we are watching when we view a program. Importantly, essence includes data (subtitles, teletext, AFD and so on). If we want to talk about data that is part of the program, we talk about data essence. Data essence is not metadata; data essence is part of the program that is intended for “viewing” by the end consumer.

Technical vs. Descriptive
Metadata tends to be treated as a single topic. But, in our mind, there are several different ways to classify metadata. One way to look at metadata is whether it is technical or descriptive. Technical metadata is metadata that is required to identify and play back essence properly. Examples include: a Unique Media Identifier (UMID), information about compression used and raster size.

Descriptive metadata is typically the sort of information operators in professional media organizations care about — program title, show number, kill date, ISCII code and so on. Descriptive metadata might also include things like shot notes, scripts and music scores.

Some metadata falls into a grey area. How would you classify GPS location, zoom and focus setting, crank rate, or robotic camera position? Is this technical metadata or descriptive metadata? Generally speaking, technical metadata is directly related to the essence or essence coding and does not include these data types.

Another way to classify metadata is to think about how frequently the metadata is expected to change. We would expect some metadata to remain constant for the duration of the content being viewed. For example, we would not expect the title to change halfway through a program. In the United States, we might expect that the technical metadata regarding frame rate would remain at 59.97Hz for the duration of the program. As you can see from these examples, the frequency of metadata change is an orthogonal classification (meaning it is completely unrelated) to the metadata type (technical or descriptive). Not surprisingly, metadata that remains constant is called static metadata.

Other metadata, such as timecode, would be expected to increase monotonically (increase by a count of one, only) on every frame. Timecode and other metadata may, therefore, be placed on a timeline that increases predictably, and should never decrease (run backwards) or unexpectedly jump ahead. This type of metadata might be called timeline-related metadata.

Finally, metadata associated with shot logging, describing actors in scenes or associating a music score to a section of a program may appear fixed for a period of time, may be duplicated in several locations during a program or may even overlap with other similar types of metadata. This metadata can be referred to as event metadata.

There is one other important family of metadata, and that is perpetual metadata. As you can guess, this is metadata that should never change. Systems rely on the fact that these metadata items are immutable. If something is immutable, it means that it cannot be changed after it is created. Examples of this include a unique ID and the location where a program was filmed. Remember, systems will count on the immutability of certain metadata. If you violate the rules, all bets are off.

Why does all this matter? Because, in the end, media professionals build and operate systems, and these systems need to be properly designed to handle all of these different types of metadata.

Metadata and Essence
How do you tie timecode or program title to a specific copy of a movie? In the tape world, the question was irrelevant. The timecode was either stored in the VBI or in the ANC data, and the program title was printed on a label on the box or the tape itself.

But, things have changed in the file-based world. Associating metadata, especially technical metadata, to a particular program can be critical. Professional file formats must contain not only video and audio, but also the technical metadata necessary to play back the content. When it comes to descriptive metadata, the door was left open as to exactly what types of descriptive metadata were to be included in the file. This is both good and bad. One of the most critical linking mechanisms between metadata and essence is the UMID. The UMID is a unique identifier that is used to identify a particular piece of content.

UMIDs and the Contract
It is important to understand that there is an implicit contract between systems that create content labeled with a UMID and systems that later manipulate the content or rely on the UMID to retrieve the content. Here is the contract: A system will never break the connection between a particular piece of content and the UMID that was originally associated with that content.

If you create a new piece of content, give it a new UMID. Simple, right? But, here is a twist: What if you create a bit-for-bit exact copy of the file? The rule is that you duplicate everything, including the UMID. But, now you have two copies of the content with the same UMID. Is this right? The answer is yes. And, if you ask a media asset management system to retrieve a piece of content using a UMID, you cannot say for certain which copy you will get. The particular instance of a copy of content is not uniquely identified by a UMID. For other reasons, a MAM system may uniquely identify each copy of the same content, but the UMID should not be used to do this.

In a file-based world, metadata becomes key. Understanding how unique identifiers work and treating them correctly is vitally important. Fortunately, manufacturers hide most of this from end users, so you may not even be aware that UMIDs exist.

By Brad Gilmer, Broadcast Engineering