JPEG 2000 over IP

JPEG 2000 (J2K) is one of a number of compression formats that are used by professional media companies every day all over the world. J2K is generally used when high quality is required, such as for backhaul of national sporting events or for transfer of content between production facilities.

J2K can be configured to provide lossless compression, meaning that it is possible to prove that the video, after a compression/de-compression cycle, is mathematically identical in every way to the video prior to compression (lossless compression).

In the 1950s, AT&T built a nation-wide, terrestrial, video network for the big three networks. This was an RF-based analog system that remained in place for many years. In the 1960s, AT&T launched communications satellites, and AT&T and other satellite operators added video capability to these platforms over time. As a result, in the 1980s, satellite became the dominant long-haul technology.

During the dot-com boom, tens of thousands of miles of fiber optic cable were installed all over the country. The boom was followed by a bust, but the fiber was already in the ground. Thanks to this, megabit and now gigabit networking has become available on long-haul networks — at surprisingly reasonable prices in some cases.

One of the keys to networking is layering and encapsulation. Packetized networks use packets composed of a header and a payload section. The header contains information that is used to perform functions associated with that layer of the network functionality, and the payload section contains the information we want to transport across the network. Each layer performs a specific function. Let’s look at a specific example — the transport of J2K with audio over IP — to see how a layered approach is applied in a working scenario.

  This shows the transport of J2K with audio over IP, illustrating how a layered approach is applied in a working scenario.

We start with live professional video and audio — perhaps the output of a sports production truck. The video out of the truck is compressed using J2K, resulting in something called a JPEG 2000 Elementary Stream (ES). The audio at the side of the truck is already an AES stream.

Using MPEG-2
The JPEG standard says nothing about audio. Fortunately, we can use a portion of the MPEG-2 specification to multiplex JPEG-2000 ES and AES audio into a single MPEG-2 Transport Stream (TS) in a standardized way. This is an important point: The MPEG-2 specification covers all sorts of things besides compression. So, even though we feed this J2K video through equipment that is following the MPEG-2 specification, it is important to realize we are using J2K compression that is then fed into an MPEG-2 multiplexer, where it is combined with the AES audio. The result is a single MPEG-2 TS.

The MPEG-2 TS contains information that helps receivers reconstruct timing between video and audio streams. While this is vital to reproducing video and audio, these timestamps do not provide everything we need in order to deal with what happens in the real world on long-haul IP networks. Let’s look at some of these networks’ characteristics.

As IP packets travel over a network, they can take different paths from a sender to a receiver. Obviously, the inter-packet arrival time is going to change. In some cases, packets can arrive out of order or even be duplicated within the network. Having information about what has happened to packets as they transit the network allows smart receiver manufacturers to do all sorts of things in order to ensure that video and audio at the receive end are presented in a smooth stream. What we need is a way to embed information in the packets when they are transmitted, so that we can adjust for network behavior at the receiver.

RTP
Real-time Transport Protocol (RTP) allows manufacturers to insert precision time stamps and sequence numbers into packets at the transmitter. If we use these time stamps to indicate the precise time when the packets were launched, then at the receiver we can see trends across the network.

Is network delay increasing? What are the implications on buffer management at the receiver? This information allows receivers to adjust in order to produce the continuous stream at the output.

RTP sequence numbers are simply numbers that are inserted in the RTP header. The numbers increase sequentially. At a receiver, if you receive a packet stream in the order [1], [2], [4], [3], you know immediately that you need to reorder packets 3 and 4 in order to present the information to the MPEG-2 TS de-multiplexer in the order in which it was transmitted.

The next layer is User Datagram Protocol (UDP) encapsulation. MPEG-2 packets are 188 bytes. This data needs to be mapped into packets for transmission. The newly created SMPTE 2022-6 standard describes how to do this. UDP is designed to provide a simple scheme for building packets for network transmission. Transmission Control Protocol (TCP) is another alternative at this layer, but TCP is a much heavier implementation that, for a variety of reasons, is not well suited to professional live video transmission.

UDP packets are then encapsulated in IP datagrams, and at the IP layer, network source and destination addresses are then added. What this does is allow the network to route data from one location to another without the use of external routing control logic.

Finally, the IP datagrams are encapsulated in Ethernet packets. The Ethernet layer adds the specification of electrical and physical interfaces, in addition to Ethernet addressing that ties a specific physical device to an address, something IP addressing does not do.

Hopefully, this real-world example helps you to understand that layered systems are critical to the success of modern networked professional video, and that each layer adds something unique to the system.

By Brad Gilmer, Broadcast Engineering

Choosing JPEG 2000: The Growing Choice for Master File Format

Broadcasters, film studios and post-production houses are currently facing a major challenge in that the volume of generated video material is increasing dramatically. The result is a significant increase in the need for storage and archive capability.

Broadcasters and video archivists are also looking for long-term digital preservation. In most cases, the source material is not digital. Instead, it is on film that needs to be scanned or high-quality analog video tape.

A production and digital archive compression format, with no concessions in video content quality and the actual fabrication process, is the obvious choice — one that reduces storage costs compared to uncompressed video, while still maintaining indefinite protection from loss or damage. Such a format should preserve original quality, while also easily enabling the generation of most of the commonly used formats.

Several questions are frequent when selecting a format. What is the best physical long-term storage media for video content? What is a good candidate for a digital preservation? Can digital content be interpreted in the future? Various options are possible, and organizations have to decide carefully.

Today’s broadcasters understand the industry’s keywords: highest image quality, flexible delivery formats, interoperability and standardized profiles for optimal preservation. They also have a vested interest in a common high-end format to store, preserve and commercialize the avalanche of video footage generated globally. JPEG 2000 is the growing choice for master file format.


Digital Storage Keys
There are three keys to digital storage preservation:

• Ensure continuous access to content over time. Archive and storage covers all activities necessary to ensure continued access to digital materials for as long as necessary. This includes strategies to ensure access to reformatted and digitally-born content, regardless of the risks of media failure and technological changes. Quality preservation is crucial. A conversion system of archived items is important for dissemination or distribution.


• Everything that belongs together fits in one package. Archiving is an enduring process concerned with the impacts of changing technologies, whether it is the support of new media and data formats or a changing user community. “Long term” may extend indefinitely.


• Use open, well-documented industry standards — no proprietary formats. Ideally, focus on standards recognized and used for archiving applications. Open-file formats are published specifications, usually maintained by standards organizations, which can therefore be used and implemented by anyone. For example, an open format can be implemented by both proprietary and free/open-source softwares, using both types of software licenses. Open formats are also called free-file formats if they are not burdened by any copyrights, patents, trademarks or other restrictions. Anyone may use it at no cost for any desired purpose.

To standardize digital preservation practices and provide a set of recommendations for preservation program, the Reference Model for an Open Archival Information System (OAIS) was developed. OAIS is concerned with all technical aspects of a digital object’s life cycle: ingest into and storage in a preservation infrastructure, data management, accessibility and distribution. Continued interoperability is strategic; one needs easy and fast format conversion, as well as playback compatibility between manufacturers. For instance, a master file format must not be linked to any specific application, production format or major user.


JPEG 2000 in OP1a MXF
JPEG 2000 is based on Discrete Wavelet Transformation (DWT), scalar quantization, context modeling, arithmetic coding and post-compression rate allocation. JPEG 2000 provides random access (i.e., involving minimal decoding) to the block level in each sub-band, thus making it possible to decode a region, a low resolution or a low-quality image version without decoding the whole picture.



  JPEG 2000 is based on discrete wavelet transformation, scalar quantization, context modeling, arithmetic coding and post-compression rate allocation.

Functionally, JPEG 2000 is a true improvement that provides lossy and lossless compression, progressive and parseable code streams, error resilience, region of interest, proxies, random access and other features in one integrated algorithm.

In the video domain, JPEG 2000 is conceived as an intra-frame codec, so it closely matches the production workflow in which each video frame is treated as a single unit. Its ability to compress frame-by-frame has made it popular in the digital intermediate space in Hollywood. If the purpose of compression is the distribution of essence, and no further editing is expected, long-GOP MPEG will typically be preferred.

JPEG 2000 brings a storehouse of features to the broadcast process, whether ingest, transcoding, captioning, quality control or audio-track management is requested. Its inherent properties fully qualify it for high-quality, intermediate creation and masters archives. JPEG 2000 supports every resolution, color depth, number of components and frame rates; in short, the codec is future-proof.

The intra-frame quality of JPEG 2000 prevents error propagation over multiple frames and allows video signal edition at any given time. Two wavelet filters are included: the irreversible 9/7 and the fully reversible 5/3. The 5/3 wavelet filter offers a pure mathematically lossless compression, allowing an average 60-percent reduction in storage, while still allowing the exact original image information to be recovered. The 9/7 wavelet filter still performs visually lossless encoding. JPEG 2000 offers uncompressed quality, with no concession in video content quality and an important reduction in bandwidth and storage consumption.



  JPEG 2000 can be fully lossless or bit-to-bit reversible. Also, the intra-frame quality of the future-proof codec prevents error propagation over multiple frames.

Additionally, its scalability features a “create once, use many times” approach for a wide range of platforms. Easy transcoding of the codec appeals to high-end applications where workflows vastly benefit from transcoding to an intermediate version. JPEG 2000 ensures a clean, quick operation when bit-rate is at a premium.



  Shown here are typical profiles in use for the JPEG 2000 MXF OP1a master for preservation.

Correctly transcoded HD1080p JPEG 2000 files compressed at 100Mb/s have been labeled “visually identical” to the 2K original footage by professional viewers. Furthermore, the wavelet-based JPEG 2000 compression does not interfere with the final — usually DCT-based — broadcast formats.

Post-production workflows consist of several encoding/decoding cycles. JPEG 2000 preserves the highest quality throughout this process, without any blocking artifacts creation. Moreover, all common bit depths, whether it is 8-bit, 10-bit, 12-bit or 16-bit, are supported.

Uniquely matching current industry needs, standardized broadcast profiles were adopted in 2010 (JPEG 2000 Part 1 Amd 3 – Profiles for Broadcast Application - ISO/IEC 15444-1:2004/Amd3), ensuring this wavelet-based codec its benchmark position in contribution, while fulfilling the industry-wide request for compression standards to archive and create mezzanine formats. A variety of media distribution channels can be transcoded.

The ongoing standardization process of the Interoperable Master Format (IMF) by SMPTE, focused on JPEG 2000 profiles, brings the adoption full-closure. The SMPTE standards also specify, in detail, how JPEG 2000 video data should be encapsulated in the widely adopted MXF.

Finally, a non-technical feature makes the JPEG 2000 open standard even more attractive for long-term projects; it is license- and royalty-free.


Other Codecs
Most other codecs are proprietary. Some have compliancy issues and several limitations to support any video formats or resolutions. The MPEG family is ideal for last-mile content delivery to viewers, but not for production and storage, since pictures have to be post-processed.



  Looking at different parameters, JPEG 2000 appears to be ideal as a mezzanine file format.

Conclusion
JPEG 2000 has gained significant attraction as a mezzanine format. Open and well-documented, the codec is future-proof and extendable. That said, it is not surprising that the Library of Congress, France’s Institut National de l’Audiovisuel and several Hollywood studios, such as 20th Century Fox, have selected the codec for storage and preservation.

JPEG 2000 is a codec like no others. It gives users a superior quality, control and a unique flexibility of the image processing chain. The growing use of JPEG 2000 to archive and create mezzanine files, and the ongoing standardization process of the IMF based on JPEG 2000, are just a few of its advantages.

By Jean-Baptiste Lorent, Broadcast Engineering

Comparing JPEG 2000 and MPEG

Back in 1988, the first work began on video compression standards that would ultimately result in MPEG. The Motion Picture Experts Group, in cooperation with the International Organization for Standardization (ISO), created multiple standards for video compression, now known as MPEG-1, MPEG-2 and MPEG-4/AVC. MPEG was devised as a means of compressing broadband video into a small bit stream that could fit in extremely narrow broadcast or satellite transmission channels. As such, it is well-suited for distribution purposes, not for applications that require internal switching and processing.

MPEG
Because MPEG is a motion image compression technology, it works on a sequence of video frames, known as a Group of Pictures (GOP). A processor examines several frames of video and assigns one frame as the reference frame for that group (the I-frame). The GOP also has several predictive frames, or P frames, which use information from the I frame and previous P frames to construct images. Finally, there are bidirectional predictive frames, or B frames, which look at preceding and following I and P frames. Motion is analyzed and the motion vectors, which predict the offset from the current frame to the reference frame, are estimated down to a quarter pixel. The motion vectors and the difference between the actual images are used to reconstruct video that looks good at low bit rates.

  MPEG GOP structure

Problems with MPEG encoding arise when there is a scene change or where a large percentage of the image changes at once. The processor can’t predict the movement, and the system creates a distortion known as macroblocking. Raising the bit rate adds more detail in each 8 x 8 block of pixels, which can reduce the number of P frames. But there will always be P frames — and the potential for macroblocking — in all implementations of MPEG.




  Macroblocking effects

Another major failing of MPEG compression is that it is difficult to edit and switch cleanly. With P and B frames, editing systems and switchers have trouble finding a clean frame on which to edit. MPEG also doesn’t react well to being repeatedly encoded and decoded. It works best when the video is encoded once and decoded at the viewing location.


JPEG 2000
Meanwhile, JPEG — short for Joint Photographic Experts Group — was originally developed as a compression standard for still images. With modern processors, however, it is now possible to take a video signal and save each frame as a separate JPEG file, creating a video format known as Motion-JPEG.

Several years after the JPEG team created its first compression scheme, it reconvened to develop an even more efficient and flexible system. Released in 2000, its new JPEG compression was called JPEG 2000. The original JPEG used discrete cosine transform (DCT), the same techniques used in MPEG, to reduce data into blocks of 8 x 8 pixels. JPEG 2000 uses discrete wavelet transform (DWT), which performs simultaneous multiresolution image analysis.

The resulting wavelet coefficients are gathered in sub-bands, each of which is partitioned into small “code blocks” and independently coded by an adaptive binary arithmetic encoder. Finally, the output of the arithmetic encoder is organized as a compressed bit stream that offers a significant degree of flexibility.

Wavelet compression provides the same processing for all pixels, and with each video frame being compressed individually, there is no error propagation from one picture to the next. As a result, there is no macroblocking at low bit rates; instead, there is blurring on the images. JPEG works on a single image, stripping away redundant data and encoding each video frame independently for consistent high-quality images. As a result, editing may be performed on any frame in a sequence of images because each video frame contains its own picture information.

With no dependency on other images, JPEG 2000 has low latency, less than 1.5 frames encode and/or decode, and maintains sync between the video and audio. In contrast, MPEG and other compression schemes that rely on predictive frames and motion estimation algorithms have high latency.

JPEG 2000 and its underlying wavelet compression can allocate 10 bits or even 12 bits at 4:4:4 quality, a level in line with the demands of digital cinema post production. In fact, there are several systems that use wavelet compression techniques similar to JPEG 2000. REDCODE and CineForm, for example, are used successfully for digital cinema production, which demonstrates the quality and robustness of wavelet compression.


Contribution
There are significant differences between MPEG-4 and JPEG 2000 for broadcast contribution. Further development of video and still image compression standards continues, and there will no doubt be future advancements for both standards. For example, High Efficiency Video Coding (HEVC), unofficially called H.265, promises improved video quality and data compression, as well as support for higher resolutions.




  JPEG 2000 and MPEG-4/AVC attributes for broadcast contribution

Broadcast contribution links seek to have the highest video quality because the signal may have to go through post-production editing. The output video quality from the contribution stage is propagated to the distribution link. As a consequence, the quality and efficiency from the distribution link is highly dependent upon the level of quality experienced during contribution. Broadcast contribution signals should be lightly compressed — almost equal to the original uncompressed signal — so there is minimal quality loss under multiple generations of encoding. JPEG 2000 can compress HD video to a significantly lower bit rate, and then extract the video with no loss of information. When Peak Signal to Noise Ratio (PSNR) performance is used to compare original footage with video compressed with JPEG 2000, the degradation is close to visually lossless and deviation from the original is low. The PSNR of footage encoded with MPEG, however, can vary greatly from picture to picture due to different methods of prediction.


Performance Comparison
Recently, the performance of JPEG 2000 and MPEG-4/AVC under multigeneration encoding was evaluated using a single vendor’s encoders and decoders. Video quality was measured in terms of PSNR for different HD interlaced sequences at the operating points of the encoders. The streams were encoded at 90Mb/s for JPEG 2000 and 45Mb/s for MPEG-4/AVC streams. These bit rates were used throughout the testing because they are common operating points.




  PSNR simulation setup

MPEG-4/AVC provided high-quality video at 45Mb/s. Under multigeneration encoding and decoding, the PSNR reduction was sharp — roughly 2.2dB after the fourth generation. JPEG 2000 also delivered high-quality video (at 90Mb/s), but it had less than 0.5dB loss after the fourth generation of encoding and decoding, which was less than one quarter of the reduction experienced by MPEG-4. In addition, the predictive coding of MPEG resulted in an end-to-end latency of approximately 1 second, which is roughly 10 times more than the latency experienced with JPEG 2000.




  Multigeneration encoding

For another round of tests, designed to evaluate the influence of contribution encoding on end-user quality, a distribution encoder was used to encode the incoming signal using 4:2:0 8-bit at 6Mb/s, 8Mb/s and 10Mb/s (common settings for IPTV applications). The tests showed that because of JPEG 2000’s high video quality, it is possible to increase the compression ratio for the final compression to broadcast without quality loss, saving from 5 percent to 15 percent of the total bit rate, depending on the final bit rate used for distribution. By spending more money on contribution, it is possible to save on broadcasting to the home, because more content can be placed in the available bandwidth.

To illustrate the bandwidth savings that can be achieved when employing JPEG 2000 video compression, consider an uncompressed HD-SDI signal, which has a bit rate of about 1.5Gb/s. A JPEG 2000 encoder/decoder can maintain the quality of the original signal with a bit stream of 500Mb/s to 800Mb/s, depending on the content. At 600Mb/s, virtually any type of high-quality HD content will be transported mathematically losslessly with a JPEG 2000 system. Reducing the bit rate from there will increase the possibility of mathematical image loss, but any errors will be visually imperceptible even at far lower bit rates.




  Bandwidth savings

Stereo 3-D
Bandwidth-intensive formats like 3-D TV raise the bar even higher for quality and precision. The creation of 3-D video and particularly the need for the best possible picture quality has ramifications for how content is captured, produced and broadcast. In the contribution phase, left- and right-eye information must be transported along with other metadata required to generate the highest quality 3-D images and must be compatible with all 3-D compression and display systems. To ensure a high-quality viewing experience, it is critical that all the data required to display the left- and right-eye images correctly is conveyed throughout the transport chain with minimal visual impairment.

JPEG 2000 ensures that the horizontal resolution, critical to the reconstruction of 3-D images in our brains, is preserved. A JPEG 2000 encoder can transport two 2K or HD-SDI channels, or the two synchronous left-eye/right-eye video streams that comprise stereo 3-D. Even after the JPEG 2000 video is compressed or decompressed, and throughout real-time transport over Metro Ethernet (metropolitan-area Ethernet), IP-MPLS (MultiProtocol Label Switching) or SDH/SONET (Synchronous Digital Hierarchy or Synchronous Optical NETwork), the video quality stays true to the original imagery.


Applications
JPEG 2000 video compression is ideal for venues where high-quality programming regularly originates, such as stadiums, arenas and remote studios. While prices vary between service providers, commonly available IP links that provide high bit rates are universally less expensive than legacy systems, which translates to significant long-term operational cost savings. Before connecting an encoder, however, it is critical that the service provider understands the Quality of Service (QoS) requirements of high-quality video and can ensure sufficient bandwidth.

JPEG 2000 requires a high-bit-rate IP network and usually works best with both a primary and redundant link. If you need to do a remote shoot from a non-fixed location, i.e., in front of a burning house, then you probably need to use a microwave or satellite truck with MPEG compression.

Final signal distribution to viewers is a perfect application for MPEG. JPEG 2000 excels as a contribution technology, but is not intended as a distribution technology. That said, stadiums, city halls, remote studios, OB trucks and fixed locations where you need the highest quality video are good choices to consider using JPEG 2000 to maintain optimal video quality. As the cost of dedicated IP networks comes down and bit rates increase, JPEG 2000 video will help control costs when compared to traditional high-end video transport systems. Careful consideration of your outside broadcast requirements, access to high-speed IP networks and JPEG 2000 capabilities provide another choice in your video transport toolkit.

By Helge Stephansen, Broadcast Engineering

JPEG 2000, from Master to Archive

Today's broadcasters are looking for the highest image quality, flexible delivery formats, interoperability and standardized profiles for interactive video transport and workflows. They also have a vested interest in a common high-end format to archive, preserve and monetize the avalanche of video footage generated globally.

This is the story behind the rapid adoption of JPEG 2000 compression in the contribution chain. Standardized broadcast profiles were adopted in 2010 to match current industry needs (JPEG 2000 Part 1 Amendment 3 — Profiles for Broadcast Application — ISO/IEC 15444-1:2004/Amd3), ensuring this wavelet-based codec's benchmark position in contribution.

In parallel, these broadcast profiles have also filled the industrywide need for compression standards to archive and create mezzanine formats, allowing transcoding to a variety of media distribution channels. The ongoing standardization process of the Interoperable Master Format (IMF) by SMPTE based on JPEG 2000 profiles brings the adoption full-circle.

The U.S. Library of Congress, the French Institut National de l'Audiovisuel (INA) and several Hollywood studios have selected the codec for the long-term preservation of a century of audio-visual contents.

JPEG 2000 is different from other video codecs. MPEG and other DCT-based codecs have been designed to optimize the compression efficiency to deliver video to viewers via a pipe with limited bandwidth. JPEG 2000, with its wavelet transform algorithm, brings features not only for image compression efficiency, but to give also the user better control and flexibility throughout the image processing chain. The codec provides unique features that are not available in any other compression method.


JPEG 2000 under the Spotlight
JPEG 2000 is based on the discrete wavelet transform (DWT) and uses scalar quantization, context modeling, arithmetic coding and post-compression rate allocation. JPEG 2000 provides random access (i.e. involving a minimal decoding) to the block level in each sub-band, thus making it possible to decode a region, a low-resolution or a low-quality image version of the image without having to decode it as a whole.

JPEG 2000 is a true improvement in functionality, providing lossy and lossless compression, progressive and parseable code streams, error resilience, region of interest (ROI), random access and other features in one integrated algorithm.

Many resolutions and different picture quality files can be derived from a single JPEG 2000 master.

In video applications, JPEG 2000 is used as an intraframe codec, so it closely matches the production workflow in which each frame of a video is treated as a single unit. In Hollywood, its ability to compress frame by frame has made this technology popular for digital intermediate coding. If the purpose of compression is the distribution of essence and no further editing is expected, long-GOP MPEG is typically preferred.


Broadcast Processes
JPEG 2000 brings valuable features to the broadcast process, including ingest, transcoding, captioning, quality control or audio track management. Its inherent properties fully qualify a codec for creating high-quality intermediate masters.

Post-production workflows consist of several encoding/decoding cycles. JPEG 2000 preserves the highest quality throughout this process, and no blocking artifacts are created. Moreover, the technology supports all common bit depths whether 8, 10, 12 bits or higher.

JPEG 2000 enables images to be compressed in lossy and visually or mathematically lossless modes for various applications. Additionally, its scalability allows a “create once, use many times” approach to service a wide range of user platforms.


Support for lossless or lossy compression gives the broadcaster more options.

The technology also enables improved editing: Even at the highest bit rates, its intrinsic flexibility makes it user-friendly on laptop and workstation editing systems, with a limited number of full bit rate real-time video tracks. Improving computing hardware is certain to increase the number of real-time layers.

Since JPEG 2000 is an intraframe codec, this prevents error propagation over multiple frames and allows the video signal to be cut at any point for editing or other purposes.

Easy transcoding appeals to high-end applications where workflows vastly benefit from transcoding to an intermediate version. JPEG 2000 ensures a clean and quick operation when bit rate is at a premium. Professional viewers have labeled correctly transcoded 1080p JPEG 2000 files compressed at 100Mb/s as “visually identical” to the original 2K footage. Furthermore, the wavelet-based JPEG 2000 compression does not interfere with the final — usually DCT-based — broadcast formats.

Last but not least, several standards specify in detail how the JPEG 2000 video stream should be encapsulated in a number of widely adopted containers such as MXF or MPEG-2 TS.


Professional Wireless Video Transmission
Wireless transmission is often challenged to improve its robustness in broadcast. Uncompressed HD wireless transmission is often seen as complex, for even if a 1080p60 transmission (3Gb/s) were possible wirelessly, it would be quite difficult to add the necessary FEC and encryption to the data stream. Of all the compression algorithms available in the market, JPEG2000 is seen as one of the top contenders for the following reasons.

JPEG 2000 is inherently more error resilient than MPEG codecs. The codestream can be configured so the most important data (the lowest frequency data contains the most visually significant information) is located in the front, while successively higher frequency, less important data can be placed in the back. Using appropriate FEC techniques, the lower frequency data can be protected while less protection can be applied to the higher frequency data, as errors in the higher frequency bands have much less effect on the displayed image quality.

Also, similar to the contribution, the low latency of JPEG 2000 is something that would be practically impossible for wireless systems using an MPEG system based on long GOP coding.


Long-term Preservation
The broadcasters and video archivists are looking for long-term digital preservation on disk. In most cases, the source material is not digital, but film that needs to be scanned or high-quality analog videotape. As such, a destination digital format must then be selected.

Key requirements often include reducing the storage costs of uncompressed video while still maintaining indefinite protection from loss or damage. Moreover, the format should preferably enable digitized content to be exploited, which means providing flexibility — workflows again — and security. For these reasons, several studies and user reports claim JPEG 2000 to be the codec for audio-visual archiving.

Several reasons make JPEG 2000 a codec of choice for audio-visual archiving:

• The JPEG 2000 standard can be used with two different wavelet filters: the 9/7 wavelet filter that is irreversible and the 5/3 wavelet filter that is fully reversible. The 5/3 wavelet filter offers a pure mathematically lossless compression that enables a reduction in storage requirement of 50 percent on average while still allowing the exact original image information to be recovered. The 9/7 wavelet filter can encode in lossy or visually lossless modes.


• The scalability allowing proxy extraction, multiple quality layers, is of huge interest to ease client browsing and retrieval or transcoding and streaming.


• JPEG 2000 is an open standard that supports every resolution, color depth, number of components and frame rate.


• JPEG 2000 is license- and royalty-free.

The Future
Several initiatives are pushing the industry beyond today's HD: NHK Super Hi-Vision, also known as 8K and UHDTV, the Higher Frame Rates in Cinema initiative by James Cameron and Peter Jackson (up to 120fps), 16-bit color depth, and the numerous manufacturers that are now offering 4K technology.

The need for efficient codecs has gained significant attraction amongst the industry. The future of JPEG 2000 is bright as it is an open standard that requires less power, consumes less space in hardware implementations and generally delivers greater scalability, flexibility and visual quality than other codecs. An increasing number of manufacturers, broadcasters and producers are using JPEG 2000 implementations to adapt today's industry to these new challenges.

By Jean-Baptiste Lorent and François Macé, Broadcast Engineering

JPEG 2000

JPEG 2000 has caught the attention of the professional media world for good reason. First, it closely matches some workflows, where the production process operates on each frame of a video stream as a discrete unit. This is different from MPEG-2 and MPEG-4 AVC Long-GOP flavors, where, during the reconstruction process, algorithms reference frames before and after the frame being reconstructed.

The ability to compress each frame as a free-standing unit has made it popular in the digital intermediate space in Hollywood. JPEG 2000 is also of interest to those who want lossless compression. It can provide a bit-perfect reconstruction of the original compressed image, although at a cost in terms of bandwidth. Also, the wavelet compression used in JPEG 2000 provides some unique opportunities that are not available in other compression methods.

The wavelet transform separates the image into four sub-bands. The first sub-band is a lowpass horizontal and lowpass vertical (LL). Images that have gone through this separation are basically lower-resolution images of the original. The other sub-bands are as follows: lowpass horizontal and highpass vertical (LH); highpass horizontal and lowpass vertical (HL); and highpass horizontal and highpass vertical (HH).

Using wavelet transforms and some clever thinking, implementers can do some interesting things. For example, they can send only the LL image, if they know that they are feeding a low-resolution display. Or they can send the LL sub-band in a highly-protected stream, in order to ensure the original image arrives intact. That said, they can then send the higher-resolution sub-bands unprotected since a momentary loss of these sub-bands is not likely to be noticed.

Given JPEG 2000's popularity, it is not surprising there have been some developments that make it particularly interesting for professional applications. First, the ITU has created an amendment (1) that outlines specific configurations for broadcast contribution applications. These configurations are intended to establish interoperability points for those implementing JPEG 2000 in professional applications. This is important because, until the amendment was released, there were so many variables in the compression tool set that interoperability was unlikely. The second important development, Amendment 5 to the MPEG 2 standard (2), provides a mapping of the JPEG 2000 Program Elementary Stream (PES) onto the MPEG-2 Transport Stream (TS).

Finally, some time ago, the Pro-MPEG Forum started to develop a standardized way to transport MPEG-2 TS over IP networks. The Video Services Forum picked up on this work and continued to develop it, finally submitting a draft for standardization within the SMPTE. This standard, SMPTE 2022-2 (3), describes a method for mapping MPEG-2 Transport Streams onto IP networks using RTP and UDP. The document was approved in 2007 and is the most common standard deployed today for professional video transport applications.

So these three developments — development of broadcast profiles; a mapping of JPEG 2000 Program Elementary Streams to MPEG 2 Transport Streams; and wide availability of MPEG-2 TS over IP transport equipment — mean now it is possible to transport JPEG 2000 over IP networks.

JPEG 2000 over IP relies on three critical developments:
broadcast contribution profiles, JPEG 2000 PES to MPEG-2 mapping,
and SMPTE 2022-2 for MPEG-2 transport over IP networks.

Starting with a video source, the image is compressed using a compression engine. This engine is configured to one of the Broadcast Contribution profiles in ITU-T Amendment 3. The compression engine produces a JPEG 2000 PES. This stream is then fed to an MPEG-2 encapsulator. The encapsulator uses the mapping rules established in the MPEG-2 specification, Amendment 5, to map the PES onto an MPEG-2 TS. This MPEG-2 TS is fully compliant with MPEG-2 specifications because, from the outside, it looks just like a normal MPEG-2 Transport Stream. As such, the output of the MPEG-2 encapsulator can be fed into a SMPTE 2022-2 compliant video transport device. This device encapsulates the MPEG-2 TS in standard RTP and UDP packets, and then those packets are wrapped in IP packets. These IP packets can now be fed into an IP network.

You might wonder why we take a relatively new compression algorithm such as JPEG 2000 and encapsulate it in MPEG-2. There are several reasons for this. First and foremost, there are already a number of specifications for how to encapsulate a number of different audio formats into MPEG-2 Transport Streams. Remember: JPEG 2000 says nothing about audio. Using MPEG-2 TS allows us to transport and present the audio alongside the JPEG 2000 compressed video using well-known audio standards. Also, this approach allows us to leverage the existing SMPTE 2022-2 MPEG-2 TS over IP standard. Finally, there are no technical issues in MPEG-2 TS that need to be fixed in this application space, so re-use of Transport Streams rather than inventing something entirely new seems like a good solution.

So, the good news is that the time is ripe for development of an interoperable, open solution for the transport of JPEG 2000 video and audio over IP networks. The standards exist, and there is a clear path forward. But, there are a few issues that need addressed.

JPEG 2000 has been around for quite some time. As such, some proprietary JPEG 2000 over IP transport solutions have already been created. Of course, these were developed in response to customer demand, so existing implementations may need to be changed. Another issue is that while the broadcast contribution profiles in Amendment 3 go a long way toward interoperability in the JPEG 2000 PES space, recent analysis suggests, without further definition, implementations based upon these profiles will not be interoperable. Finally, until the industry actually tries to connect devices from different manufacturers together, interoperability cannot be assured.

Fortunately, the industry is becoming aware of these issues, and steps are being taken to begin work in earnest on interoperable, open transport of professional JPEG 2000 images over IP networks. I would expect to see some developments around this in the first half of the coming year.

Footnotes:

1. “Profiles for Broadcast Applications” ISO/IEC 15444-1:2004 Amd.3-2010 (ISO/IEC, Geneva, Switzerland: 2010) |Rec. ITU-T T.800 Amd.3 (06/2010) (ITU, Geneva, Switzerland:2010)
2. Amendment 5: Transport of JPEG 2000 Part 1 (ITU-T Rec T.800 | ISO/IEC 15444-1) video over ITU-T Rec H.222.0| ISO/IEC 13818-1
3. SMPTE ST20 22-2:2007 “Unidirectional Transport of Constant Bit Rate MPEG-2 Transport Streams on IP Networks”

By Brad Gilmer, Broadcast Engineering