Choosing JPEG 2000: The Growing Choice for Master File Format
Broadcasters, film studios and post-production houses are currently facing a major challenge in that the volume of generated video material is increasing dramatically. The result is a significant increase in the need for storage and archive capability.
Broadcasters and video archivists are also looking for long-term digital preservation. In most cases, the source material is not digital. Instead, it is on film that needs to be scanned or high-quality analog video tape.
A production and digital archive compression format, with no concessions in video content quality and the actual fabrication process, is the obvious choice — one that reduces storage costs compared to uncompressed video, while still maintaining indefinite protection from loss or damage. Such a format should preserve original quality, while also easily enabling the generation of most of the commonly used formats.
Several questions are frequent when selecting a format. What is the best physical long-term storage media for video content? What is a good candidate for a digital preservation? Can digital content be interpreted in the future? Various options are possible, and organizations have to decide carefully.
Today’s broadcasters understand the industry’s keywords: highest image quality, flexible delivery formats, interoperability and standardized profiles for optimal preservation. They also have a vested interest in a common high-end format to store, preserve and commercialize the avalanche of video footage generated globally. JPEG 2000 is the growing choice for master file format.
Digital Storage Keys
There are three keys to digital storage preservation:
- Ensure continuous access to content over time. Archive and storage covers all activities necessary to ensure continued access to digital materials for as long as necessary. This includes strategies to ensure access to reformatted and digitally-born content, regardless of the risks of media failure and technological changes. Quality preservation is crucial. A conversion system of archived items is important for dissemination or distribution.
- Everything that belongs together fits in one package. Archiving is an enduring process concerned with the impacts of changing technologies, whether it is the support of new media and data formats or a changing user community. “Long term” may extend indefinitely.
- Use open, well-documented industry standards — no proprietary formats. Ideally, focus on standards recognized and used for archiving applications. Open-file formats are published specifications, usually maintained by standards organizations, which can therefore be used and implemented by anyone. For example, an open format can be implemented by both proprietary and free/open-source softwares, using both types of software licenses. Open formats are also called free-file formats if they are not burdened by any copyrights, patents, trademarks or other restrictions. Anyone may use it at no cost for any desired purpose.
To standardize digital preservation practices and provide a set of recommendations for preservation program, the Reference Model for an Open Archival Information System (OAIS) was developed. OAIS is concerned with all technical aspects of a digital object’s life cycle: ingest into and storage in a preservation infrastructure, data management, accessibility and distribution. Continued interoperability is strategic; one needs easy and fast format conversion, as well as playback compatibility between manufacturers. For instance, a master file format must not be linked to any specific application, production format or major user.
JPEG 2000 in OP1a MXF
JPEG 2000 is based on Discrete Wavelet Transformation (DWT), scalar quantization, context modeling, arithmetic coding and post-compression rate allocation. JPEG 2000 provides random access (i.e., involving minimal decoding) to the block level in each sub-band, thus making it possible to decode a region, a low resolution or a low-quality image version without decoding the whole picture.
Functionally, JPEG 2000 is a true improvement that provides lossy and lossless compression, progressive and parseable code streams, error resilience, region of interest, proxies, random access and other features in one integrated algorithm.
In the video domain, JPEG 2000 is conceived as an intra-frame codec, so it closely matches the production workflow in which each video frame is treated as a single unit. Its ability to compress frame-by-frame has made it popular in the digital intermediate space in Hollywood. If the purpose of compression is the distribution of essence, and no further editing is expected, long-GOP MPEG will typically be preferred.
JPEG 2000 brings a storehouse of features to the broadcast process, whether ingest, transcoding, captioning, quality control or audio-track management is requested. Its inherent properties fully qualify it for high-quality, intermediate creation and masters archives. JPEG 2000 supports every resolution, color depth, number of components and frame rates; in short, the codec is future-proof.
The intra-frame quality of JPEG 2000 prevents error propagation over multiple frames and allows video signal edition at any given time. Two wavelet filters are included: the irreversible 9/7 and the fully reversible 5/3. The 5/3 wavelet filter offers a pure mathematically lossless compression, allowing an average 60-percent reduction in storage, while still allowing the exact original image information to be recovered. The 9/7 wavelet filter still performs visually lossless encoding. JPEG 2000 offers uncompressed quality, with no concession in video content quality and an important reduction in bandwidth and storage consumption.
Additionally, its scalability features a “create once, use many times” approach for a wide range of platforms. Easy transcoding of the codec appeals to high-end applications where workflows vastly benefit from transcoding to an intermediate version. JPEG 2000 ensures a clean, quick operation when bit-rate is at a premium.
Correctly transcoded HD1080p JPEG 2000 files compressed at 100Mb/s have been labeled “visually identical” to the 2K original footage by professional viewers. Furthermore, the wavelet-based JPEG 2000 compression does not interfere with the final — usually DCT-based — broadcast formats.
Post-production workflows consist of several encoding/decoding cycles. JPEG 2000 preserves the highest quality throughout this process, without any blocking artifacts creation. Moreover, all common bit depths, whether it is 8-bit, 10-bit, 12-bit or 16-bit, are supported.
Uniquely matching current industry needs, standardized broadcast profiles were adopted in 2010 (JPEG 2000 Part 1 Amd 3 – Profiles for Broadcast Application - ISO/IEC 15444-1:2004/Amd3), ensuring this wavelet-based codec its benchmark position in contribution, while fulfilling the industry-wide request for compression standards to archive and create mezzanine formats. A variety of media distribution channels can be transcoded.
The ongoing standardization process of the Interoperable Master Format (IMF) by SMPTE, focused on JPEG 2000 profiles, brings the adoption full-closure. The SMPTE standards also specify, in detail, how JPEG 2000 video data should be encapsulated in the widely adopted MXF.
Finally, a non-technical feature makes the JPEG 2000 open standard even more attractive for long-term projects; it is license- and royalty-free.
Other Codecs
Most other codecs are proprietary. Some have compliancy issues and several limitations to support any video formats or resolutions. The MPEG family is ideal for last-mile content delivery to viewers, but not for production and storage, since pictures have to be post-processed.
Conclusion
JPEG 2000 has gained significant attraction as a mezzanine format. Open and well-documented, the codec is future-proof and extendable. That said, it is not surprising that the Library of Congress, France’s Institut National de l’Audiovisuel and several Hollywood studios, such as 20th Century Fox, have selected the codec for storage and preservation.
JPEG 2000 is a codec like no others. It gives users a superior quality, control and a unique flexibility of the image processing chain. The growing use of JPEG 2000 to archive and create mezzanine files, and the ongoing standardization process of the IMF based on JPEG 2000, are just a few of its advantages.
By Jean-Baptiste Lorent, Broadcast Engineering
