Worldwide 3D Broadcasting Firing Up Competition for Industry Standard
In December 2007, BS11 was the first broadcaster in the world to begin 3D broadcasting on an almost continuous basis. It stood alone for years, but the situation changed dramatically in 2010 as a host of broadcasters around the world began offering 3D broadcasting and distribution services using the same technology, and 3D-capable TVs became generally available from multiple manufacturers.
"It's a whole new ball game this time, because 3D TVs are on the shelves, and the environment is ready for 3D broadcasting. Finally 3D broadcasting and 3D sets are both getting ready at the same time," says Hiroshi Endo, Nippon BS Broadcasting Corp. (BS11) of Japan.
The 3D broadcasting method adopted by BS11 is called side-by-side broadcasting. It offers excellent compatibility with the MPEG-2 high-efficiency coding scheme commonly used in broadcast data, which seems to be emerging as the de facto standard for 3D broadcasting. It is unclear whether or not it will be established as the standard, however, and it is entirely possible that side-by-side could end up as merely a transitional step.
In the broadest sense, side-by-side applies to all transmission schemes sending the image streams for left and right eyes in parallel. 3D Video of the US began trial 3D broadcasting in Mexico in 1954, and today the general opinion in the industry is that side-by-side is not covered by a specific patent. In 1991, however, RealD claimed that it had acquired the basic patent for side-by-side. Concretely, two image streams compatible with existing broadcasting are compressed as left and right images positioned next to each other horizontally, transmitted, decompressed at the TV set, and displayed in a time-multiplexed manner. The image display is synched to the LCD shutter speed of the glasses worn by the viewer. The patent is US5193000 and it expires in August 2011.
There are a number of issues involved with side-by-side, namely (1) existing side-by-side technology has limitations in terms of resolution (it cannot display full-high definition imagery) and 3D image fidelity, (2) there are a number of varieties of side-by-side, possibly making it impossible to guarantee interoperability between various broadcasts and sets, and (3) The International Telecommunication Union (ITU) and other standardization organizations are talking about standardizing H.264-based 3D broadcasting into addition to MPEG-2. In other words, 3D broadcasting standardization is only really starting now that a number of broadcasters and TV manufacturers are actually offering the commercial services (Fig. 1).
RealD Method Sweeping the Industry
It would be no exaggeration to say that the 3D broadcasting starting up around the world is all "private brand," because the vast majority of it is using proprietary broadcasting methods. The 3D TV manufacturers have no obligation to support them all, and there are no guarantees that broadcast interoperability will be achieved. Worse, the various 3D broadcasters working on standardization are not pulling in the same direction: Most of them are primarily concerned about not falling behind in the surging "3D broadcasting" boom.
One 3D broadcasting/distribution technology has made great strides in the market in 2010, though: the RealD format for side-by-side. Following the announcement by Sony in December 2009 that it had adopted the format, there was a rush of similar announcements at the 2010 International Consumer Electronics Show (CES) held in January 2010. Companies announcing adoption of the method include JVC, Panasonic, Toshiba, Samsung and DIRECTV.
RealD is famous for its 3D filming technology and viewing glasses, using polarized light, but it also holds a number of crucial patents in side-by-side transmission and LCD shutter glasses. The flood of announcements at CES was driven by these resources. JVC, for example, cited one of the key reasons for their selection of the RealD format as the fact that the firm already holds basic patents to side-by-side and 3D glasses.
Side-by-side specs other than the RealD format are far from dead, though. In Japan, the MT method from NHK Media Technology has been picked up by broadcasters including BS11 and BS-TBS. In the US and UK the SENSIO 3D scheme from Sensio Technologies has been is used for some time now in broadcasting popular sporting events, such as National Basketball League (NBA) All-Star games. At the 2010 CES, VIZIO announced that it has chosen SENSIO 3D, marking a sudden pause in the rush to adopt the RealD format.
TV Manufacturers Participating in Broadcasting
Another intriguing characteristics of 3D broadcasting and distribution is that in many cases the TV manufacturers are getting involved from the planning stage. Until now, broadcasting standards have been decided on the national scale (or larger), adopted by broadcasting stations, and watched on receivers developed for those standards by TV manufacturers. This new development is a total reversal.
Concretely, Panasonic is supporting DIRECTV's 3D broadcasting initiative. As the exclusive sponsor they not only have an active promotional campaign for their 3D TVs and other products, implemented via TV commercials, but also help out by providing the 3D cameras and editing equipment used in producing 3D content.
Sony is active, too. It tied up with Comcast in April 2010 to handle photography and other aspects of 3D coverage of the Masters Golf Tournament. The company also garnered the rights to 3D photography and Blu-ray Disc sale for the Soccer World Cup beginning in June 2010. It is also working on a 3D-only broadcasting channel scheduled for rollout in the US in 2011, with major TV network Discovery Communications.
Outside of Japanese manufacturers, Samsung Electronics and LG Electronics of Korea are especially active. Together with government agencies such as the Korea Communication Commission (KCC), they are standardizing 3D broadcasting (including using terrestrial broadcasting), and developing new technologies.
TV manufacturers are hardly sitting by and waiting for 3D broadcasting to start, though, because even if they start marketing 3D TVs in a big way there is a serious risk involved should 3D broadcasting fizzle. Even though 3D movies are popular and Blu-ray Disc supports 3D media, "there are just not so many movies with the draw of Avatar," says one source in the broadcasting industry. Rather than waiting passively for the next hit item, it will be essential to build up a rich selection of 3D content for broadcasting. It is entirely possible, in fact, that the actual implementation of 3D broadcasting may be strongly influenced—if not decided—by the TV manufacturers.
Safety Net: Ensuring Compatibility with Existing Broadcasting
Even though broadcasters and set manufacturers are forging ahead with plans for 3D broadcasting in spite of the fact there is no standard, there are few complaints that a set cannot show 3D broadcast content. One of the reasons is that many broadcasters have adopted the above-mentioned side-by-side method, which helps prevent a sudden rush of mutually incompatible approaches.
The broadcast signal input interface in television sets is actually settling on side-by-side technology (Fig. 2). Different manufacturers have adopted different approaches to displaying the 3D imagery, such as frame sequential (FS) or Xpol, but the input interface is fairly independent of the display technology. Panasonic and Sony say their 3D sets can handle 3D broadcasting and video distribution from BS11, Jupiter Telecommunications (J:COM) and SKY Perfect JSAT. BS11's Endo adds "We are still verifying interoperability of sets from various manufacturers, but there aren't any problems so far."
Most 3D image broadcasting systems have adopted side-by-side because it ensures frame compatibility with existing broadcast technology (Fig. 3). In side-by-side, two images for left and right eye captured with the 3D camera are compressed into two frames, and broadcast as a single video frame. The television splits out the left and right images from the received frame, decompresses each to 2x and displays the result.
Data is transmitted in a single stream, just as with two-dimensional (2D) broadcasting, and viewers have no need to replace their existing tuners or set-top boxes (STB); all they need to do is buy a 3D-capable TV set. Likewise, broadcasters will not need to replace their existing broadcasting or relay equipment, with the exception of cameras and editing systems. As a source at BS11 explains, "As long as broadcasting is based on MPEG-2, side-by-side is the only choice."
Compression/Decompression Technology Makes the Difference in Image Quality
That alone will not wholly resolve the compatibility issues between different 3D broadcasters and TVs, though. The problem is that there is a variety of implementation for the above-mentioned RealD and SENSIO 3D formats. Concretely, there are differences in image compression and decompression technology, and which (if any) error correction technologies are used. These differences manifest themselves as differences in image resolution and data processing speed in many cases. The details of the physical implementation will sway image fidelity and price.
Assuming that side-by-side is the easiest method to implement, then the horizontal image resolution after compression will degrade to half the source for both left and right images (Fig. 4). This compression is, in principle, non-recoverable, which means significant quality loss is unavoidable when the images are decompressed for viewing on the set. A number of proprietary specifications have emerged as engineers search for ways to minimize this image deterioration. A source at one broadcaster says "The difference in compression/decompression technology between the first 3D sets in about 2007, and the sets today, makes an enormous difference in image quality."
Of these proprietary specifications, the three most commonly used are the RealD format, SENSIO 3D and NHK Media Technology's MT. The detailed specifications are not available for any of them, but the differences are clear in some areas. The different image compression method, for example, raises worries about compatibility between SENSIO 3D and the other methods.
SENSIO 3D samples the left and right source images like a checkerboard for compression. In decompression, the eliminated pixels are interpolated using degradation-free pixels (Fig. 4b). According to Sensio, the technology takes advantage of the facts that people are not as sensitive to resolution drop in the diagonal direction, and that leaving lossless pixels also leaves the high-frequency component to make possible more detailed interpolation. Sensio Marketing and Communication Director Esther Hotter explains "Of course there is some degradation in the decompressed imagery, but it's almost invisible to human eye." The amount of processing needed for decompression is minimal, and can be adequately handled by the inexpensive Spartan-3 series of field-programmable gate arrays (FPGA) from Xilinx.
The RealD and MT formats, on the other hand, do not degrade resolution in the diagonal direction in video streams for TV broadcasting. Shigeru Terada, NHK Media Technology, explains "Diagonal lines in the image cause the jaggies." The Science & Technology Research Laboratories of Japan Broadcasting (NHK) developed a technology called "Falcon" to degrade information in the diagonal direction, similar to the Sensio concept, but abandoned the approach when it was unable to overcome the problem with jagged edges. It then switched to the present MT scheme. Sensio rebuts: "SENSIO 3D does not experience such a problem, as has been demonstrated in any number of actual sports events."
Fight for Dominance to Continue
At present, most of the major TV manufacturers with the exception of VIZIO use the RealD format. Sensio's Hotter counters, "We are confident that major TV manufacturers other than VIZIO will rate the 3D image quality of our system highly. Broadcasters using video on demand service to compete with Blu-ray Disc are very interested." The competition for leadership in the industry seems likely to continue for some time to come, and one of the key reasons is that RealD's US patent on side-by-side technology will expire in August 2011.
The TV manufacturers may, of course, decide to implement both methods. Hyundai IT, in fact, has signed agreement for both the RealD format and SENSIO 3D. Some of the Japanese manufacturers using the RealD format are rumored to be ready to sign with SENSIO 3D in addition.
Developers in both camps, according to NHK Media Technology's Terada, "agree that 3D image quality is best when compression and decompression are implemented with the same technology." Even with different methods, though, deterioration should be minimal in most cases. Images compressed into the MT format, are said to have no problems when decompressed using the RealD format.
Combinations of SENSIO 3D and other methods, though, are yet unknown, and opinions are mixed inside the industry as well. One Japanese broadcaster expressed worry that major problems could develop with compatibility. A source at JVC, however, which released a commercial display capable of showing Xpol 3D imagery in 2009, said "We haven't implemented SENSIO 3D, but even when a video stream compressed using SENSIO 3D is decompressed on our display, it still shows 3D. There are some minor differences, but I don't think the average viewer would even notice them."
Side-by-side is not the only possibility when it comes to broadcasting and distributing 3D image streams. If there is no need to retain frame compatibility with existing MPEG-2 based broadcasting, there are a number of ways to transmit natural 3D images with no resolution degradation. One of these is the dual-stream approach (Fig. 5).
In this technology, the left and right images are divided over multiple channels for transmission, being reassembled and displayed by the TV. Korea would be the first to adopt the new technology. KCC plans to begin trial broadcasting in selected regions in October 2010, using MPEG-2 coded left imagery and H.264-coded right imagery in the 6MHz terrestrial broadcasting waveband. The idea is that this design will preserve compatibility with 2D broadcasting, essential for terrestrial broadcasting, while making 3D broadcasting possible. The use of H.264 (with its high coding efficiency) for the right image will mean an image with resolution far superior to the halved resolution provided by side-by-side technology compression.
The use of multiple streams was also the idea used in the H.264/Multiview Video Coding (MVC) video specification standardized by the ITU/International Electrotechnical Commission (IEC) MPEG in 2008. The concept there was not only for 3D imagery, but also sending multiple image streams simultaneously and allowing the TV to switch between them. The second stream would be sent as a differentiated data from the first stream instead of the whole right image, making it possible to cut the bandwidth by about 25%.
Standardization Vectoring Toward H.264
The industry is still in turmoil over which technique to use for 3D broadcasting, and standards yet unclear. Many national standards bodies and industry bodies like 3D@Home seem likely to standardize side-by-side. International standards bodies like the ITU, which have significant effect in the broadcasting field, have pretty much quietly dropped side-by-side, and are working on new standards based on H.264. The H.264 encoding technology is being adopted for 2D broadcasting across the world as a replacement for MPEG-2. It also offers a technical advantage that neither MPEG-2 nor side-by-side provide: the ability to broadcast natural 3D imagery at high resolution. This is the primary reason why so many people are interested in making H.264 the basis for 3D broadcasting standards.
In Japan both SKY Perfect JSAT and Actvilla use H.264 in high-definition (HD) image stream distribution. SKY Perfect JSAT still uses side-by-side even with H.264 because it wanted to continue using its current set of tuners, but Actvilla has not disclosed details of the left and right image distribution.
A new 3D broadcasting specification is under development, using an enhanced version of H.264/MVC. The major point of the new specification is resolving the change in perceived depth due to a change in screen size. This will be handled by first transmitting the complete 3D imagery dataset to the received before splitting off the left and right images, and allowing the set itself to select the best value of the parallax. The major technical problem is how to transit as many viewpoints as possible in a limited bandwidth.
3D imagery will continue to evolve in the years to come, and as the technology continues to change the road to an industry-wide standard for 3D broadcasting looks to be a rocky one.
By Tetsuo Nozawa, Nikkei Electronics Asia
