3-D TV: The EE Times Guide to the Shifting Landscape

Systems makers will ship a new generation of 3-D enabled Blu-ray drives in tandem with the new TVs this year. They will sport their own format and the latest 1.4 version of the HDMI interconnect built with the bandwidth and signaling capabilities to handle 3-D content in full high def. Other content and broadcasting companies including British Sky Broadcasting, DirecTv, Discovery Channel and ESPN said they will turn on dedicated 3-D services in 2010.

"This is a living, breathing science project but we are comfortable going ahead," said Chuck Pagano, executive vice president of technology at ESPN, which plans to launch multiple stereo 3-D channels starting in June with its broadcast of the World Cup.

The broadcasters, however, bring along one of the many technical wrinkles ahead. They have yet to release details of what formats they will use, and they are expected to adopt differing approaches. All the broadcast signals are expected to be "frame compatible" with today's content coming into cable TV plants and set-top boxes. However each will require conversion. Chip and system makers say they can handle that job with new firmware in existing silicon. However they will need three months and sample video to write and test their components before systems will be ready to ship.

Broadcasters' networks lack the bandwidth to support a full 1,080-progressive image for both the left and right eye, so they cannot deliver the full high definition video seen on today's top TVs. They have been testing a variety of formats to pack two images into one frame to see which gives the best results on their networks. Those formats include multiple ways of putting two images side-by-side, over and under each other, using a checkerboard configuration or interleaving lines or columns.

"You can create so many permutations, it can be a mess," said David Broberg, the vice president of consumer video technology at CableLabs. "We have reduced it down to a preference for a single over/under format, are working with TV makers and content providers to settle on that and so far results have been positive," he added.

The over/under format is said to be royalty free. But satellite providers DirecTv and BskyB expect to use a side-by-side format.

RealD, which provides 3-D technology for the lion's share of theaters, says it has a fundamental patent on side-by-side. RealD announced partnership deals at CES with virtually all the top TV makers that involved licensing its technology on formats, glasses and other aspects of 3-D. RealD's already announced 3-D format licensees include: Samsung, Panasonic, Sony, Toshiba, JVC and DirecTv.

"A number of top consumer companies have done due diligence on us and determined we have a patent on side-by-side," said Josh Greer, president of RealD.

"This is a battle for intellectual property," added Rick Doherty, principal of market watcher Envisioneering Group (Seaford, N.Y.).

No doubt IP battles are raging all up and down the chain of 3-D TV technologies. To date the field lacks a patent pool, a weak spot in the business case for 3-D TV.

"The patent trolls will come out, they always do with a new technology," said David Naranjo, director of product development for Mitsubishi Digital Electronics America.

Beyond the obvious technology licensing issue, momentum is just what every 3-D technology supplier is looking for today. RealD, in terms of market perception, might have just accomplished that goal at this year's CES.

RealD believes it has built enough momentum to sweep the TV market with its technology for 3-D broadcasting. RealD's Koji Hase, president of worldwide consumer electronics said, "The industry can now see a thread -- content, delivery and display -- all using our 3-D technology."

While standardization for 3-D broadcast formats over satellite, cable and terrestrial TV is far from set, RealD's Hase said, "This won't be a decision by a committee. Three-D will be a de facto standard that will be embraced by industry groups."

Advantages of RealD's 3-D technology are that it is "display agnostic," meaning that it works with any display type. Moreover, it works either with polarized glasses or active shutter glasses, according to Hase. Most important, he added, "It's implementable in existing infrastructure."

The Society of Motion Picture and Television Engineers (SMPTE) started an effort in June to set a standard for 3-D distribution formats. However, given the competitive issues, some say de facto standards will be set by the marketplace.

Whatever format emerges, many observers said consumers may not even notice the resolution downgrade in the 3-D broadcast content. Silicon and systems engineers said they may be able to use resolution enhancement techniques to help make up for the shortfall.

Ultimately broadcasters expect to expand their bandwidth and move to new codecs such as the MPEG-4 Multiview Video Coding (MVC) widely seen as the optimal solution for stereo 3-D. But such a migration will take several years and cost billions.

It would take twice the bandwidth of today's satellite set-top boxes to support 60 frames/second of 1,080p content, said Brian Lenz, director of product design at BskyB. "You will not see that in next few years," he said.

Broberg of CableLabs said many cable TV set tops now support 1080p at 24 frames per second. Within two years many could be upgraded to 60 frames per second.

Terrestrial broadcasters have the biggest hurdles because they have even less bandwidth to work with than satellite and cable companies. However ATSC, which manages the U.S. terrestrial standard, has given 3-D support a back seat as it tries to get its mobile broadcasting technology off the ground in the next two years.

"Optical media has the best scenario, then its cable and satellite and probably last is terrestrial because the challenges just get harder for them," said Wendy Aylsworth, vice president of engineering for SMPTE and a chief technologist for Warner Brothers studios.

Getting Fitted for Glasses
Due in part to the multiple formats for optical disks and broadcasts, the first sets may require users to select what kind of 3-D content they want to watch.

"Someday sets will automatically detect stereo 3-D formats, but for today users still have to choose the right one from an on-screen menu," said Phil Lelyveld, a 3-D program manager at the Entertainment Technology Center, a branch of the University of Southern California's School of Cinematic Arts.

"That's not something you want in the consumer marketplace," he said, adding that USC is showing TV makers technology it has developed to automate the process.

Likewise, CableLabs has found a way to take the 3-D control codes from HDMI 1.4 and make them available to the many HDMI 1.2 and 1.3 version set tops now in consumers' homes. The group also helped convince the HDMI Licensing LLC to relax its specs, in part to allow for the workaround.

CableLabs is calling for TV makers to take part in interoperability tests in its 3-D labs to make sure the first systems work smoothly. "HDMI is very powerful but it has so many options that if people don't implement them in the same way it won't work well," Broberg said.

Like the broadcasters, TV makers are adopting different display and glasses technologies to show stereo 3-D. Thus, for example, glasses used on a Toshiba set may not work for a Samsung TV. Vendors such as Panasonic will brand their glasses for the first-generation products to avoid confusion and flaunt what they claim is proprietary technology.

All the top vendors at CES showed prototype systems using active shutter glasses which are said to be the best solution for rendering a full 1,080p signal for both eyes. The cheaper passive polarized glasses used in theaters only deliver 540 horizontal pixels per frame, although some TV makers may use them for lower cost sets or for use in public places like sports bars.

"We have to deliver [source video] to both types of glasses, and we want to treat each equally and fairly," said Broberg of CableLabs. "There are advantages to each," he said.

It's not clear what display technologies TV makers are using or where they get it. However the vast majority of active shutter glasses at CES carried the RealD logo.

Like it or not, users will need glasses of some kind to view high def content, at least for the next decade. Researchers say it could require displays with four times today's resolution to deliver without visible artifacts high def stereo 3-D that can be seen by the naked eye.

Content producers say the situation is even worse on their end. It will require at least eight lenses per camera, and perhaps dozens, to properly capture video that can be seen in 3-D without glasses.

Technicolor showed a demo of so-called auto-stereoscopic 3-D TV at CES using its interpolation algorithm. However, it had a relatively narrow viewing area of ten degrees so users saw blurring when they moved their heads.

"It could require 8K resolution screens to do it well, so this could be 10-15 years away," said Thierry Borel, a researcher at Technicolor working on the project.

The good news is, users in the CEA survey showed a tolerance for glasses. About a third said they found 3-D glasses annoying before the saw a 3-D movie, but afterwards only 20 percent said the glasses were annoying.

3-D Interfaces and Conversion
Standards groups are playing catch up with the industry on 3-D. The CEA has started an effort to define a standard for the infrared signaling to active shutter glasses. However it is not expected to be finished until after first products ship. Greer of RealD said his company is not participating in the effort.

For its part the Digital Video Broadcasting Project in Europe is just starting to explore standards for stereo 3-D TV. The first meeting of a DVB group to set market requirements for the technology will be held Jan. 26.

David Daniels, a senior technologist at BskyB, said a parallel group looking into technical requirements is also just getting set up. It is expected to explore several areas including formats for source video, signaling over HDMI and codecs such as MVC.

"We're still in the early days of understanding how H.264 works," he said, noting the BBC recently reported it has achieved a new low bit-rate capability with the MPEG-4.

The DVB will also take up the hot issue of how to show graphics and subtitles in stereo 3-D space.

"If you want to check out how a 3-D TV vendor handles 2-D graphics, just hit the menu button on one of their demos," said Hays of Sony. "Some people treat it very elegantly, and others make your head explode," he said.

Some companies see the lack of good 2- and 3-D interfaces as an opportunity. A representative of Motorola's set-top group said it has developed proprietary technology for the subtitle problem that it will supply to its customers. Stereo 3-D camera company 3ality Digital Systems (Burbank, Calif.) said it is working with a startup on 3-D navigation software.

Another hot technology is real-time 2- to 3-D conversion. Both Toshiba and Samsung promised to offer it as a differentiating feature on their TVs in the initial years when there will be limited 3-D content available. Both companies claim they have unique algorithms running on proprietary muscular processors they are putting inside their systems.

Toshiba, which is using its version of the eight-core Cell CPU co-developed with IBM, cautioned that the conversion will offer a limited version of stereo 3-D. He described it as providing some depth behind the screen where possible, but no effects of depth in front of the screen.

Studio stereographers and competing TV makers panned the techniques.

"We considered 2-D to 3-D conversion, but once you watch real 3-D content it's easy to see where the converted video breaks up, so we decided to stick with content produced in 3-D," said Nandhakumar of LG.

"It's a very, very difficult thing to do, and I've never seen a good version of it," said Phil McNally at Dreamworks, also known as Captain 3-D. "It's a question of whether there is any information available for a computer to understand what's in front or behind in a scene," he added.

Steve Schklair, chief executive of 3ality which is carving out a business in stereo 3-D capture, said the technique was tried in a football game in Dallas where the converted video was put up on a stadium monitor. "It was booed off the screen in three minutes, and when they turned it off the audience applauded," he said.

"It does require the hands of an artist to do this," said Hays of Sony. "I know everyone is trying to jump on the 3-D bandwagon, but some people will fall off," he said.

Unknowns of 3-D Art, Science
Studio and broadcasters have their own artistic challenges with stereo 3-D.

"When you open your eyes in the morning you see in 3-D, but in our world cuts, dissolves and close ups don't exist so we need a language to handle them," said Hays, noting directors including Steven Spielberg and Peter Jackson are learning stereo 3-D techniques.

"Sony started this center I work at so they could work out how to use those tools," Hays said. "There are not a lot of knowledgeable people out there in stereo 3-D, and we want to change that soon," he added.

Habib Zargarpour, a senior art director at Electronic Arts, said he is working on a stereo 3-D version of the Need for Speed game. Chip maker Nvidia showed at a Siggraph conference three years ago an automatically converted version of the game but it exposed sometimes embarrassing flaws in its 3-D logic, he said.

"We do a lot of cheats, and all of a sudden there we were caught with our pants down," he said.

Ted Kenney, a stereographer at 3ality said stereo 3-D content needs to be produced in a much more simple fashion that today's sports shows.

"There are 27 cameras for a routine NFL game to create energy, but 3-D lets the energy happen in a frame, so I think we need to slow down the cuts," Kenney said. "Living with one 3-D camera for 60 seconds you get more information out of it, so I've talked about shooting a game from one seat for an entire quarter," he said.

"There are just too many graphics and too many shots used on Fox and ESPN," he added.

"There is a whole new generation of cinematographers coming up, learning about stereography from scratch," said Lelyveld of ETC. "Someday this will be a field just like cinematography is today," he said.

Lelyveld also hopes to conduct research into the public health aspects of viewing stereo 3-D content to address concerns about its impact on eyesight.

"This is just a subject of conversation at conferences today and it's all anecdotal evidence, there is no data," he said. "We believe four percent of the public cannot perceive stereo 3-D, and we have found many bloggers who were critical of 3-D turned out to be unable to see stereo effects," he added.

It will take until next year's CES to assess whether the industry's new 3-D TV strategy is working. What was clear this year is the industry has embarked down the 3-D path, hopeful in the face of numerous challenges for the builders of chips, systems, networks and content.

By Rick Merritt and Junko Yoshida, EE Times