3D Image Sensors - What Are They?

"When I first saw the announcement of 3D image sensor chips cross the newswire, I must admit my first reaction was a bit dismissive. Mistakenly, I assumed this was some kind of gimmick that paired two conventional image sensors a few inches apart to create stereoscopic images, something that has been readily doable with two separate, independent image sensors for years. "A more convenient package," I thought to myself. I couldn't have been more wrong.

What Canesta, the company behind these sensors, and Tower Semiconductor, the foundry have announced are true depth-perceiving image sensor chips that work on a totally different principle than ordinary image sensors and ordinary stereoscopic 3D. The 3D image is not created by stereo vision -- essentially two separate 2D images, the way our eyes work -- but by actually sensing the depth to various objects in the scene using a single image sensor. It's kind of like light wave based radar on a pixel-by-pixel basis."
By Cliff Roth, Video Imaging DesignLine

CanestaVision Chips
Most people understand that light takes a finite time to travel between two points -- that photons of light from two different stars, for example, may have started their journeys years, or even millennia apart. Since light travels essentially at a constant speed, if you know the time, you can calculate the distance.

The light illuminating each individual pixel in an image sensor comes from a different feature in the scene being viewed. Canesta recognized that if you could determine the amount of time that light takes to reach each pixel, you then could calculate with certainty the exact distance to that feature. In other words, you could develop a three-dimensional "relief" map of the surfaces in the scene. In three dimensions, objects previously indistinguishable from the background, for example, metaphorically "pop" out. For a broad class of applications, this proves extremely helpful in reducing the mathematical and physical complexity that has plagued computer vision applications from the start.


In a recently-granted U.S. patent, Canesta describes several of its inventions for "timing" the travel time of light to a unique, new class of low-cost sensor chips.

Fundamentally, the chips work in a manner similar to radar, where the distance to remote objects is calculated by measuring the time it takes an electronic burst of radio waves to make the round trip from a transmitting antenna to a reflective object (like a metal airplane) and back. In the case of these chips, however, a burst of unobtrusive light is transmitted instead.

The chips, which are not fooled by ambient light, either then time the duration it takes the pulse to reflect back to each pixel, using high speed, on-chip timers in one method, or simply count the number of returning photons, an indirect measure of the distance, in another.

In either case, the result is an array of "distances" that provides a mathematically accurate, dynamic "relief" map of the surfaces being imaged. The image and distance information is then handed off to an on-chip processor running Canesta's proprietary imaging software that further refines the 3-D representation before sending it off chip to the OEM application.

Source: Canesta