Portland, Ore. -- Remember how digital converters for audio started out at 8 bits, then went to 16 and 24 bits before resetting to 1 bit with oversampling? Engineers at Rice University will propose this week that we reset our megapixel cameras to 1 pixel and our video cameras to 1 voxel, both with oversampling.
They will make their case at the Optical Society of America's 90th annual meeting--Frontiers in Optics 2006--in Rochester, N.Y.
The 1-pixel camera takes tens of thousands of rapid-fire shots to capture the equivalent of 1 million pixels in an image. So instead of expensive megapixel sensors with separate detectors for red, green and blue, the Rice EEs' approach needs only a 1-pixel multispectral sensor, simplifying hardware resources while enabling images to be formed from spectra never before imaged. "There are all sorts of detectors used in the physics lab--now most of them can be used to make images too," said electrical and computer engineering professor Kevin Kelly. "Physics labs, for instance, can now make images from neutron detectors using our approach, and astronomers can make images from radio waves." He performed the work with Richard Baraniuk, the Victor E. Cameron professor of electrical and computer engineering.
The enabling chip for the 1-pixel camera is not the detector used to sense an application-specific spectrum--that could use any technology. Instead, Texas Instruments Inc.'s digital micromirror array is used to project light from the lens onto the sensor. The micromirror array is the same chip that's used in Digital Light Processor televisions. Here, the lens focuses light onto the 1,024 x 768-pixel digital micromirror chip, which in turn projects all of its light into a single photodiode.
"What we are essentially doing is running a Digital Light Processor backwards and replacing the light source with a photodiode," said Baraniuk.
You probably thought a 1-pixel camera would work like a flatbed scanner, moving the single sensor over the area ordinarily occupied by the film. In fact, the only moving part--the micromirror array--is not taking the place of a scanner for two reasons. First, it would be too slow to measure the light from each pixel location separately, requiring 1 million cycles per megapixel. And second, you would need an ultrasensitive photodiode to detect the light from just one micromirror at a time.
In contrast, the Rice algorithm relies on tens or hundreds of thousands of measurements, not millions. In each clock cycle, it also focuses all the light from the micromirror array onto the photodiode, so it doesn't have to be ultrasensitive. Each mirror in the array either focuses on the photodiode and thus is "on" or away from it and thus "off." The secret is that the micromirror array takes on random configurations each clock cycle. The Rice algorithm then measures the sum of the light from that set of coefficients in a mathematical decomposition of the original image--essentially deducing its most obvious features from a set of random measurements.
"The math is pretty dense, but the critical point is that instead of making millions of measurements at millions of pixel locations, we just take tens of thousands of measurements using a single sensor," said Baraniuk. "We are using some very new image reconstruction techniques to form an image from a series of random projections from the micromirror array."
Just as the hardware setup is the same as a projector--only backwards--so, too, the algorithm that deduces what the original image must have been is backwards, making its calculations from the random configurations of the micromirror.
A normal megapixel camera uses a sensor at every pixel location, taking millions of highly accurate measurements simultaneously. Later, the raw data is compressed into a series of coefficients for a hypothetical filter bank that could reconstruct the original image from white noise as an input. The Rice algorithm runs that backwards by starting with random noise as its input, then directly measuring a series of coefficients that enable reconstruction of the original image. "Instead of sampling millions of pixels in the light field and then compressing all that information . . . we can go directly from the light field to the compressed data," said Baraniuk.
Cameras everywhere
Today it's possible to make images only with sensors that can be built in arrays, but many wavelengths are impractical or too expensive. For instance, terahertz sensors could enable cameras that see through clothing at airport checkpoints, but terahertz sensors are impractical to build today. Likewise, sensors for low light need detectors that could use avalanche photodiodes, but again those are too expensive to build in arrays.
"With our approach, you could use the most expensive processes there are," said Kelly. "For instance, you could use an avalanche diode for a low-light camera that would be ultrasensitive, but impractical to build in big arrays. Today, the best photodetectors are built with indium gallium arsenide that you can't make in large arrays, but with our camera you can use the most exotic sensor you can find. For instance, for unrivaled color discrimination on a visible-light camera you could afford to get Foveon [Inc. in Santa Clara, Calif.] to make you a multispectral sensor with eight, 16 or even 24 layers of color sensitivity instead of just red, green and blue."
Next, EEs will try their method with sensors at different wavelengths, including infrared, terahertz and multispectral sensors such as those from Foveon.
Professor Dave Brady at Duke University is also working on a camera that uses the same 1-pixel approach but that is completely planar, because it requires no optical lens. Other new designs, the research- ers said, are constructing images from medical sensors such as computed tomography and magnetic resonance.
The theory behind the 1-pixel camera has been knocking around for about two decades, but crystallized in the past couple of years with the breakthrough work of cooperating scientists led by mathematician Emmanuel Candès, a professor at the California Institute of Technology, who received the National Science Foundation's Alan T. Waterman Award this year.
