Making Assumptions -- In 3D

More often than not, making assumptions gets you in trouble. But for Stanford University doctoral student Ashutosh Saxena and computer science professor Andrew Ng, making assumptions is at the heart of their technique for extracting 3D models from 2D images.

The traditional approach to synthesizing 3D models is to analyze multiple images of a scene. However, the approach Ng and Saxena take is to infer depth from single images by combining assumptions about what must be ground or sky with simple cues such as vertical lines in the image that represent walls or trees. And they've implemented this technique in an interactive website called Make3d, which lets you upload your 2D images and get back 3D results.

To "teach" the algorithm about depth, orientation, and position in 2D images, Ng and Saxena used images of 2D scenes along with 3D data of the same scenes gathered with laser scanners. The algorithm correlates the two sets together, identifying trends and patterns associated with being near or far. To make these judgments, the algorithm divides images into superpixels. By looking at a superpixel in context with its neighbors, the algorithm makes assumptions about how far it is from the viewer and what its orientation in space is.

Along with Min Sun, Ng and Saxena have written a really interesting article entitled Learning 3-D Scene Structure from a Single Still Image that goes into this in detail.

-- Jonathan Erickson
jerickson@ddj.com

Posted by Jon Erickson at 01:47 PM  Permalink