Super-realistic 3D scans for AR/VR using a smartphone camera with flash


Since the introduction of 3D capture technology, a scan’s fidelity to the real world has been judged primarily the metric of accuracy. With the rise of AR/VR applications, however, other metrics are becoming increasingly important. Is the scan’s texture photorealistic? Does it react to light sources like a real object would in its place?

A recent conference paper presented at SIGGRAPH Asia demonstrates a method that captures accurate geometry, photorealistic detail, and even reflectance, using nothing more than a single camera with a flash—like the one you have in your smartphone.

Min H. Kim, lead author of a conference paper presented at SIGGRAPH Asia earlier this month explains that ”traditionally, this has been either done manually by 3D artists, which is a labor-intensive task, or by using specialized, expensive hardware. Our method is straightforward, cheaper and efficient, and reproduces realistic 3D objects by just taking photos from a single camera with a built-in flash.”

Many existing methods for mobile 3D capture of this type run into familiar problems. Some require existing geometry data as a basis, complicating the process of capture significantly, and others are only capable of capturing reflectance for planar objects. Multi-view stereo approaches tend to assume diffuse reflectance, which leads to smoothed out models. Often the only reliable approach is to use a professional capture system, which often costs $200,000 or more.

If you’re into optical engineering, I recommend digging into the paper here. If you’re not, the results below—captured with a single smartphone—speak for themselves.


About Author

SPAR 3D Editor Sean Higgins produces SPAR 3D's weekly newsletters for 3D-scanning professionals, and Sean has previously worked as a technical writer, a researcher, a freelance technology writer, and an editor for various arts publications. He has degrees from Hampshire College in Amherst, Massachusetts and the University of Aberdeen in Scotland, where he studied the history of sound-recording technologies. Sean is a native of Maine and lives in Portland.

Leave A Reply

© Diversified Communications. All rights reserved.