Coherence As Texture – Passive Textureless 3D Reconstruction by Self-interference

Wei-Yu Chen       Aswin C. Sankaranarayanan       Anat Levin       Matthew O'Toole      
Carnegie Mellon University, Technion

CVPR 2024 (Highlight)

Paper Supplementary Simulation Code Setup 3D Model

Setup illustration

Setup viewing a plane target

Conventional camera observation

Our depth map reconstruction under indoor white light

Sideview of reconstruction

Passive depth estimation based on stereo or defocus relies on the presence of the texture on an object to resolve its depth. Hence, recovering the depth of a textureless object---for example, a large white wall---is not just hard but perhaps even impossible.

Or is it?

We show that spatial coherence, a property of natural light sources, can be used to resolve the depth of a scene point even when it is textureless. Our approach relies on the idea that natural light scattered off a scene point is locally coherent with itself, while incoherent with the light scattered from other surface points; we use this insight to design an optical setup that uses self-interference as a texture feature for estimating depth. Our lab prototype is capable of resolving depths of textureless objects in sunlight as well as indoor lights.

Principle

Stereo imaging requires point correspondence across view to resolve disparity. However, textureless targets lack feature details and thus are challenging to find point correspondence between views.

However, under natural illumination, the waves from each scene point can only interfere with itself. Thus, it provides a unique cue to match point between views

With our optical setup, we can make only one line of a scene visible in interefence pattern at a time and thus can find point correspondence. Please refer to our paper for more details.

Results

Swept-angle laser illumination results

Lightsource type is a key factor in our method. Ideally, we want an monochromatic area light source. Thus, we starts from using a swept-angle laser illumination, and we can achieve precised 3D reconstruction results.


Natural illumination results

To demonstrate our method is valid for natural light source, we conduct experement under sunlight illumination and show result above. Our method can achieve reasonable reconstructions. Also, the plane result at the top of the website are reconstructed under a lamp.

Citations

@article{chen2024coherence,
      author = {Chen, Wei-Yu and Sankaranarayananan, Aswin C and Levin, Anat and O'Toole, Matthew},
      journal = {CVPR},
      number = {},
      pages = {},
      title = {Coherence As Texture – Passive Textureless 3D Reconstruction by Self-interference},
      year = {2024},
     }