Passive micron-scale time-of-flight with sunlight interferometry

Alankar Kotwal, Anat Levin, Ioannis Gkioulekas

CVPR 2023 (highlight)

Four panels. A photograph of the interferometer built on a wheeled optical table outdoors on grass. A 3D schematic labelling the sunlight beam, tracker, splitter, camera, mirror and scene. A photograph of a Raspberry Pi circuit board with a red inset magnifying its surface-mount components and tracks at a 1 mm scale bar. And the reconstruction: a colour-coded depth map beside two shaded 3D renderings of the recovered surface, with green insets on the fine detail.
Using sunlight interferometry to passively reconstruct part of a circuit board. (a) and (b) show a schematic and photograph of the sunlight interferometer we build for passive time-of-flight imaging. We use this system to reconstruct part of a Raspberry Pi circuit board that has multiple resistors, soldering pads, and tracks. (c) and (d) show a picture of the scene as seen through the imaging camera, along with an inset highlighting fine geometric features. (e) shows the estimated depth map, and (f) and (g) the corresponding rendered 3D surface. Our technique reconstructs fine features such as the PCB tracks and through-holes, despite operating outdoors under adverse environment conditions.

Abstract

We introduce an interferometric technique for passive time-of-flight imaging and depth sensing at micrometer axial resolutions. Our technique uses a full-field Michelson interferometer, modified to use sunlight as the only light source. The large spectral bandwidth of sunlight makes it possible to acquire micrometer-resolution time-resolved scene responses, through a simple axial scanning operation. Additionally, the angular bandwidth of sunlight makes it possible to capture time-of-flight measurements insensitive to indirect illumination effects, such as interreflections and subsurface scattering. We build an experimental prototype that we operate outdoors, under direct sunlight, and in adverse environment conditions such as machine vibrations and vehicle traffic. We use this prototype to demonstrate, for the first time, passive imaging capabilities such as micrometer-scale depth sensing robust to indirect illumination, direct-only imaging, and imaging through diffusers.

Hardware implementation

Left: four grayscale frames from the tracking camera, each showing the Sun as a bright disc at a different position. (a) Setup schematic: sunlight from a distant source reflects off a steerable tracking mirror and passes through two beamsplitters that feed a tracking camera and a main camera, with one arm going to a reference mirror and the other to the scene; an inset decomposes the main camera's measurement into scene, reference and interference fringes. (b) Photograph of the physical prototype on a wheeled optical table, with dashed coloured boxes keyed to the schematic's tracking assembly, main camera and reference mirror.
We build an experimental interferometer prototype that uses sunlight as its only light source. Our prototype uses a regular camera for imaging, and a Sun tracking assembly comprising a steerable mirror and a tracking camera. As sunlight is temporally incoherent, we can perform time of flight imaging by scanning a reference mirror and measuring interference at each scan location.

Data pipeline

Our method takes as input a video of a scene under sunlight that includes minute speckle changes during scanning. After some simple processing (basic filtering operations), our method produces as output a direct-only transient video and a depth map of the scene.

input video
direct-only transient video
Colour-coded depth map of the R and F region of the coin, shading from blue at the nearer left to yellow at the farther right, with the raised letters and their frames clearly resolved.
depth

Video

Passive direct-only transient imaging

We show transient videos of various scenes in outdoor environments, which we capture passively using only sunlight.

Grayscale close-up of a French coin, showing the engraved tree emblem, the letters R and F, and part of the surrounding LIBERTE EGALITE FRATERNITE lettering.
scene
direct-only transient video
Grayscale close-up of a round pill resting on a dark surface, its face covered in fine bright speckle and faint embossed markings.
scene
direct-only transient video
Grayscale close-up of a curved solder trace looping across a flat surface.
scene
direct-only transient video

Resources

 Paper: Our paper and supplement are available on CVF open access, on arXiv, and locally (paper, supplement).

 Poster: Our poster is available here.

 Presentation: Our presentation slides are available here.

 Data and code: The data to reproduce our experiments, alongside processing code, is available here. The paper supplement includes more details about the code.

Citation

@InProceedings{Kotwal:2023:Passive,
	author    = {Kotwal, Alankar and Levin, Anat and Gkioulekas, Ioannis},
	title     = {Passive Micron-Scale Time-of-Flight With Sunlight Interferometry},
	booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
	month     = {June},
	year      = {2023},
	pages     = {4139-4149}
}

Acknowledgments

We thank Sudershan Boovaraghavan and Yuvraj Agrawal, who provided the samples for some of our experiments. This work was supported by NSF awards 1730147, 2047341, 2008123 (NSF-BSF 2019758), and a Sloan Research Fellowship for Ioannis Gkioulekas.