Adaptive Gating for Single-Photon 3D Imaging

The core of our method relies on a well established algorithm in the field of multi-armed bandit problems known as Thompson sampling. Thompson sampling bridges the gap between exploration and exploitation by obtaining estimates of hidden variable through sampling the posterior distribution. For our adaptive gating algorithm, we employ a pulse-by-pulse adaptive procedure. For every SPAD cycle we do the following: 1. we sample the current depth distribution to obtain a proxy for true depth, 2. we set the gate as the sampled depth, 3. we wait for a photon detection, and 4. update our running depth distribution. We then obtain the final depth estimate from the resulting depth distribution at the end of acquisition.

Adaptive Exposure

Through the adaptive gating framework, we can reduce total acquisition time by stopping SPAD acquisition when the depth distribution has already converged, we call this method adaptive exposure. Note that time for convergence greatly depends on background illumination of the scene point, meaning certain points in the scene may take longer integration times to obtain accurate estimates. Adaptive exposure assigns varying integration times for each scene point, making sure errors aren’t concentrated in certain parts of a scene and reducing total acquisition time while maintaining equal or better reconstruction accuracy.

Up-stream Compatibility

In our adaptive gating algorithm, without prior knowledge of scene depth, we must initialize the depth distribution with a uniform distribution. However, we are able to trivially incorporate depth priors from external sources by initializing the depth distribution based on these priors, further improving the performance of adaptive gating.

We validated the performance of our algorithm by capturing data from real outdoor scenes. In order to capture data from scenes under direct sunlight, where photon pile-up is most severe, we moved our gated SPAD setup onto a trolley. Below are some images of our setup.

Comparison in direct sunlight

Our method provides better depth reconstruction accuracy while capturing data under direct sunlight at noon. Alternatively, our method facilitates faster acquisition (cutting total integration time by 67%) while maintaining equal or better depth reconstruction accuracy.

Comparison in bright indoors setting

Our method still performs better than previous state-of-the-art gating strategies even under conditions with less severe pile-up.

Comparison using external depth priors

The probabilistic basis of our method allows adaptive gating to be up-stream compatible. This means that we can utilize depth priors from external sources to initialize our adaptive gating algorithm. We experimented with using a off-the-shelf neural monocular depth estimator as prior to adaptive gating. Results show that using appropriate priors leads to better reconstruction accuracy and lower integration times.

For an in-depth description of the technology behind this work, please refer to our paper, supplementary material, CVPR poster, and CVPR talk video.

Ryan Po, Adithya Pediredla, and Ioannis Gkioulekas. "Adaptive Gating for Single-Photon 3D Imaging", CVPR 2022

Code and data are available at the following GitHub repository.