Temporally Sparse, but Spatially Dense Scenes

One nice aspect of temporally modulating the PSF is that such an approach can handle a wider range of sparse scenes when compared to previous approaches based on spatial-multiplexing optics. In particular, this methodology is better able to reconstruct temporally sparse, but spatially dense scenes. This is because our methodology uses the entire sensor for the signal at each timestamp, while previous approaches only use a subset, giving our approach an edge in the amount of information that is captured. This allows our approach to handle strobed scenes.

A rotating fan strobed twice during the exposure. Our approach can resolve each individual strobe.

λsparsity=0, λdxdy=1e-2, λdt=1e-2
θstart=0, θend=90
Reference frame
Captured frame with rotating PSF
Reconstructed video

Simulated comparison with previous approaches.

Rotating PSF capture
Rolling shutter + diffuser capture (1 row exposure)
ROI + grating capture (2 row ROI)
Rolling shutter + diffuser capture (64 row exposure)
ROI + grating capture (16 row ROI)

Observe that our method reconstructs the temporally sparse but spatially dense dancers significantly better than prior work. Even with significantly longer exposure times or larger bandwidth requirements, past approaches do not match the performance of a rotating PSF.

Ground truth
Rotating PSF reconstruction
Rolling shutter + diffuser reconstruction (1 row exposure)
Rolling shutter + diffuser reconstruction (64 row exposure)
ROI + grating reconstruction (2 row ROI)
ROI + grating reconstruction (16 row ROI)

Parameters

Rotating PSF:
λsparsity=1e-6, λdxdy=4e-6, λdt=1e-6
Rolling shutter + diffuser (1 row exposure):
λsparsity=0, λdxdy=1e-5, λdt=1e-5
Rolling shutter + diffuser (64 row exposure):
λsparsity=0, λdxdy=5e-7, λdt=1e-8
ROI + grating (2 row ROI):
λsparsity=0, λdxdy=1e-3, λdt=0
ROI + grating (16 row ROI):
λsparsity=1e-4, λdxdy=1e-4, λdt=0