As the overall light level in a scene decreases, photons arrive farther apart on average and become individually resolvable. As the light level decreases even further, pixels become “blind” to variations that unfold over progressively longer timespans. We analyse and show robust videography in this inter-photon-limited condition, where scene variations may outpace photon arrivals

Inter-photon flux parameterization redefines a scene's time-varying flux in units of periods per photon. The inter-photon flux is independent of any absolute timescale or illumination level and shows the equivalence of the signals that would have parameter estimates with identical statistical uncertainty.

The inter-photon spectra, the Fourier decomposition of the inter-photon flux, also has several ramifications within and beyond single-photon camera modeling. Firstly, it exposes imaging constraints not captured by conventional flux representations.

The inter-photon spectra also provides a timescale-invariant basis for performance assessment—replacing absolute metrics such as illuminance (lux) or frame rate with an inter-photon frequency response and reveals a unified imaging landscape spanning more than fourteen orders of magnitude in inter-photon frequency, encompassing both passive and active imaging systems. We find that the centroid \((f_p)\) of the inter-photon spectra provides a good estimate of intrinstic signal reconstruction difficulty.

We address the problem of inter-photon-limited videography by training a Neural Flux Field (NFF)—a neural network that estimates time integrals of the flux function from photon detection data across a sensor. NFFs \(\Phi_\theta\) takes a temporal coordinate as input, passes it through a temporal frequency encoding function \(\gamma\). This encoding is processed by a multi-layer perceptron, whose output is shaped into a spatial tensor and processed by convolutional layers. Finally, the resulting predicted video frame \(\hat{v}(\mathbf{x})\) is compared to the measurement frame \(\tilde{v}(\mathbf{x})\), and we optimize \(\Phi_\theta\) to maximize the likelihood of the photon measurements.

Below, an elevator scene is played back at 225fps (top) and 98.4kfps (bottom) along with the original corresponding binary frames captured by the SPAD512 single-photon camera. We show two playback speeds to illustrate the range of inter-photon frequencies captured: from the fast flickering elevator lights to the large motion of the jumping person to the slow moving elevator doors. Each frame has an exposure of 10us. We then simulated more difficult conditions with binomial thinning up to a factor of 1024.

We show binary frames captured by the SPAD512 array as well as video reconstructions of the optical flux. The scene is made more challenging via binomial thinning increasing \((f_p)\). NFFs retains more detail and better temporal consistency deeper in the inter-photon-limited regime.

We compare our method on diverse simulated scenes. Ground truth flux is obtained by temporally interpolating high-speed video sequences and reducing the DC level (and thus the inter-photon spectra) to match the low-light regime where single-photon imaging is most effective.