The Episolar Constraint: Monocular Shape from Shadow Correspondence

Austin Abrams, Kylia Miskell, and Robert Pless.


Abstract

Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams "in the wild."


Citation




In this paper, we exploit the inherent structure of cast shadows to recover shape from a single view. Given a time-lapse sequence from a geographically-calibrated camera, we create correspondences (shown as a yellow line) between a shadow (blue) and its occluding object (red). Repeated across the image and across many lighting directions, these tens of thousands of correspondences can be used as a cue to recover a sparse depth map from a single viewpoint. Depth increases from blue to red.

A. Abrams, K. Miskell, and R. Pless. The Episolar Constraint: Monocular Shape from Shadow Correspondence. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2013. [BibTeX]

Download the .pdf


Project

In this paper, we treat shadows as a strong geometric cue: if a pixel is under shadow, then it must be the case that some other object along the lighting direction is casting a shadow onto it. For outdoor imagery, a geolocated camera and accurate timestamps cause this colinearity to have a known georeferenced direction. If the camera also has known geometric calibration, we can express this property as a linear constraint over the depth of each pixel involved. From this geometry, we derive three novel results:

  • An image-space constraint between a shadow and its occluder,
  • An approach to geometrically calibrate a camera from shadow correspondences, and
  • A convex optimization to solve for the unknown depths for a sparse set of pixels from shadow correspondence.


Links

Paper
Code and Data (47MB)



Acknowledgements

This work was partially supported under NSF grants DEB1053554, IIS1111398, and EF1065734.