The Episolar Constraint: Monocular Shape from Shadow Correspondence
Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from webcams "in the wild."
A. Abrams, K. Miskell, and R. Pless. The Episolar Constraint: Monocular Shape from Shadow Correspondence. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, June 2013.Download the .pdf
In this paper, we treat shadows as a strong geometric cue: if a pixel is under shadow, then it must be the case that some other object along the lighting direction is casting a shadow onto it. For outdoor imagery, a geolocated camera and accurate timestamps cause this colinearity to have a known georeferenced direction. If the camera also has known geometric calibration, we can express this property as a linear constraint over the depth of each pixel involved. From this geometry, we derive three novel results:
- An image-space constraint between a shadow and its occluder,
- An approach to geometrically calibrate a camera from shadow correspondences, and
- A convex optimization to solve for the unknown depths for a sparse set of pixels from shadow correspondence.
This work was partially supported under NSF grants DEB1053554, IIS1111398, and EF1065734.