Heliometric Stereo: Shape from Sun Position
In this work, we present a method to uncover shape from webcams "in the wild." We present a variant of photometric stereo which uses the sun as a distant light source, so that lighting direction can be computed from known GPS and timestamps. We propose an iterative, non-linear optimization process that optimizes the error in reproducing all images from an extended time-lapse with an image formation model that accounts for ambient lighting, shadows, changing light color, dense surface normal maps, radiometric calibration, and exposure. Unlike many approaches to uncalibrated outdoor image analysis, this procedure is automatic, and we report quantitative results by comparing extracted surface normals to Google Earth 3D models. We evaluate this procedure on data from a varied set of scenes and emphasize the advantages of including imagery from many months.
A. Abrams, C. Hawley, and R. Pless. Heliometric Stereo: Shape From Sun Position. In Proc. European Conference on Computer Vision, October 2012.Download the .pdf
This paper presents an approach to heliometric stereo - using the sun as a moving light source to recover surface normals of objects in an outdoor scene. This is a classic application of photometric stereo because the position of the sun is known very accurately, but made challenging because of variations in lighting and weather. Additionally, most long term imagery is captured by webcams that may not share geometric or radiometric calibration information. Thus, we explore what it would take to fully automate the solution to the photometric stereo problem for uncalibrated, outdoor cameras.
There are three major contributions of this work. First, we adapt the photometric stereo algorithm to work for outdoor scenes by integrating a richer image formation model. We present a gradient descent approach and methods for initialization and regularization of this optimization.
Second, we test this across a variety of types and scales of natural scenes and highlight the ability to capture very small scale surface structure. We report surface normals geo-referenced to an "East-North-Up" coordinate system, and we are the first to offer quantitative comparisons between our results and 3D geometry from Google Earth. These highlight both the accuracy of our results and limitations in the completeness and resolution of the Google Earth geometry.
Third, we characterize performance under which this approach gives good results. This emphasizes the importance of using imagery from many months, because over the course of one day or one week, the solar path does not give sufficient constraints to recover surface normals.
Scripts to download the data used in this work
Ground truth depth maps used in this work, from Google Earth models .
Cameras were geometrically calibrated through ProjectLive3D.
Partial support from: NSF-EF1065734, NSF-IIS1111398, and NSF-DEB1053554