Heliometric Stereo: Shape from Sun Position

Austin Abrams, Christopher Hawley, and Robert Pless.


Abstract

In this work, we present a method to uncover shape from webcams "in the wild." We present a variant of photometric stereo which uses the sun as a distant light source, so that lighting direction can be computed from known GPS and timestamps. We propose an iterative, non-linear optimization process that optimizes the error in reproducing all images from an extended time-lapse with an image formation model that accounts for ambient lighting, shadows, changing light color, dense surface normal maps, radiometric calibration, and exposure. Unlike many approaches to uncalibrated outdoor image analysis, this procedure is automatic, and we report quantitative results by comparing extracted surface normals to Google Earth 3D models. We evaluate this procedure on data from a varied set of scenes and emphasize the advantages of including imagery from many months.


Citation



In this work, we use a set of imagery (one example shown in the top left) to recover a variety of reflectance properties, including geo-centric surface normals (top right), albedo (middle right), and an estimate of the unknown radiometric calibration (bottom left), among other things.

A. Abrams, C. Hawley, and R. Pless. Heliometric Stereo: Shape From Sun Position. In Proc. European Conference on Computer Vision, October 2012. [BibTeX]

Download the .pdf


Project

This paper presents an approach to heliometric stereo - using the sun as a moving light source to recover surface normals of objects in an outdoor scene. This is a classic application of photometric stereo because the position of the sun is known very accurately, but made challenging because of variations in lighting and weather. Additionally, most long term imagery is captured by webcams that may not share geometric or radiometric calibration information. Thus, we explore what it would take to fully automate the solution to the photometric stereo problem for uncalibrated, outdoor cameras.


Using the geo-centric surface normals generated from our heliometric stereo algorithm, we can recover 3D shapes of objects in the scene. Notice that the shape of the shell is captured nicely, even from extreme viewpoints (the figure on the right is taken as a side-on view)

There are three major contributions of this work. First, we adapt the photometric stereo algorithm to work for outdoor scenes by integrating a richer image formation model. We present a gradient descent approach and methods for initialization and regularization of this optimization.

Second, we test this across a variety of types and scales of natural scenes and highlight the ability to capture very small scale surface structure. We report surface normals geo-referenced to an "East-North-Up" coordinate system, and we are the first to offer quantitative comparisons between our results and 3D geometry from Google Earth. These highlight both the accuracy of our results and limitations in the completeness and resolution of the Google Earth geometry.

Third, we characterize performance under which this approach gives good results. This emphasizes the importance of using imagery from many months, because over the course of one day or one week, the solar path does not give sufficient constraints to recover surface normals.



Links

Code
Scripts to download the data used in this work
Ground truth depth maps used in this work, from Google Earth models .
Cameras were geometrically calibrated through ProjectLive3D.



Acknowledgements

Partial support from: NSF-EF1065734, NSF-IIS1111398, and NSF-DEB1053554