Abstract Abstract—We prove that the set of all Lambertian reflectance functions the mapping from surface normals to intensities obtained with arbitrary distant light sources lies close to a 9D linear subspace. This implies that, in general, the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace, explaining prior empirical results. We also provide a simple analytic characterization of this linear space. We obtain these results by representing lighting using spherical harmonics and describing the effects of Lambertian materials as the analog of a convolution. These results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce nonnegative lighting functions.
|Published (Last):||22 May 2016|
|PDF File Size:||18.46 Mb|
|ePub File Size:||6.33 Mb|
|Price:||Free* [*Free Regsitration Required]|
This implies that, in general, the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace, explaining prior empirical results. We also provide a simple analytic characterization of this linear space.
We obtain these results by representing lighting using spherical harmonics and describing the e ects of Lambertian materials as the analog of a convolution. These results allow us to construct algorithms for object recognition based on linear methods as well as algorithms that use convex optimization to enforce non-negative lighting functions.
Finally, we show a simple way to enforce non-negative lighting when the images of an object lie near a 4D linear space. Even when lights are isotropic and distant, smooth Lambertian objects can produce in nite-dimensional sets of images Belhumeur and Kriegman 1]. But recent experimental work 7, 12, 30] has indicated that the set of images produced by an object under a wide range of lighting conditions lies near a low dimensional linear subspace in the space of all possible images.
This can be used to construct e cient recognition algorithms that handle lighting variations. In this paper we explain these empirical results analytically and use this understanding to produce new recognition algorithms. When light is isotropic and distant from an object, we can describe its intensity as a function of direction. Light, then, is a non-negative function on the surface of a sphere.
Our approach begins by representing these functions using spherical harmonics. This is analogous to Fourier analysis, but on the surface of the sphere. To model the way surfaces turn light into an image we look at re ectance as a function of the surface normal assuming unit albedo. This kernel acts as a low-pass lter with This suggests that in general the set of images of a convex, Lambertian object can be approximated accurately by a low dimensional linear space.
We further show how to analytically derive this subspace from an object model. This allows us to better understand several existing methods. For example, we show that the linear subspace methods of Shashua 25] and Moses 20] use a linear space spanned by the three rst order harmonics, but that they omit the signi cant DC component.
Also, it leads us to new methods of recognizing objects with unknown pose and lighting conditions. In particular, we discuss how the harmonic basis can be used in a linear-based object recognition algorithm, replacing bases derived by performing SVD on large collections of rendered images.
Furthermore, we show how we can enforce non-negative light by projecting this constraint to the space spanned by the harmonic basis. With this constraint recognition is expressed as a non-negative least-squares problem that can be solved using convex optimization. This leads to an algorithm for recognizing objects under varying pose and illumination that resembles Georghides et al.
The use of the harmonic basis, in this case, allows us to rapidly produce a representation to the images of an object in poses determined at runtime. Finally, we discuss the case in which a rst order approximation provides an adequate approximation to the images of an object. The set of images then lies near a 4D linear subspace.
In this case we can express the non-negative lighting constraint analytically. We use this expression to perform recognition in a particularly e cient way, without complex, iterative optimization techniques. It has been very popular in object recognition to represent the set of images that an object can produce using low dimensional linear subspaces of the space of all images. Ullman and Basri 28] analytically derive such a representation for sets of 3D points undergoing scaled orthographic projection.
Hayakawa 13] uses factorization to build 3D models using this linear representation. Koenderink and van Doorn 18] extend this to a 4D space by allowing the light to include a di use component. Researchers have collected large sets of images and performed PCA to build representations that capture within class variations 16, 27, 4] and variations due to pose and lighting 21, 12, 30]. Hallinan 12], Epstein et al. More recently, analytically derived, convex representations have been used by Belhumeur and Kriegman 1] to model attached shadows.
Georghides et al. Spherical harmonics have been used in graphics to e ciently represent the bidirectional re ection distribution function BRDF of di erent materials by, e.
Nimero et al. Dobashi et al. With this representation, after truncating high order components, the re ection process can be written as a linear transformation, and so the low order components of the lighting can be recovered by inverting the transformation. He used this analysis to explore ambiguities in lighting. We extend this work by deriving subspace results for the re ectance function, providing analytic descriptions of the basis images, and constructing new recognition algorithms that use this analysis while enforcing non-negative lighting.
Independent of and contemporaneous with our work, Ramamoorthi and Hanrahan 24] have described the e ect of Lambertian re ectance as a convolution. Also, preliminary comments on this topic can be found in Jacobs, Belhumeur and Basri 15]. In summary, the main contribution of our paper is to show how to analytically nd low dimensional linear subspaces that accurately approximate the set of images that an object can produce.
We can then carve out portions of these subspaces corresponding to non-negative lighting conditions, and use these descriptions for recognition. This relatively simple model has been analyzed and used e ectively in a number of vision applications. Our objective is to analyze properties of the illumination cone. For the analysis it will be useful to consider the set of re ectance functions obtained under di erent illumination conditions. A re ectance function is related to an image of a convex object illuminated by the same lighting con guration by the following mapping.
We will discuss the e ect of this mapping later on in this section. In this coordinate frame the poles are set at 0; 0; 1 , denotes the solid angle between p and 0; 0; 1 , and it varies with latitude, and varies with longitude. Since we assume that the sphere is illuminated by a distant and isotropic set of lights all points on the sphere see these lights coming from the same directions, and they are illuminated by identical lighting conditions.
Thus, we can express the light re ected by the sphere as a function r ; whose domain is the set of surface normals of the sphere. If light reaches a point from a multitude of directions then the light re ected by the point would be the sum of or in the continuous case the integral over the contribution for each direction.
Thus, the operation that produces r ; is the analog of a convolution on the sphere. The convolution is obtained by rotating k so that its center is aligned with the surface normal at p. This still leaves one degree of freedom in the rotation of the kernel unde ned, but since k is rotationally symmetric this ambiguity disappears. The surface spherical harmonics are a set of functions that form an orthonormal basis for the set of all functions on the surface of the sphere.
The spherical harmonics, written hnm x; y; z , then become polynomials of degree n in x; y; z. We do this primarily so that we can take advantage of the analog to the convolution theorem for surface harmonics. An immediate consequence of the Funk-Hecke theorem see, e. In the rest of this section we derive a representation of k as a harmonic series. We use this derivation to show that k is nearly a low-pass lter. Speci cally, almost all of the energy of k resides in the rst few harmonics.
This will allow us to show that the possible re ectances of a sphere all lie near a low dimensional linear subspace of the space of all functions de ned on the sphere. In Appendix A we derive a representation of k as a harmonic series.
The middle row shows the energy accumulated up to order n. The bottom row shows a lower bound on the quality of this approximation due to the non-negativity of the light. Relative energies are given in percents. The energy captured by every harmonic term is measured commonly by the square of its respective coe cient divided by the total squared energy of the transformed function.
It can be seen that the kernel is dominated by the rst three coe cients. Thus, a second order approximation already accounts for Figure 2 shows a 1D slice of the Lambertian kernel and its various approximations. This space is spanned by a small set of what we call harmonic re ectances. Note that harmonic lights generally are not positive everywhere, so they do not correspond to real, physical lighting conditions; they are abstractions. As is explained below every re ectance function 6 1.
A point source is a delta function. The re ectance of a sphere illuminated by a point source is obtained by a convolution of the delta function with the kernel, which results in the kernel itself. Due to the linearity of the convolution, if we approximate the re ectance due to this point source by a linear combination of the rst three zonal harmonics, r , r , and r , we account for Similarly, rst and fourth order approximations yield respectively If the sphere is illuminated by a single point source in a direction other than the z direction the re ectance obtained would be identical to the kernel, but shifted in phase.
Shifting the phase of a function distributes its energy between the harmonics of the same order n varying m , but the overall energy in each n is maintained. Consequently, a rst order approximation requires four harmonics. A second order approximation adds ve more harmonics yielding a 9D space. The third order harmonics are eliminated by the kernel, and so they do not need to be included.
Finally, a fourth order approximation adds nine more harmonics yielding an 18D space. We have seen that the energy captured by the rst few coe cients ki 1 i N directly indicates the accuracy of the approximation of the re ectance function when the light includes a single point source. Other light con gurations may lead to di erent accuracy.
Better approximations are obtained when the light includes enhanced di use components of low-frequency. Worse approximations are anticipated if the light includes mainly high frequency patterns. However, even if the light includes mostly high frequency patterns the accuracy of the approximation is still very high. This is a consequence of the non-negativity of light. A lower bound on the accuracy of the approximation for any light function can be derived as follows.
It is simple to show that for any non-negative function the amplitude of the DC component must be at least as high as the amplitude of any of the other components. One way to see this is by representing such a function as a non-negative sum of delta functions. In such a sum the amplitude of the DC component is the weighted sum of the amplitudes of all the DC components of the di erent delta functions. The amplitude of any other frequency may at most reach the same level, but often will be lower due to interference.
CA2347645A1 - Lambertian reflectance and linear subspaces - Google Patents
Step d is preferably repeated for each of a red, green, and blue color component for each three-dimensional model. The linear subspace is preferably either four-dimensional or nine-dimensional. Field of the Invention The present invention relates generally to computer vision and, more particularly, to image recognition and model reconstructions systems. Prior Art One of the most basic problems in vision is to understand how variability in lighting affects the images that an object can produce. Even when lights are isotropic and relatively far from an object, it has been shown that smooth Lambertian objects can produce infinite-dimensional sets of images. It has been very popular in object recognition to represent the set of images that an object can produce using low dimensional linear subspaces of the space of all images. There are those in the art who have analytically derived such a representation for sets of 3D points undergoing scaled orthographic projection.
Lambertian reflectance and linear subspaces
Abstract Abstract—We prove that the set of all Lambertian reflectance functions the mapping from surface normals to intensities obtained with arbitrary distant light sources lies close to a 9D linear subspace. This implies that, in general, the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace, explaining prior empirical results. We also provide a simple analytic characterization of this linear space. We obtain these results by representing lighting using spherical harmonics and describing the effects of Lambertian materials as the analog of a convolution. These results allow us to construct algorithms for objectrecognition based on linear methods as well as algorithmsthat use convex optimizationto enforce nonnegative lighting functions.