Noise-Resistant Fitting for Spherical Harmonics Ping-Man Lam , Chi-Sing Leung , and Tien-Tsin Wong in IEEE Transactions on Visualization and Computer Graphics, Vol. 12, No. 2, March-April 2006, pp. 254-265.

Abstract

Spherical harmonic (SH) basis functions have been widely used for representing spherical functions in modeling various illumination properties. They can compactly represent low-frequency spherical functions. However, when the unconstrained least square method is used for estimating the SH coefficients of a hemispherical function, the magnitude of these SH coefficients could be very large. Hence, the rendering result is very sensitive to quantization noise (introduced by modern texture compression like S3TC, IEEE half float data type on GPU, or other lossy compression methods) in these SH coefficients. Our experiments show that as the precision of SH coefficients is reduced, the rendered images may exhibit annoying visual artifacts. To reduce the noise sensitivity of the SH coefficients, this paper first discusses how the magnitude of SH coefficients affects the rendering result when there is quantization noise. Then, two fast fitting methods for estimating the noise-resistant SH coefficients are proposed. They can effectively control the magnitude of the estimated SH coefficients, and hence suppress the rendering artifacts. Both statistical and visual results confirm our theory.

Download Paper

• Acrobat: shfit.pdf (5.9 MB)

Experiments

Due to the page limit, we are not able to include all experimental results in the paper. For completeness, all meaningful results are presented in the following tables. There are 3 test data: "ring", "torus", and "dama".

"ring"

The test data "ring" is an example of image-based relighting. Multiple images of the scene is rendered, from the same viewpoint, but illuminated by a directional light source from different directions. Since only those lighting directions above the table are meaningful, the apparent BRDF at each pixel is reduced to a hemispherical function of radiance values.

Each of these apparent BRDF is converted to SH coefficients using ULS (unconstrained least square method), Sloan ULS (ULS method with the third SH basis function removed before estimation), CLS (the proposed contrained least square method), and CEB (the proposed constrained eigen basis method). The SH coefficients are further compressed to test their noise-resistances. Half float representation (16 bits per coeff.), S3TC-like compression (4.667 bits per coeff.), and wavelet-based compression (1.33 bits and 0.33 bit per coeff.) are used.

In the following table, we show the visual results and the PSNRs. From left to right, we decrease the bits to represent the SH coefficients, or in other words, we increase the quantization noise. As expected, ULS (row 1) cannot effectively resists the noise as the noise increases. Sloan ULS (row 2) is better but still exhibits rendering artifacts. The proposed CLS (row 3) and CEB (row 4) effectively resist the noise and suppress the rendering artifacts.

Click the images to see the full-size images.

	32 bits (original)	16 bits (half float)	4.667 bits (S3TC-like)	1.33 bits (wavelet)	0.33 bit (wavelet)
ULS		41.1 dB	15.4 dB	9.7 dB	8.2 dB
Sloan ULS		44.3 dB	18.1 dB	20.6 dB	17.1 dB
Our CLS		62.8 dB	35.6 dB	38.5 dB	34.8 dB
Our CEB		63.1 dB	37.2 dB	39.2 dB	35.3 dB

Remark:

• In computing the PSNR values, the images from the 32-bit SH coefficients estimated by the corresponding methods are referred as the control images. That is, the PSNR of image from 1.33 bits SH by CLS is computed by comparing it with the image from 32 bits SH by CLS, not other estimation methods. Similarly, the PSNR of image from 1.33 bits SH by ULS is computed by comparing it with the image from 32 bits SH by ULS. Our rationale is to measure the degradation of image quality as the noise is introduced. This setting does not harm the fairness of comparison as all estimation methods produce similar image qualities when the estimated SH coefficients are represented as 32-bit floating point. This is evidenced by the similarity of images in the first column of the table.

Example "torus" is an opaque 3D object with diffuse surface and self-shadowing being accounted. Interreflection is ignored. At each vertex, we record the reflectance (collapsed with self-shadow). Again it is a hemispherical function, defined in the local frame. The following table shows the results with two different illuminants, distant environment (columns 1-3) and directional light source (columns 4-6). The SH coefficients are further compressed with uniform quantization (8 bits per coeff.) and wavelet-based compression (4 bits per coeff.).

ULS (row 1) gives the worst result. Sloan ULS (row 2) gives much better result, but visual artifact is still observable, especially when illuminated by a directional light source (columns 4-6). Our CLS (row 3) and CEB (row 4) give no observable visual artifact.

	32 bits (original)	8 bits (uniform quant.)	4 bits (wavelet)		32 bits (original)	8 bits (uniform quant.)	4 bits (wavelet)
ULS		21.5 dB	23.8 dB			22.1 dB	24.0 dB
Sloan ULS		44.6 dB	43.9 dB			36.2 dB	36.6 dB
Our CLS		55.0 dB	52.5 dB			46.0 dB	45.3 dB
Our CEB		56.6 dB	53.6 dB			45.3 dB	46.1 dB

Remark:

• All PSNR values in the above table only account for pixels occupied by the foreground torus. All background pixels are excluded as they must be always the same.
• In computing the PSNR values, the images from the 32-bit SH coefficients estimated by the corresponding methods are referred as the control images. That is, the PSNR of image from 1.33 bits SH by CLS is computed by comparing it with the image from 32 bits SH by CLS, not other estimation methods. Similarly, the PSNR of image from 1.33 bits SH by ULS is computed by comparing it with the image from 32 bits SH by ULS. Our rationale is to measure the degradation of image quality as the noise is introduced. This setting does not harm the fairness of comparison as all estimation methods produce similar image qualities when the estimated SH coefficients are represented as 32-bit floating point. This is evidenced by the similarity of images in the 1st and 4th columns of the table.

Example "dama" is a 3D object with specular surface and self-shadowing. Interreflection is again ignored. The lighting configuration is fixed. At each vertex, we recorded the reflected radiances as viewed from various viewing directions. Only the upper hemisphere is meaningful. The reflected radiances form a hemispherical function defined in a local frame. Again the proposed CLS and CEB effectively resist the quantization noise, as evidenced by the high PSNR values.

	32 bits (original)	16 bits (half float)	8 bits (uniform quant.)
ULS		41.0 dB	18.3 dB
Our CLS		66.7 dB	55.2 dB
Our CEB		68.4 dB	56.6 dB

Remark:

• All PSNR values in the above table only account for pixels occupied by the foreground dama. All background pixels are excluded as they must be always the same.
• In computing the PSNR values, the images from the 32-bit SH coefficients estimated by the corresponding methods are referred as the control images. That is, the PSNR of image from 1.33 bits SH by CLS is computed by comparing it with the image from 32 bits SH by CLS, not other estimation methods. Similarly, the PSNR of image from 1.33 bits SH by ULS is computed by comparing it with the image from 32 bits SH by ULS. Our rationale is to measure the degradation of image quality as the noise is introduced. This setting does not harm the fairness of comparison as all estimation methods produce similar image qualities when the estimated SH coefficients are represented as 32-bit floating point. This is evidenced by the similarity of images in the first column of the table.