Papers: Radiance Transfer Papers

By far the most important work going on at this SIGGRAPH was the continuing research into methods for representing radiance transfer. This unwieldy term refers to methods that capture the overall transfer of light energy onto, and then back off of, and object. At the last couple SIGGRAPH conferences, there was a great deal of excitement about using Spherical Harmonics as an approximation to incoming radiance and the resulting images. This year there was one session with three outstanding papers expanding on this topic in various ways. Clint Hanson from EAC will be doing a fuller writeup on this topic, but a brief description of the three papers is below.

None of these techniques are immediately implementable in our games, especially given the data storage requirements. However, it's important to realize that extensive precomputation for lighting is already a feature of most EA products; EAC, EAUK, and EALA already use a custom Mental Ray plug-in to do a Final Gather of photons to precompute the lighting on environments. While our range of performance targets for the PC and the current generation of the consoles won't permit direct use of what was shown this year, as we look at making our code run on the next generation consoles, and at the more level-oriented style of play found on the consoles, these are techniques we should consider.

Bi-scale Radiance Transfer

The Microsoft Graphics Research group presented a paper on sumamrizing radiance transfer as a combination of effects at two scales: a so-called "meso-scale" that captures the lighting effects of a texture, and a macro-scale that captures overall effects such as self-shadowing. The meso-scale effects are calculated by pre-processing the color and geometry (aka displacement/bump) information in the texture into an RTT, Radiance Transfer Texture. The macro-scale effects are calculated by precomputing the overall radiance transfer (PRT, for precomputed radiance transfer). When lighting the object at runtime, the incoming light for a point is passed through the PRT to get incoming illumination at a point; then the result is looked up through the RTT. Because the RTT is still direction-dependent for the incoming lights, the RTT's local lighting effects are strongly directional (see the basket-weave examples in the paper). This ability to get very small-scale lighting effects -- much smaller than the other papers permitted -- was the primary advance in this paper.

The resulting images rendered on an ATI 9700 at about 15fps. The directionality and expressiveness of the lighting (from environment lighting) was spectacular. Unfortunately, apart from the raw time, the precomputed data was far too large to consider this for immediate use on most objects. The Stanford bunny's precomputed data added up to about 40MB.

The full title of this paper was "Bi-scale Radiance Transfer" by Peter Pike-Sloan, Xinguo Liu, Heung-Yeung Shum, and John Snyder of Microsoft Research.

Wavelet Lighting Approximations

The last couple years, as I mentioned, there's been a lot of excitement about using spherical harmonic approximations to incoming light in order to render realistic lighting on objects. In this paper, they use a non-linear wavelet approximation instead, showing clearly better results in terms of reproducing shadow outlines, when using the same number of terms in either formulation. Unfortunately, like the previous and following papers, a fairly dense tessellation of surfaces is required in order to achieve the results; a single large flat poly for the floor is not satisfactory. Like the previous paper, they precompute the lighting response given a small set of look-up tables for possible incoming lighting. The precomputed data again ranges from 10-100MBytes per mid-poly-count object (e.g., Stanford bunny).

They then play back their precomputed lighting in real-time; for moving either of the viewpoint or the lighting, their frame rates were 6-10fps. Precomputation times for their moderate-complexity objects were several hours. The primary advantage of their technique is that the wavelet basis allowed higher-frequency response, which is mostly noticeable as much better-defined shadows than the spherical harmonic technique.

The full title of this paper was (breathe in) "All-Frequency Shadows Using Non-linear Wavelist Lighting Approximation" by Ren Ng, Ravi Ramamoorthi, and Pat Hanrahan from Stanford University.

Principal Component Analysis of Precomputed Radiance Transfer

A different take on the Spherical Harmonics lighting idea was presented by a Microsoft/University of Illinois group. They use a statistical processing idea called principal component analysis to find a much small set of precomputed radiance transfer component to render each vertex with. The result is primarily in framerate improvement: their images of the bird and the Buddha rendered in the range of 25-60fps. The primary result they showed is that they could get specular, fresnel, and even translucency effects precomputed via these methods without locking down either the camera or the lights. However, in order to get those effects, they were performing quite a bit of overdraw.

Again, Clint Hanson and some others have promised much longer writeups of the specifics of these algorithms -- or come borrow the papers. The full titles of this paper was "Clustered Principal Comonents for Precomputed Radiance Transfer" by Peter-Pike Sloan, Jesse Hall, John Hart, and John Snyder from Microsoft Research or the University of Illinois.