When I move the virtual viewpoint around, the rendered image suddenly becomes severely distorted. Why?
This is a known problem cause by our view-dependent geometry reconstruction algorithm. Our algorithm has the limitation that if the synthesized light rays are too far away from the input capturing cameras, the rendering result will be very bad. Similar effect was reported in the literature (see our 2004 Eurographics Symposium on Rendering paper for references). We are still working on this problem.