CSC 2530 Visual Modeling - Project #1 Proposal: Video Mosaics

Team Members

• George ElKoura (gelkoura@dgp.toronto.edu)
• Sam Hasinoff (hasinoff@cs.toronto.edu)

Paper To Be Implemented

• S. Peleg and J. Herman, Panoramic Mosaics by Manifold Projection, CVPR'97, June 1997, pp. 338-343.

Implementation Plan

Algorithms To Implement

• Laplacian/Gaussian pyramids for coarse-to-fine alignment and smooth splining between image seams
• Translation-only alignment of images using the Lucas-Kanade registration algorithm
• Additional alignment using both 2D translation and rotation, following the approach described in Szeliski and Shum
• Combining the aligned images into a single panorama by averaging overlapping portions, using the Voronoi tesselation technique described in Peleg, or some hybrid of the two

User Interface

Most of the technique will be automated, but by using command-line parameters, or a Fluid-based interface, the user will be able to modify the default parameters of the mosaicing operation (for example, the stopping conditions for the image alignment step).

Functionality

Definitely included

• generating cylindrical panoramas
• generating panoramas from arbitrary camera motions (roughly) parallel to the image plane

We hope to include

• artificially generating stereo panoramas (as described in Peleg and Ben-Ezra, CVPR'99)
• global alignment, to better align the first and last images

If time permits

• rectification of curled panoramas, which are caused by camera tilting
• generalizing the approach to allow arbitrary (for example, forward) camera motion
• view interpolation to help compensate for motion parallax
• handling moving objects

Additional References

• P. Burt and E. Adelson. The Laplacian pyramid as a compact image code, IEEE Transactions on Communications, 31:482-540, 1983
• P. Burt and E. Adelson. A multiresolution spline with application to image mosaics, ACM Transactions on Graphics, 2:215-236, 1983.
• B. D. Lucas and T. Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision, International Joint Conference on Artificial Intelligence, pages 674-679, 1981.
• R. Szeliski and H.Y. Shum, Creating Full View Panoramic Image Mosaics and Environment Maps, Proc. ACM SIGGRAPH, pp. 251-258, 1997.
• S. Peleg and M. Ben-Ezra, Stereo Panorama with a Single Camera, CVPR'99, Ft. Collins, June 1999, pp. 395-401.
• A. Zomet, S. Peleg, and C. Arora, Rectified Mosaicing: Mosaics without the Curl, CVPR'00, Hilton Head Island, June 2000, Vol II, pp. 459-465