3D reconstruction by Matrix Factorization

Monash University

Institute for Vision Systems Engineering

One class of methods for reconstructing 3D structure/models from image sequences is the matrix factorization method devised by Tomasi and Kanade.

In this method, one tracks points and then assembles the x and y coordinates of these tracked points into a large matrix $W$. It can easily be shown that, assuming the camera is orthographic (more generally, affine):

$$W=MS$$

where $M$ is a matrix giving the relative object camera motion and $S$ is a matrix giving the 3D structure.

However, because tracks are not usually followed throughout the whole sequence (for example when the tracker loses track or when the point being tracked goes behind the object), $W$ is usually quite sparse.

We have developed a method to ``fill in'' the missing data and, at the same time, ``denoise'' the matrix $W$ [CS04].

Click here to see movie - approx 3Mb MPEG.