Three-Dimensional Scene Flow (Vedula et al.)

October 6, 2026

scene flow - how a 3D point in the world moves

optical flow - how a pixel in an image moves

-the 2D projection of scene flow

-“an optical flow” refers to a single optical-flow field from one camera view between two consecutive time steps

💡

Pinhole Camera Review

-by the pinhole camera model:

-focal length - distance from the camera center to the image plane

-virtual image plane - because the camera center is in front of the image plane, the image is flipped; the virtual image has the correct orientation

f_x

and

f_y

are focal lengths measured using horizontal and vertical pixel dimensions

-they differ only when pixels are non-square

f_x = \frac{f\text{ mm}}{x\text{ mm per pixel (horizontal)}}

f_y = \frac{f\text{ mm}}{y\text{ mm per pixel (vertical)}}

\begin{bmatrix} u \\ v \\ 1 \end{bmatrix} = K [R \mid t] I_w

-for

f = 100

px,

P(1, 0, 5)

, and default camera extrinsics:

K = \begin{bmatrix}100 & 0 & 0 \\ 0 & 100 & 0 \\ 0 & 0 & 1\end{bmatrix}

[R \mid t] = \begin{bmatrix}1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0\end{bmatrix}

I_w = \begin{bmatrix}1 \\ 0 \\ 5 \\ 1\end{bmatrix}

I = K [R \mid t] I_w = \begin{bmatrix}100 \\ 0 \\ 5\end{bmatrix}

-dividing by the homogeneous coordinate gives

(u, v) = (20, 0)

-if

P_t(1, 0, 5)

moves to

P_{t+1}(1.5, 0, 5)

P_{t+1}

projects to

(u, v) = (30, 0)

-the optical flow of

P_t

(10, 0)

-the scene flow of

P_t

(0.5, 0, 0)

-a single optical flow cannot be inverted into scene flow: multiple 3D motions can produce the same 2D optical flow

dense non-rigid scene flow

-dense field - a 3D motion vector for every visible surface point, rather than only a sparse set of tracked features

-non-rigid motion - each scene point can move independently of every other scene point