Cameras as Relative Positional Encoding

October 19, 2026

Raymaps — R(u,v)=[ox,oy,oz,dx,dy,dz]R(u, v) = [o_x, o_y, o_z, d_x, d_y, d_z] — provide explicit 3D information.

-o∈R3o \in \mathbb{R}^3 defines the ray origin
-d∈R3d \in \mathbb{R}^3 defines the ray direction
-t≥0t \geq 0 is the distance multiplier
-useful for determining how pixels from different camera views relate geometrically, without depth
-require choosing a frame of reference, which can hinder generalization

SE(3)-relative attention tells models how the coordinate frames of two cameras relate.

-does not require defining a world coordinate system
-provides no explicit 3D information about a single view
Aij=QiKjTd+bθ(Ti−1Tj)A_{ij} = \frac{Q_i K_j^T}{\sqrt{d}} + b_\theta(T_i^{-1} T_j)