Representation Alignment for Generation: Training Diffusion Transformers is Easier Than You Think

September 18, 2026

REPresentation Alignment (REPA) - align projections of input latents with clean image representations from pretrained visual encoders

-use DINO features to supervise DiT hidden features

-project DiT hidden layer outputs and minimize alignment loss to DINO features

Clean image x

│

├───────────────→ DINOv2 ─────────→ DINO features y

│ ↑

│ │ alignment loss

↓ │

VAE Encoder │

↓ │

latent z │

↓ + noise │

noisy latent z_t │

↓ │

DiT layer 1 → ... → DiT layer 8 ─→ h_8 → projection P

↓

remaining DiT layers

↓

diffusion prediction

Goal: DiT hidden states should predict noise-invariant, clean visual representations from noisy inputs that contain useful semantic information

-learn a projection head MLP

P

such that:

1.starting from VAE latent

z_t

2.given intermediate DiT hidden representation

z_t \xrightarrow{\text{DiT}} h_t

3.given DINO features

x \xrightarrow{\text{DINO}} y_{\text{DINO}}

4.we have

P(h_t) \approx y_{\text{DINO}}

-use patch-wise similarity for loss → supervise latent DiT tokens to be similar to specific DINO feature patches

-weighted training objective:

\mathcal{L} := \mathcal{L}_\text{velocity} + \lambda \mathcal{L}_\text{REPA}

1.REPA makes diffusion training faster

2.Plain SiT/DiT representations are much worse than DINO (on ImageNet), while SiT + REPA closes much of that gap

3.Better representation = better generation

4.Agnostic to teacher representation (not DINO-specific)

5.Aligning relatively early layers is sufficient (i.e. DiT layer 8)

6.REPA scales with the size of the diffusion model