Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning
August 4, 2026
-choice of early features and BatchNorm are crucial for performance
-the pre-trained model contains priors from a wide range of real-world images
-will they transfer to simulation?
Batch normalization
-convolutions take images (for example, ) and output feature maps (filters)
-because they learn different things, the mean and standard deviation of feature maps can vary widely
-BatchNorm normalizes them to mean and standard deviation , so inputs to the subsequent layer have a consistent scale
Key findings
1.Early layers of a neural network provide better representations for visual RL generalization than the final feature vector
1.early layers learn edges, corners, textures, and similar low-level features
2.these features are more relevant for learning contact points and geometry
3.the feature maps of the second layer are flattened and passed through another fully connected layer to form representations
2.Updating BatchNorm statistics while freezing layer weights helps ResNet match the robot's observation distribution to ImageNet's
Implementation details
-DMControl-GB: input size ; discount factor ; action repeat for Cartpole and otherwise; frame stack ; learning rate ; random-shifting padding ; training steps; evaluation episodes; Adam optimizer
-Drawer: input size ; discount factor ; action repeat ; frame stack ; learning rate ; random-shifting padding ; training steps; evaluation episodes; Adam optimizer
-World Manipulation Tasks: input size ; discount factor ; action repeat ; frame stack ; learning rate ; random-shifting padding ; training steps; evaluation episodes; Adam optimizer
-CARLA: input size ; discount factor ; action repeat ; frame stack ; learning rate ; random-shifting padding ; training steps; evaluation episodes; Adam optimizer