Abstract
End-to-end (E2E) autonomous driving models achieve strong planning performance but remain largely opaque, limiting observability and trust. We apply mechanistic interpretability to E2E driving by training Sparse Autoencoders (SAEs) on the internal activations of a simple, single-camera end-to-end path planning model. Unsupervised clustering of SAE latents reveals interpretable structure aligned with route intent, vehicle speed, and braking behavior, recovering route intent with 98.3% accuracy without supervision. Through paired counterfactual image edits, we further show that specific SAE features are visually grounded and systematically respond to scene changes such as traffic light state and stop sign presence. Steering individual SAE latents produces predictable, monotonic changes in planner outputs including lateral displacement, curvature, and braking. Finally, we demonstrate a practical application of these representations: we build a lightweight adapter trained on SAE latents that reduces trajectory selection error by 3.44%, outperforming an equivalent adapter that omits SAE latents, confirming that the sparse representation captures useful information for interpretability.
Latent Steering
SAE Feature Activation Over Time
Counterfactual Visual Edits
Scene
Change in SAE feature activations
Planner output
SAE Features Have Aligned Steering Effects
BibTeX
@inproceedings{gagvani2026mechanistic,
title = {Mechanistic Interpretability for End-to-End Self-Driving},
author = {Gagvani, Manav and Namikas, Benjamin and
Velmurugan, Sivamurugan and Kumar, Nysa and
Sharma, Shrey and Peterson, Thomas and
Mathur, Pratyush and Chang, Hyunseo and
Paleja, Rohan},
booktitle = {Conference on Robot Learning},
year = {2026},
url = {https://mgagvani.github.io/mechanistic-interpretability-for-e2e-driving/}
}