Physical and Functional Inductive Biases for Visual Representation Learning
Stanford Investigators
Partner University Investigators
-
Josh Tenenbaum (MIT)
-
Frédo Durand (MIT)
TRI Co-PIs
Project Summary
Autonomous driving systems should not only perform well in scenarios they have seen before, but robustly generalize to previously unseen scenarios. This is also a major challenge to current AI models, which mostly learn by memorizing examples they have seen. In this project, we propose to build robust and generalizable autonomous driving systems by leveraging physical and functional inductive biases, such as object permanence, affordances, and light transport. These priors may serve as i) constraints enforcing realistic predictions compatible with common sense and natural laws, and ii) “free annotations” enabling learning from large-scale datasets spanning a wide range of perception and prediction tasks (e.g., scene and agent modeling).