Skip to main content Skip to secondary navigation

Exploiting Depth and Motion Cues for Robust Semantic, Instance-level Segmentation and Tracking

Main content start

Principal Investigator:

Jeannette Bohg

TRI Liaison:

Jie Lie

Project Summary

We aim to develop a semantic, instance segmentation method that exposes increased robustness under challenging real-world conditions. Central to our approach is (i) to avoid the vulnerable, modularized process in current state-of-the-art region-proposal based methods and (ii) to exploit the powerful cues of depth and motion.

Robust, semantic, instance segmentation is fundamental for both, future intelligent vehicles and robots. It provides an accurate understanding of the environment under diverse and challenging conditions. Thereby, it enables safe and robust decision-making. Our approach leverages characteristics typical of both, autonomous cars and robots. It supports the themes of Self-Driving Cars and Robotics and Smart Environments.

Research Goals

  • Reformulate instance segmentation as one-stage regression problem in 3D to ease training, improve robustness and efficiency
  • Exploiting motion cues and assuming rigidity of objects to further increase robustness
  • Tracking semantically-annotated objects over a long time horizon to maintain coherent semantic segmentation over time