Low-latency monitoring of intelligent vehicle fleets at scale is critical to their safe, reliable, and efficient operation. In this project, we will develop data processing infrastructure capable of macroscopic anomaly detection and explanation for intelligent vehicle fleets. Given a lack of suitable existing solutions, we will develop an analytics engine capable of (1) analysis over thousands or more sensors on thousands or more vehicles (2) in near-real-time, minimizing delay from observation to insight, alert, or decision while (3) providing a generic programming interface for fleet operators. Our goal is a high-performance, real-world software prototype capable of a range of analyses, including alerting for systemic vehicular component failures, software errors, and aberrant driving.
Driving Goal: design and implement the first scale-out data processing system for anomaly detection over sensor streams from a fleet of smart vehicles
- Dataflow-based anomaly detection: How can we express best-of-breed anomaly detection algorithms in the dataflow model, the lingua franca of Big Data processing?
- Scale-out: Partitioning, replication, fault tolerance: How can we scale a dataflow engine to enable smart vehicle operation at low latency while operating over imprecise data?
- Incremental computation: How can we co-design infrastructure and algorithms to allow fast refresh of models?