Persistent Automated Tracking from Unmanned Aerial Vehicles (UAVs)


Project Description

Empowering unmanned aerial vehicles (UAVs) with automated computer vision capabilities (e.g. tracking, object/activity recognition, etc.) is becoming a very important research direction in the field and is rapidly accelerating with the increasing availability of low-cost, commercially available UAVs. In fact, aerial tracking has enabled many new applications in computer vision (beyond those related to surveillance) including search and rescue, wild-life monitoring, crowd monitoring/management, navigation/localization, obstacle/object avoidance, and videography of extreme sports. Aerial tracking can be applied to a diverse set of objects (e.g. humans, animals, cars, boats, etc.), many of which cannot be physically or persistently tracked from the ground. In particular, real-world aerial tracking scenarios pose new challenges to the tracking problem, exposing areas for further research. Visual tracking on UAVs is a very promising application, since the camera can follow the target based on visual feedback and actively change its orientation and position to optimize for tracking performance. This marks the defining difference compared to static tracking systems, which passively analyze a dynamic scene. In this project, we will develop novel tracking strategies that are designed for real-time operation on a UAV. These tracking methods should be fast, reliable, and accurate. For evaluation purposes, we will use the newly developed aerial tracking benchmark that the IVUL group has developed. Moreover, we will test out these trackers within the aerial simulator that the IVUL group developed based on a photo-realistic game engine and a VR setup, which allows the user to move the object to be tracked in the simulated environment. Finally, these tracking methods will be embedded in a fully functioning UAV, which will be able to automatically and persistently track an object of interest on the ground.​​​​ ​​​
Program - Electrical Engineering
Division - Computer, Electrical and Mathematical Sciences and Engineering
Center Affiliation - Visual Computing Center
Field of Study - ​Computer, Electrical , Mathematical Sciences , Engineering​

About the

Bernard Ghanem

Associate Professor, Electrical and Computer Engineering

Bernard Ghanem
Professor Ghanem's research interests focus on topics in computer vision, machine learning, and image processing. They include:
  • Modeling dynamic objects in video sequences to improve motion segmentation, video compression, video registration, motion estimation, and activity recognition.
  • Developing efficient optimization and randomization techniques for large-scale computer vision and machine learning problems.
  • Exploring novel means of involving human judgment to develop more effective and perceptually-relevant recognition and compression techniques.
  • Developing frameworks for joint representation and classification by exploiting data sparsity and low-rankness.

Desired Project Deliverables

​Statistics on the nuisances commonly faced in aerial tracking scenarios. Novel techniques to track a single object from a single aerial viewpoint. Novel techniques to search for an object when it moves outside the field of view of the camera. A fully functioning prototype UAV that runs the tracking method locally and that interfaces tracking results into the UAV’s navigation system.