Persistent Automated Tracking from Unmanned Aerial Vehicles (UAVs)

Persistent Automated Tracking from Unmanned Aerial Vehicles (UAVs)

Internship Description

Empowering unmanned aerial vehicles (UAVs) with automated computer vision capabilities (e.g. tracking, object/activity recognition, etc.) is becoming a very important research direction in the field and is rapidly accelerating with the increasing availability of low-cost, commercially available UAVs. In fact, aerial tracking has enabled many new applications in computer vision (beyond those related to surveillance) including search and rescue, wild-life monitoring, crowd monitoring/management, navigation/localization, obstacle/object avoidance, and videography of extreme sports. Aerial tracking can be applied to a diverse set of objects (e.g. humans, animals, cars, boats, etc.), many of which cannot be physically or persistently tracked from the ground. In particular, real-world aerial tracking scenarios pose new challenges to the tracking problem, exposing areas for further research. Visual tracking on UAVs is a very promising application, since the camera can follow the target based on visual feedback and actively change its orientation and position to optimize for tracking performance. This marks the defining difference compared to static tracking systems, which passively analyze a dynamic scene.

In this project, we will develop novel tracking strategies that are designed for real-time operation on a UAV. These tracking methods should be fast, reliable, and accurate. For evaluation purposes, we will use the newly developed aerial tracking benchmark that the IVUL group has developed. Moreover, we will test out these trackers within the aerial simulator that the IVUL group developed based on a photo-realistic game engine and a VR setup, which allows the user to move the object to be tracked in the simulated environment. Finally, these tracking methods will be embedded in a fully functioning UAV, which will be able to automatically and persistently track an object of interest on the ground.​​​​

Deliverables/Expectations

Statistics on the nuisances commonly faced in aerial tracking scenarios.

Novel techniques to track a single object from a single aerial viewpoint.

Novel techniques to search for an object when it moves outside the field of view of the camera.

A fully functioning prototype UAV that runs the tracking method locally and that interfaces tracking results into the UAV’s navigation system.

Faculty Name

Bernard Ghanem

Field of Study

Computer, Electrical , Mathematical Sciences , Engineering​