4.03 Image aided flight control


Recent advances in the field of machine vision together with the rapidly increasing computing power available on miniaturized platforms open the perspective of utilizing image based or derived measurements as auxiliary or primary source of measurement for flight control applications. Besides the need for deterministic real-time behavior at high frame-rates and low latencies, rigid requirements addressing accuracy, availability and integrity of the image derived measurements represent challenges to be overcome to successfully exploit the technologies for flight control purposes. The primary motivation to use image based information as a further signal source is to be capable of flying in areas where other sources of navigation data are unavailable, like for example GPS during indoor navigation and to orient in non-cooperative dynamic environments, where the location of obstacles and other objects cannot be determined by comparing the own position derived from conventional navigation sources with a database.

The particular research focus of the project is on the tight and close integration of the contributing components. New specific sensor data fusion algorithms are being implemented. The tight bi-directional integration of image based and conventional navigation sources aims at increasing the operational robustness and reliability of the overall system. To fully utilize the capabilities of the approach, custom flight control algorithms are developed. Specific challenges on the image processing side is to maximize frame rates, decrease latencies and provide robustness against changing environmental conditions like light, shadow and contrast while simultaneously minimizing the amount of control power required.

To prove the theoretical results obtained in the project, actual flight tests are conducted with quad-rotor helicopters (Quadrocopter), with the ultimate goal of autonomous navigation and exploration in indoor environments as well as high bandwidth control.

Watch videos on our project teamĀ“s research here.


Klose, S. et al.: "Efficient Compositional Approaches for Real-Time Robust Direct Visual Odometry from RGB-D Data", 2013.

Wang, J., Raffler, T. et al.: "Nonlinear Position Control Approaches for Quadcopters using a Novel State Representation", USA, 2012.

Wang, J., Bierling, T. et al.: "Novel Dynamic Inversion Architecture Design for Quadrocopter Control", Heidelberg, 2011.

Wang, J., Bierling, T. et al.: "Attitude Free Position Control of a Quadcopter using Dynamic Inversion", USA, 2011.

Klose, S. et al.: "Markerless, Vision-Assisted Flight Control of a Quadrocopter", 2010.


Project team leader

Dr. Giorgio Panin
Informatics 6 - Chair of Robotics and Embedded Systems


Dr.-Ing. Jian Wang
Chair of Flight System Dynamics

Principal investigator

Prof. Dr. -Ing. Florian Holzapfel
Chair of Flight System Dynamics

Principal investigator

Professor Tiauw Hiong (Yongki) Go
Department of Mechanical Aerospace Engineering, Florida Institut of Technology


Principal investigator

Prof. Dr.-Ing. Alois Christian Knoll
Informatics 6 - Chair of Robotics and Embedded Systems