Wide Synthetic Aperture Sampling for Motion Classification

• Applicants: Professorin Dr.-Ing. Sanaz Mostaghim; Dr. Dmitriy Shutin
• Subject Area: Image and Language Processing, Computer Graphics and Visualisation, Human Computer Interaction, Ubiquitous and Wearable Computing
• Term: since 2023
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 502167710

With Airborne Optical Sectioning (AOS) we have introduced a wide synthetic aperture imaging technique that employs drones to sample images within large areas from above forests. These images are computationally combined to integral images which suppress strong occlusion and make visible hidden targets. The main advantages of AOS over alternatives, such as LiDAR or Synthetic Aperture Radar, are its real-time performance; its applicability to other wavelengths, such as far infrared for wildlife observations or search and rescue, or near infrared for agriculture and forestry applications; and its high spatial resolution. AOS and its successful applications to fully autonomous search and rescue drones is the overall result of two subsequent FWF projects: In Directional Super-resolution through Coded Sampling and Guided Up-Sampling we have investigated directional super-resolution techniques for sparse sampling devices (camera array and light-stages) that do not require scene depth, disparity estimation, or exact image correspondences. In Wide Synthetic Aperture Sampling we then extended these scientific findings to AOS with the goal to maximize spatial and directional resolut ions as well as de-focusing efficiency while minimizing depth of field and flying time for wide synthetic aperture sampling. Objectives: The sequential sampling nature of AOS, however, currently limits applications to static targets. Moving targets, such as walking people or running animals lead to motion blur in integral images that are nearly impossible to classify. Yet, the detection and interpretation of motion is essential for tracking and counting animals or people, and for finding hiding or lost people which walk, run, or move. The goal of this follow-up project is to investigate wide synthetic aperture sampling principles for motion detection and interpretation. In short, we are seeking novel solutions for enabling AOS to deal with moving targets, and to consequently open new application fields. This, however, requires gaining entirely new insights to wide synthetic aperture sampling strategies in theory and in practice. Approach: We want to investigate sequential sampling strategies together with adapted deblurring and anomaly detection techniques, novel parallel sampling strategies with first aerial camera arrays, dynamically reconfigurable drone swarms, as well as efficient combinations of sequential and parallel sampling to reduce complexity and to avoid undersampling. Motion itself could be an important feature for target classification. Innovation: New AOS sampling approaches for identifying moving targets will enable entirely new applications for areas such as wild-life observation, search and rescue, border control, surveillance, and others.

• DFG Programme: Research Grants
• International Connection: Austria
• Cooperation Partner: Professor Dr.-Ing. Oliver Bimber