Innovative Application of AI/ML to Situational Awareness and Decision-Making

The NATO Communications and Information (NCI) Agency announced the winners of its challenge focused on UAS data on 20 May.

NCI ran the challenge between February and May as part of its hosting the International Conference on Military Communications and Information Systems (ICMIS), which this year focused on the application of AI and machine learning (ML) to military situational awareness and decision-making.

The challenge tested participants’ use of the latest tracking, data fusion and machine learning techniques to detect, track and identify small UAS. Participants fused several sources of data provided by the Agency to track the drones. The challenge focuses on Class I UAS, which includes systems with a mass lower than 150kg and includes typical ‘hobby’ drones. The competition was run on Kaggle, a popular platform for AI/ML challenges.

“It was really impressive to see the great interest shown by the participants in the challenge. This was the first challenge organized by the Agency using Kaggle, in a public domain where data science enthusiasts and research groups equally compete on diverse topics relevant for our society today,” commented Dr Michael Street, NCI’s Head of Innovation and Data Science.

After evaluating submissions, by comparing the test set with the ground truth data, the top four were asked to share their approach and techniques in greater depth with a wider community, during the special session on C-UAS and RF technologies at ICMCIS.

The top four teams were:

• Centre For Research and Technology Hellas (CERTH) Information Technologies Institute (ITI) Virtual and Augmented Reality Lab (VARlab) Team: proposed a tracker based on ML technologies – something new in the tracking domain;
• Defence Science & Technology Laboratories (DSTL): used well-studied tracking and data fusion techniques available under the open source project known as Stone Soup, solving specific challenges such as data association, track filtering and track management;
• CERTH ITI Visual Computing Lab Team: used a mix of techniques, including the Hungarian algorithm, to solve the data association problem and an ML solution for the filtering part;
• Horizon Lab: proposed an innovative solution based on ML, and although with less focus on challenges such as data association, track management or data fusion; their solution scored relatively high in the Mean Root Square Error (MRSE) parameter used in the ranking.

“The teams achieved very good results using classical tracking and data fusion techniques, as well as new machine learning approaches applied to solve the problem,” said Dr Cristian Coman, Principal Scientist at NCI. “The NCI Agency will continue to periodically challenge researchers and technology enthusiasts with practical security challenges, linked to realistic datasets.”

This challenge is part of a larger NCI R&D effort aimed at developing effective remote sensing technologies suitable for detecting, tracking and identifying Class I UAS. The data released for this challenge was recorded in 2020, during a measurement campaign at the Dutch Ministry of Defence’s Counter-UAS Nucleus. NATO sponsored the measurement campaign through its Defence Against Terrorism Programme of Work, and the event was further supported by several government and industry partners.

Another C-UAS Technical Interoperability Exercise (TIE21) will be organized in November 2021, the intention being again to collect sensor data that can be used in another challenge.