Milestone in Improving Transparency of Machine Learning Systems
BAE Systems recently delivered its MindfuL software to the US Defense Advanced Research Projects Agency (DARPA) under the agency’s Competency-Aware Machine Learning (CAML) programme, the company announced on 14 July.
The delivery marks the first milestone in the programme to improve the transparency of machine learning (ML) systems. Transitioning artificial intelligence (AI)-based systems from decision-making tools into true partners requires users to trust their machine counterpart. While ML technology has matured, these systems are unable to communicate context and confidence in their decisions – including task strategies, the completeness of their training relative to a given task, factors that may influence their actions, or the likelihood of success under specific conditions. To meet these challenges, BAE Systems developed MindfuL, a system which will independently audit an ML-based system and provide the end user with insights to build trust in the technology. The first software release of the system provides a baseline capability to detect when the system encounters a new environment for which it has not been trained.
“The technology that underpins machine learning and artificial intelligence applications is rapidly advancing, and now it’s time to ensure these systems can be integrated, utilized, and ultimately trusted in the field,” explained Chris Eisenbies, Product Line Director of the Autonomy, Control and Estimation group at BAE Systems. “The MindfuL system stores relevant data in order to compare the current environment to past experiences and deliver findings that are easy to understand.”
The programme will produce statements such as “The machine learning system has navigated obstacles in sunny, dry environments 1,000 times and completed the task with greater than 99 percent accuracy under similar conditions.” Or, alternatively, “The machine learning system has only navigated obstacles in rain 100 times with 80 percent accuracy in similar conditions; manual override recommended.”
The MindfuL software was designed and built as part of a collaboration between BAE Systems and Massachusetts Institute of Technology’s Computer Science and Artificial Intelligence Laboratory. Under the terms of the approximately $5 million, three-year CAML programme awarded in 2019, BAE Systems’ FAST Labs research and development organization will demonstrate the technology in both simulation and prototype hardware later this year.
