July 29, 2021 – While autonomous car companies strive to build a self-driving car that people can trust, a Navy engineering team designed a decision aid that Sailors operating a high-energy laser can trust.
The common denominator in both cases is autonomy via artificial intelligence.
In the case of an autonomous system that uses artificial intelligence to assist warfighters in the process of neutralizing targets with a high-energy laser, Dr. Tyler Ferro pondered a question.
How accurate does artificial intelligence have to be in order to gain the warfighters’ trust?
“Sailors need to be able to trust it,” said Ferro. “If they don’t trust it they’re just going to toss it aside and go back to what they were using before.”
Ferro – an operational design and performance engineer at Naval Surface Warfare Center Dahlgren Division (NSWCDD) – contemplated another question.
What can we develop to maintain trust between human and machine while ensuring the warfighter does not become dependent on the system in cases of system failure?
“Trust can only be maintained if the human and machine act as a team,” said Ferro. “We later confirmed that they are indeed stronger and perform better by working together as a human-machine team rather than working independently.
In response to his initial questions, Ferro and a team of laser and operational performance engineers developed the High Energy Laser Fire Control Decision Aid (HEL FCDA) and embarked on a user performance study to find the answers.
Moreover, they coordinated with Sailors who participated in a human performance experiment that used the simulated decision aid to gather data about the impacts on kill chain time, trust, neutralization rate, usability and work load.
In effect, the simulation – a machine learning algorithm – became the basis for the Fire Control Decision Aid and human performance testing to predict the efficacy of the final product.
“We arranged it very specifically to gauge the accuracy of the decision aid system’s artificial intelligence,” said Ferro. “We did this based on the literature about autonomous cars that describe the precise point where people either trust or don’t trust self-driving cars,” said Ferro. “We wanted to determine what accuracy and performance we need out of an artificial intelligence machine learning model to assist the warfighter in a tactical situation with enough information to trust it and actively engage with the model as opposed to turning it off and just setting it aside.”
Several months before the pandemic, Sailors from various commands arrived at the NSWCDD Human Systems Integration (HSI) Laboratory to engage in the human-machine teaming study.
“We discovered that warfighters can become overwhelmed with a multitude of computer and system interactions,” said Ferro. “These interactions include crosschecking all of their documentation and information about possible targets that they may engage in different areas. We wanted to show the benefit to artificial intelligence and machine language approaches to increase decision-making efficiency for warfighters who operate high-energy laser weapons.”
When mandatory telework requirements took effect in March 2020 due to the COVID-19 pandemic, Ferro updated the simulation by converting the software that ran on stand-alone testing computers in the lab to work on Navy-Marine Corps Intranet (NMCI) and research, development, test and evaluation (RDT&E) computers. The update enabled Sailors from Norfolk Virginia Beach, Dahlgren (Aegis Training and Readiness Center) and a New Jersey based command with access to NMCI and RDT&E networks to participate in the experiment remotely.
The human-machine teaming study via 26 warfighters – 10 participating in person at the HSI Lab and 16 remotely – demonstrated an overwhelming need for implementation of the High Energy Laser Fire Control Decision Aid in the fleet.
It also proved the need for the NSWCDD Combat Systems Intelligent Automation Branch in collaboration with the Laser Weapon System Technology Development Branch to further investigate intelligent automation and decision aids in other tactical decision making processes.
“The study revealed that warfighters were significantly faster in responding to and neutralizing targets while using the Fire Control Decision Aid,” said Ferro. “They were more accurate and able to neutralize the target by quickly selecting the correct aimpoint. The system was rated by the warfighter as more usable, and we were able to show that the warfighters were far less stressed by using the decision aid.”
In other words, the results proved that warfighters perform much better when artificial intelligence and machine learning is incorporated in a tactical situation.
“The simulation automatically provided nudges in the right direction while suggesting the next step in the process,” said Ferro.
The simulated aid showed Sailors the best aimpoint of a given target for optimal performance of the laser, saving them valuable time and energy necessary for other engagements as time goes on.
“I analyzed the results from 20 or so different viewpoints or angles and every single time it showed the need for these decision aids to help our warfighters,” said Ferro. “The decision aid system increases their ability to neutralize targets in a shorter amount of time and more accurately. The results of our study kept showing over and over again that this aid is vital.”
Ferro and his colleagues are continuing to develop the High Energy Laser Fire Control Decision Aid. Meanwhile, the NSWCDD team is planning its next step – incorporating the decision aid into current projects to enhance capabilities in the current and future fleet.
The team plans to make it interoperable with the Navy’s integrated combat system – a complex combat system vital to the future fleet’s capabilities.
“As a scientist this human-machine teaming study leaves me with a burning desire to keep diving deeper and seeking new insights,” said Ferro. “As an engineer I can see the immediate need and know we need to move quickly to make full use of this technology before our adversaries do.”
Future warfighter testing will comprise: a wider accuracy range, multivariate analysis; interactions analysis; non-linear regression; single versus multiple aid suggestions; display aid confidence level; fast accept for high aid confidence; affect of unreasonable correct or incorrect aid; proclaimed system accuracy and origin; teammate versus manager role; limited supply of alternatives; larger occlusion variance; an investigation into the limited use of alternate engagement; and forced feedback.