Naval Surface Warfare Center, Philadelphia Division in April 2022 launched a two-phase prize challenge aimed at identifying providers of a capability that would automate document and data assessment for the controlled unclassified information (CUI) content using artificial intelligence (AI) and /or machine learning (ML). Recently, NSWCPD announced the winners of the challenge.
Taking first place of the Phase Two challenge was IBM followed by Deloitte, SERCO/Nuix and the Navy’s own Naval Information Warfare Center (NIWC) Pacific Division’s (PAC’s) Machine Learning and Natural Language Processing (ML/NLP) Group. Each team received prize money. Eligible teams received prize monies for a total of $120,000 (first place-$50,000; second place-$40,000; third place-$30,000).
As participants in the 2021/22 cohort of the Navy’s Bridging the Gap leadership development program, a team was assembled to pursue this unique and challenging Action Learning project.
Sponsored by the Office of Naval Research (ONR) and championed by Director, Mission Support Office of Naval Research Alonzie Scott III, the team was led by NSWCPD Acquisition Policy and Oversight Division Manager from Doris Tung and NSWC Crane Division Command Chief Engineer Lori Zipes. Team members on the project from across the Navy included: Synia Oxendine, Naval Sea Systems Command (NAVSEA); Anthony Pete Manupella, Naval Criminal Investigative Service (NCIS); as well as Alexander Miyaji and Kevin Leonard from ONR.
The goal of the project was to create an easy and automated mechanism to define and apply the required 297 individual categories of CUI markings as mandated by Executive Order (EO) 13556. The CUI program standardizes the way the government handles unclassified information that requires safeguarding or dissemination controls pursuant to and consistent with law, regulations, and policies. Certain characteristics of data in a file dictate that it be marked as CUI, and the relevant category identified so that the file can be properly marked.
Representing ONR, Scott said, “The CUI Prize Challenge is one of the most exciting ways to tackle an innovative project by linking industry and government. Like clockwork, we ask and partner with experts to share how they would solve the CUI problem.”
The Department of Defense (DoD) generates extensive amounts of unclassified information that is subject to control, based upon CUI guidance in accordance with DoD Instruction (DODI) 5200.48. Tung pointed out that due to the numerous CUI categories and the importance of contextual information, assessment of data files for CUI content is laborious, and authors and reviewers are often inconsistent, mis-categorize content, or fail to properly identify CUI content, putting DoD at risk of exposing information subject to control.
CUI was intended to speed the disclosure of information to other agencies, but due to the seriousness of the problem and inconsistency of its applications, the Senate Armed Services Committee called for a process to use controlled unclassified information in a consistent way after the project was launched.
According to EO 13556, the President of the United States recognized that “executive departments and agencies employ ad hoc, agency-specific policies, procedures, and markings to safeguard and control… information that involves privacy, security, proprietary business interests, and law enforcement investigations.” EO 13556 represented the efforts of the Obama administration to standardize controls for unclassified information in the interests of both protection and transparency.
According to the DoD’s Air Land Sea Space Application Center (ALSSA), the multi-service organization established by the doctrine centers to develop tactical-level solutions of multi- Service interoperability issues consistent with Joint and Service doctrine, “The implementation of DODI 5200.48 has not been smooth or clear for DoD as a whole. One of the effects of the implementation has been the creation of a potential barrier to information sharing with inter-agency partners. Use of CUI involves specified guidelines for safeguarding of that information which may create unnecessary barriers to efficient disclosure that result in both errors and, more importantly, negative consequences to partner trust.”
“Within the federal government we are generating so many documents every day especially in this electronic age. As the creator of the document, we have to make the decision on whether the document needs to be marked as CUI or not. This can be an email, a word document, excel file, etcetera,” Tung said. “The CUI guidance requires specific requirements on how you mark your document. The creator makes the decision on whether or not to mark it CUI and how to mark it, which could be correct or incorrect.”
The team determined that artificial intelligence and machine learning (AI/ML) technologies could be leveraged. In order to attract non-traditional vendors and encourage innovation, a prize challenge approach was chosen. The goal of the CUI Prize Challenge was to reduce the burden on employees to identify CUI, improve the quality and consistency of CUI markings, and find time efficiencies by automating the process of CUI identification and marking as much as possible.
Tung and her team defined a two-phase prize challenge, which resulted in the development of eight different prototypes. These prototypes were trained on simulated CUI data and competitively tested in the AI development lab at NSWC Crane.
“Elements of innovation are evident throughout this effort,” Tung said, explaining that the decision to focus on AI/ML as the core technology of the solution encouraged and enabled prototype developers to apply the technology into a new realm of usage.
“Leveraging a prize challenge approach successfully drew in non-traditional participants,” Tung said. “After recognizing that providing actual CUI to participants for training was unacceptable, the team rallied to generate over 80 simulated CUI documents. An additional 70 documents were developed or gathered to support the test event, along with actual CUI. To enable information sharing and submission of containerized prototypes for testing, a GitHub repository was created and managed by the team.”
“Testing eight different prototypes also brought unique challenges, all addressed with creative and thoughtful solutions. With members from NAVSEA as well as ONR, NRL, and NCIS, this team displayed an exemplary ‘One NAVSEA,’ arguably ‘One Navy’ attitude,” Tung said.
She continued, “The level of possible success for this effort was largely unknown at the outset. The varied backgrounds and expertise of the team members blended to result in a well-run, programmatically and technically creative, and rigorous effort that yielded very promising results. While the Bridging the Gap program ended in April 2022, the team saw the prize challenge event through to completion in October 2022.”
“Most Action Learning project results in briefings with recommendations. This effort produced functional prototypes. This team’s initial success is a valuable first step in solving an important but burdensome challenge with potential to benefit to the entire DoD workforce,” Tung said.
“I enjoyed working with an exceptional Bridging The Gap (BTG) cohort led by one of Philly’s best and brightest talents, Doris Tung. The BTG created a first-step prototype to meet the CUI needs of every naval organization. I plan to develop the software within the next fiscal year (FY) and beyond,” Scott said.
NSWCPD employs approximately 2,800 civilian engineers, scientists, technicians, and support personnel. The NSWCPD team does the research and development, test and evaluation, acquisition support, and in-service and logistics engineering for the non-nuclear machinery, ship machinery systems, and related equipment and material for Navy surface ships and submarines. NSWCPD is also the lead organization providing cybersecurity for all ship systems.