DOD launches exercise that explores generative AI

The Chief Digital and Artificial Intelligence Office today launched the first of two artificial intelligence bias bounty exercises aimed at unearthing unknown risk areas in Large Language Models.

“Bias bounties are new crowdsourced efforts to help detect bias in AI systems,” a Defense Department release said.

Both exercises aim to develop new routes to “algorithmically auditing and red teaming AI models,” as well as conducting experiments that address the risks involved with LLMs.

The CDAO Responsible AI Division, which is leading both exercises, launched the first bias bounty today, noting that the second will “soon follow.”

The first public bounty will run from Jan. 29 through Feb. 27.

"The RAI team is thrilled to lead these AI Bias Bounties, as we are strongly committed to ensuring that the Department's AI-enabled systems -- and the contexts in which they run -- are safe, secure, reliable and bias free," Matthew Johnson, acting chief of the DOD's RAI Division, said in the release.

The bounty exercise launched today specifically looks at Large Language Models to discover potential areas of risk, starting with open source chatbots.

"Given the Department's current focus on risks associated with LLMs, the CDAO is actively monitoring this area; the outcome of the AI Bias Bounties could powerfully impact future DOD AI policies and adoption," Craig Martell, chief digital and artificial intelligence officer, said in the release.

In December, Bill Streilein, CDAO’s chief technology officer, said that DOD is focusing on responsibly adopting and leveraging technology, like Generative AI, across the department.

In October, the Space Force put a temporary ban on government computers allowing Generative AI and Large Language Models in efforts to enhance data protection. Inside Defense was not immediately able to determine if the ban is still in place.

A few weeks prior to the ban being announced, Heidi Shyu, under secretary of defense for research and engineering, released a memo establishing a Defense Science Board task force charged with “balancing security, reliability, and technological advantage” related to Generative AI.

The release today encourages public involvement, citing no coding experience necessary, to help uncover bias, adding that participants can earn money from scoring and evaluation through DOD funding.

ConductorAI-Bugcrowd and BiasBounty.AI have partnered with CDAO to help with development and execution of both projects, with the CDAO Defense Digital Service Directorate advising.