Report: Cyber threat actors embracing use of generative AI, especially for info ops

Security firm Mandiant sees growing use of artificial intelligence tools by cyber threat actors in areas like disinformation and slower adoption in "intrusion campaigns," but cautions that generative AI is poised to accelerate uses in both areas.

“Based on our own observations and open-source accounts, adoption of AI in intrusion operations remains limited and primarily related to social engineering,” according to the report, “Threat Actors Are Interested in Generative AI, but Use Remains Limited,” released Thursday.

“In contrast,” Mandiant says, “information operations actors of diverse motivations and capabilities have increasingly leveraged AI-generated content, particularly imagery and video, in their campaigns, likely due at least in part to the readily apparent applications of such fabrications in disinformation. Additionally, the release of multiple generative AI tools in the last year has led to a renewed interest in the impact of these capabilities.”

The report says, “We anticipate that generative AI tools will accelerate threat actor incorporation of AI into information operations and intrusion activity.”

John Hultquist, chief analyst at Mandiant Intelligence and Google Cloud, commented, “While we expect the adversary to make use of generative AI, and there are already adversaries doing so, adoption is still limited and primarily focused on social engineering. There’s no doubt that criminals and state actors will find value in this technology, but many estimates of how this tool will be used are speculative and not grounded in observation.”

Mandiant is a Google subsidiary.

According to Mandiant, “Generative AI will enable information operations actors with limited resources and capabilities to produce higher quality content at scale. … Hyper-realistic AI-generated content may have a stronger persuasive effect on target audiences than content previously fabricated without the benefit of AI technology.”

The report provides a detailed look at AI-generated images, video and text, and explains the evolving uses of AI in social engineering and areas like voice impersonation.

Mandiant also “anticipates that threat actors will increase their use of [large language models] to support malware development. LLMs can help threat actors write new malware and improve existing malware, regardless of an attacker's technical proficiency or language fluency.”

But it notes, “LLMs possess shortcomings in their malware generation that may require human intervention for correction, [although] the ability of these tools to significantly assist in malware creation can still augment proficient malware developers, and enable those who might lack technical sophistication.”