Adversaries Leveraging AI for Vulnerability Exploitation and Augmented Operations

The Google Threat Intelligence Group (GTIG) has observed an increasing trend of adversaries leveraging AI to augment various phases of the attack lifecycle. This includes supporting vulnerability discovery and exploit development, facilitating autonomous command execution, enabling targeted reconnaissance, and improving the efficacy of social engineering and information operations. State-sponsored actors, particularly those associated with the People’s Republic of China (PRC) and the Democratic People’s Republic of Korea (DPRK), have demonstrated sophisticated approaches to AI-augmented vulnerability discovery. Additionally, AI-driven coding accelerates the development of infrastructure suites and polymorphic malware, and AI-enabled malware like PROMPTSPY facilitates autonomous attack orchestration. TeamPCP (UNC6780) has begun targeting AI environments and software dependencies as an initial access vector.

Attack Chain

1. Reconnaissance: Adversaries use AI to perform in-depth reconnaissance on target systems and networks, identifying potential vulnerabilities and weaknesses.
2. Vulnerability Discovery: AI models are leveraged to analyze code, reverse-engineer applications, and identify zero-day vulnerabilities. UNC2814 uses expert cybersecurity personas to prompt Gemini for vulnerability research into embedded device targets.
3. Exploit Development: AI tools are used to generate sophisticated exploits for identified vulnerabilities.
4. Initial Access: TeamPCP (UNC6780) targets AI environments and software dependencies as an initial access vector, exploiting supply chain vulnerabilities.
5. Defense Evasion: AI-driven coding accelerates the development of polymorphic malware with AI-generated decoy logic to evade detection. Suspected Russia-nexus threat actors use AI for obfuscation.
6. Command and Control: AI-enabled malware, like PROMPTSPY, dynamically generates commands and manipulates victim environments, offloading operational tasks to AI.
7. Lateral Movement: Threat actors attempt to pivot from compromised AI software to broader network environments.
8. Impact: Disruptive activities, such as ransomware deployment and extortion, are carried out after gaining access to the broader network.

Impact

Successful exploitation can lead to unauthorized access to sensitive data, system compromise, and deployment of ransomware. Supply chain attacks targeting AI environments can result in widespread disruption and compromise of dependent systems. The use of AI in information operations enables the fabrication of digital consensus through synthetic media, potentially influencing public opinion.

Recommendation

• Monitor network traffic for unusual patterns indicative of reconnaissance or exploit attempts targeting AI environments and software dependencies.
• Implement robust security measures to protect AI development environments and software supply chains, mitigating the risk of initial access via compromised components.
• Deploy the Sigma rule “Detect Gemini API Abuse via User Agent” to identify potential misuse of AI services (rule below).
• Monitor process creation events for unusual processes or command-line arguments indicative of exploit execution or lateral movement.
• Implement endpoint detection and response (EDR) solutions to detect and respond to polymorphic malware and AI-enabled malware such as PROMPTSPY.
• Regularly update and patch systems and applications to address known vulnerabilities.
• Block access to known malicious domains or IP addresses associated with threat actors (if any are identified in follow-up reporting).