M365 Copilot Impersonation Jailbreak Attempt Detection

This detection identifies attempts to jailbreak Microsoft 365 Copilot by manipulating the AI's persona. Users attempt to circumvent safety controls and acceptable use policies by prompting Copilot to adopt alternate identities or unrestricted behaviors. The activity is detected by analyzing exported eDiscovery prompt logs for specific keywords and phrases indicative of roleplay or impersonation requests. This includes prompts instructing the AI to "pretend you are," "act as," or adopt amoral stances. The detection categorizes these prompts into types such as "AI_Impersonation," "Malicious_AI_Persona," and "Unrestricted_AI_Persona" to pinpoint efforts to override the AI's guardrails through persona injection attacks. This matters because successful jailbreaks can lead to Copilot generating harmful or inappropriate content, revealing sensitive information, or performing unauthorized actions, potentially impacting organizational security and compliance. The detection utilizes prompt logs exported from M365 eDiscovery.

Attack Chain

1. User crafts a prompt containing roleplay keywords (e.g., "pretend you are," "act as," "amoral").
2. The prompt is submitted to M365 Copilot through a supported application (e.g., Teams, Word).
3. Copilot processes the prompt and generates a response based on the instruction.
4. The prompt and response are logged within the M365 environment.
5. An administrator or security analyst exports eDiscovery prompt logs from the Microsoft Purview compliance portal.
6. The exported logs, containing fields like Subject_Title (prompt text) and Sender, are ingested into a SIEM or security analytics platform.
7. The detection rule identifies prompts matching the defined impersonation patterns and categorizes the impersonation type.
8. If successful, the attacker gains the ability to influence Copilot's behavior beyond intended boundaries, potentially extracting sensitive data or generating malicious content.

Impact

A successful M365 Copilot jailbreak can lead to several negative consequences. The AI might generate responses that violate compliance regulations, reveal confidential company information, or be used to craft phishing emails or social engineering attacks. While specific victim counts are unavailable, organizations that heavily rely on M365 Copilot for sensitive tasks are at heightened risk. If the attack succeeds, the AI's output is no longer trustworthy, potentially damaging brand reputation, incurring legal penalties, and compromising internal security.

Recommendation

• Configure Microsoft Purview eDiscovery to export M365 Copilot prompt logs, capturing user interactions and prompts (as described in how_to_implement).
• Deploy the Sigma rule M365 Copilot Impersonation Jailbreak Attack to your SIEM and tune the m365_copilot_impersonation_jailbreak_attack_filter macro to reduce false positives based on your environment.
• Investigate alerts generated by the Sigma rule, focusing on users with high-risk scores as identified by the RBA message User $user$ attempted M365 Copilot impersonation jailbreak with impersonation type $impersonation_type$.