Detection of M365 Copilot Jailbreak Attempts via Prompt Injection

This brief focuses on the detection of jailbreak attempts targeting Microsoft 365 Copilot. Attackers use prompt injection techniques to circumvent built-in safety controls. These attempts involve manipulating the AI's behavior through direct instruction, such as rule manipulation, system bypass commands, and unauthorized persona adoption. The attacks are identified by analyzing eDiscovery prompt logs for specific keywords associated with jailbreaking, including "pretend you are," "act as," "rules=," "ignore," "bypass," and "override." Successful jailbreak attempts can enable unauthorized access, data exfiltration, and policy violations, potentially leading to significant security breaches. This activity is detected by Splunk ES using the "M365 Copilot Jailbreak Attempts" detection rule.

Attack Chain

1. Attacker crafts a malicious prompt containing jailbreaking keywords (e.g., "ignore all previous rules").
2. The crafted prompt is submitted to M365 Copilot through a supported application.
3. M365 Copilot processes the prompt and attempts to fulfill the request, potentially bypassing safety controls.
4. The eDiscovery logs capture the prompt text in the Subject_Title field, along with user and timestamp information.
5. A security monitoring system, such as Splunk, ingests the eDiscovery logs.
6. The detection rule identifies prompts containing jailbreak keywords and assigns a severity score.
7. If the jailbreak score meets or exceeds a predefined threshold, an alert is triggered.
8. The attacker gains unauthorized control over Copilot's behavior, potentially leading to data exfiltration or other malicious activities.

Impact

A successful jailbreak of M365 Copilot can lead to several negative outcomes. It allows attackers to bypass the intended safety measures and manipulate Copilot for unauthorized purposes. This can result in the exposure of sensitive information, violation of data governance policies, and potentially the execution of malicious commands within the M365 environment. The exact scope of impact depends on the level of access granted to Copilot and the attacker's objectives.

Recommendation

• Enable and configure M365 eDiscovery to capture Copilot prompt logs, ensuring the Subject_Title, Sender, timestamps, and workload metadata are included.
• Deploy the provided Sigma rule M365 Copilot Jailbreak Attempts to your SIEM and tune the jailbreak_score threshold based on your environment.
• Investigate any alerts generated by the rule, focusing on the user, Subject_Title, and jailbreak_score fields to understand the nature of the attempted jailbreak.
• Implement user awareness training to educate employees about the risks of prompt injection and social engineering attacks targeting AI assistants.