{"description":"Trending threats, MITRE ATT\u0026CK coverage, and detection metadata. Fed continuously.","feed_url":"https://feed.craftedsignal.io/tags/persona-injection/feed.json","home_page_url":"https://feed.craftedsignal.io/","items":[{"_cs_actors":[],"_cs_cpes":[],"_cs_cves":[],"_cs_exploited":false,"_cs_has_poc":false,"_cs_poc_references":[],"_cs_products":["Microsoft 365 Copilot"],"_cs_severities":["medium"],"_cs_tags":["copilot","jailbreak","ai","persona-injection"],"_cs_type":"advisory","_cs_vendors":["Microsoft"],"content_html":"\u003cp\u003eThis detection identifies attempts to jailbreak Microsoft 365 Copilot by manipulating the AI's persona. Users attempt to circumvent safety controls and acceptable use policies by prompting Copilot to adopt alternate identities or unrestricted behaviors. The activity is detected by analyzing exported eDiscovery prompt logs for specific keywords and phrases indicative of roleplay or impersonation requests. This includes prompts instructing the AI to \u0026quot;pretend you are,\u0026quot; \u0026quot;act as,\u0026quot; or adopt amoral stances. The detection categorizes these prompts into types such as \u0026quot;AI_Impersonation,\u0026quot; \u0026quot;Malicious_AI_Persona,\u0026quot; and \u0026quot;Unrestricted_AI_Persona\u0026quot; to pinpoint efforts to override the AI's guardrails through persona injection attacks. This matters because successful jailbreaks can lead to Copilot generating harmful or inappropriate content, revealing sensitive information, or performing unauthorized actions, potentially impacting organizational security and compliance. The detection utilizes prompt logs exported from M365 eDiscovery.\u003c/p\u003e\n\u003ch2 id=\"attack-chain\"\u003eAttack Chain\u003c/h2\u003e\n\u003col\u003e\n\u003cli\u003eUser crafts a prompt containing roleplay keywords (e.g., \u0026quot;pretend you are,\u0026quot; \u0026quot;act as,\u0026quot; \u0026quot;amoral\u0026quot;).\u003c/li\u003e\n\u003cli\u003eThe prompt is submitted to M365 Copilot through a supported application (e.g., Teams, Word).\u003c/li\u003e\n\u003cli\u003eCopilot processes the prompt and generates a response based on the instruction.\u003c/li\u003e\n\u003cli\u003eThe prompt and response are logged within the M365 environment.\u003c/li\u003e\n\u003cli\u003eAn administrator or security analyst exports eDiscovery prompt logs from the Microsoft Purview compliance portal.\u003c/li\u003e\n\u003cli\u003eThe exported logs, containing fields like Subject_Title (prompt text) and Sender, are ingested into a SIEM or security analytics platform.\u003c/li\u003e\n\u003cli\u003eThe detection rule identifies prompts matching the defined impersonation patterns and categorizes the impersonation type.\u003c/li\u003e\n\u003cli\u003eIf successful, the attacker gains the ability to influence Copilot's behavior beyond intended boundaries, potentially extracting sensitive data or generating malicious content.\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"impact\"\u003eImpact\u003c/h2\u003e\n\u003cp\u003eA successful M365 Copilot jailbreak can lead to several negative consequences. The AI might generate responses that violate compliance regulations, reveal confidential company information, or be used to craft phishing emails or social engineering attacks. While specific victim counts are unavailable, organizations that heavily rely on M365 Copilot for sensitive tasks are at heightened risk. If the attack succeeds, the AI's output is no longer trustworthy, potentially damaging brand reputation, incurring legal penalties, and compromising internal security.\u003c/p\u003e\n\u003ch2 id=\"recommendation\"\u003eRecommendation\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003eConfigure Microsoft Purview eDiscovery to export M365 Copilot prompt logs, capturing user interactions and prompts (as described in \u003ccode\u003ehow_to_implement\u003c/code\u003e).\u003c/li\u003e\n\u003cli\u003eDeploy the Sigma rule \u003ccode\u003eM365 Copilot Impersonation Jailbreak Attack\u003c/code\u003e to your SIEM and tune the \u003ccode\u003em365_copilot_impersonation_jailbreak_attack_filter\u003c/code\u003e macro to reduce false positives based on your environment.\u003c/li\u003e\n\u003cli\u003eInvestigate alerts generated by the Sigma rule, focusing on users with high-risk scores as identified by the RBA message \u003ccode\u003eUser $user$ attempted M365 Copilot impersonation jailbreak with impersonation type $impersonation_type$\u003c/code\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n","date_modified":"2024-01-27T14:30:00Z","date_published":"2024-01-27T14:30:00Z","id":"https://feed.craftedsignal.io/briefs/2024-01-27-m365-copilot-jailbreak/","summary":"This detection identifies M365 Copilot impersonation and roleplay jailbreak attempts by analyzing exported eDiscovery prompt logs, searching for users manipulating the AI into adopting alternate personas or bypassing safety controls via roleplay keywords, categorizing specific impersonation types to identify persona injection attacks.","title":"M365 Copilot Impersonation Jailbreak Attempt Detection","url":"https://feed.craftedsignal.io/briefs/2024-01-27-m365-copilot-jailbreak/"}],"language":"en","title":"CraftedSignal Threat Feed - Persona-Injection","version":"https://jsonfeed.org/version/1.1"}