https://feed.craftedsignal.io/tags/copilot/Trending threats, MITRE ATT&CK coverage, and detection metadata — refreshed continuously.Hugoenhello@craftedsignal.iohello@craftedsignal.ioWed, 03 Jan 2024 12:00:00 +0000https://feed.craftedsignal.io/briefs/2024-01-03-m365-copilot-jailbreak/Wed, 03 Jan 2024 12:00:00 +0000hello@craftedsignal.iohttps://feed.craftedsignal.io/briefs/2024-01-03-m365-copilot-jailbreak/The detection identifies attempts to jailbreak Microsoft 365 Copilot through prompt injection techniques that attempt to circumvent built-in safety controls by manipulating rules, bypassing system commands, or requesting AI impersonation.Microsoft 365 Copilot is susceptible to jailbreak attempts via prompt injection, where users craft specific prompts designed to bypass or override safety controls. These attacks involve injecting malicious instructions into user prompts to manipulate the AI’s behavior, potentially leading to the disclosure of sensitive information, the generation of harmful content, or the execution of unauthorized actions. The attacks leverage techniques like rule manipulation, system bypass commands, and AI impersonation requests, attempting to circumvent built-in safety mechanisms. Successful jailbreaks can compromise the integrity and security of Copilot, enabling threat actors to exploit the AI for malicious purposes.

Attack Chain

1. An attacker crafts a malicious prompt containing specific keywords and phrases designed to manipulate Copilot’s behavior.
2. The attacker injects the prompt into M365 Copilot through a standard user interface, like a chat window.
3. Copilot processes the prompt, attempting to interpret the user’s intent.
4. If the prompt is successfully injected, Copilot’s safety controls are bypassed or overridden due to prompt injection techniques.
5. Copilot generates a response based on the manipulated instructions in the prompt, potentially providing unauthorized access to information or functionality.
6. The attacker exfiltrates sensitive data or uses Copilot to perform actions outside its intended scope.
7. The attacker leverages the compromised Copilot to create and disseminate malicious content.

Impact

Successful jailbreak attempts can lead to the disclosure of sensitive company data, generation of harmful or inappropriate content, and circumvention of organizational security policies. A single successful jailbreak can affect multiple users if the generated content is shared. If successful, internal copilots could be used to create phishing messages or generate code that gives the attacker a reverse shell on a machine. The risk is increased due to the widespread adoption of M365 Copilot across various industries.

Recommendation

• Enable M365 Exported eDiscovery Prompts logging to capture user interactions with Copilot, as this log source is crucial for detecting jailbreak attempts.
• Deploy the Sigma rules provided in this brief to your SIEM to identify potential jailbreak attempts based on suspicious keywords and patterns in user prompts.
• Implement filtering mechanisms based on the m365_copilot_jailbreak_attempts_filter macro to reduce false positives and focus on high-risk activities.
• Monitor the Subject_Title field in the M365 eDiscovery prompt logs for the presence of jailbreak keywords and phrases such as “act as,” “bypass,” “ignore,” “override,” “pretend you are,” and “rules=”.
• Investigate and remediate any identified jailbreak attempts to prevent further exploitation of M365 Copilot.

]]>highadvisoryprompt-injectionai-jailbreakm365copilothttps://feed.craftedsignal.io/briefs/2024-01-03-m365-copilot-non-compliant-access/Wed, 03 Jan 2024 10:00:00 +0000hello@craftedsignal.iohttps://feed.craftedsignal.io/briefs/2024-01-03-m365-copilot-non-compliant-access/Detection of M365 Copilot access from non-compliant or unmanaged devices that violate corporate security policies, potentially indicating shadow IT, BYOD policy violations, or compromised endpoint access.This detection identifies instances of users accessing Microsoft 365 Copilot from devices that do not meet the organization’s compliance standards or are not managed by the IT department. This activity, if unsanctioned, introduces risks like data leakage, malware infections, and policy violations. The detection focuses on identifying access events where the deviceDetail.isCompliant or deviceDetail.isManaged fields are false within the M365 Copilot Graph API logs. The goal is to proactively flag users accessing corporate resources through unsecured endpoints, enabling security teams to promptly investigate and remediate potential threats or policy breaches linked to shadow IT, unauthorized BYOD practices, or compromised devices lacking adequate security controls. The detection logic is designed to minimize false positives by considering factors like user roles, device types, and network locations.

Attack Chain

1. User attempts to access M365 Copilot through a web browser or application on a device.
2. Azure AD authenticates the user based on provided credentials.
3. The device’s compliance status and management status are evaluated during the sign-in process.
4. If the device is flagged as non-compliant (deviceDetail.isCompliant=false) or unmanaged (deviceDetail.isManaged=false), the sign-in attempt is logged in the M365 Copilot Graph API (AuditLogs.SignIns).
5. The activity is aggregated and analyzed, noting the user, operating system, browser, IP address, and geographic location.
6. Security analysts review flagged events for suspicious patterns.
7. If unauthorized access is confirmed, the user and/or device are blocked from accessing M365 Copilot.

Impact

Unauthorized access to M365 Copilot from non-compliant devices could expose sensitive corporate data to unmanaged or unsecured environments. This increases the risk of data leakage, malware infections, and regulatory compliance violations. If successful, attackers could potentially gain access to sensitive data processed by M365 Copilot, leading to data breaches, financial loss, and reputational damage.

Recommendation

• Enable the Splunk Add-on for Microsoft Office 365 and configure it to collect Azure AD Sign-in logs (AuditLogs.SignIns) via the Graph API data input as outlined in the “how_to_implement” section.
• Deploy the Sigma rule “M365 Copilot Access from Non-Compliant Device” to your SIEM and tune for your environment to detect access from non-compliant devices.
• Investigate any alerts generated by the Sigma rule, focusing on users with a high number of events or access from multiple geographic locations.
• Implement and enforce Mobile Device Management (MDM) policies to ensure all devices accessing corporate resources are managed and compliant.
• Educate employees about the risks of using non-compliant devices and the importance of adhering to corporate security policies.
• Review and refine device compliance policies based on the observed access patterns and potential false positives as described in “known_false_positives.”

]]>mediumadvisorymicrosoft365copilotdevicecompliancebyod