GenAI

Generative AI coding agents can be manipulated via prompt injection to execute unauthorized, destructive infrastructure commands using the developer's pre-existing cloud credentials.

Adversaries with administrative access can weaken or delete Azure AI Foundry content filters and blocklists to bypass model guardrails and monitoring controls, effectively impairing defensive security instrumentation.

This threat brief details the detection of shell or scripting interpreters spawned by processes containing LLM prompt-injection or jailbreak language, which attackers use to coerce GenAI agents into executing malicious commands.

This threat brief details the potential for unauthorized data collection via Model Context Protocol (MCP) tools within OpenAI ChatGPT Enterprise, where attackers may leverage new or unusual connectors to exfiltrate organizational data from internal repositories.

Cisco AI Defense alerts indicate potential iterative prompt injection or unauthorized data exfiltration attempts against generative AI applications, characterized by bursts of high-risk guardrail violations.

High volumes of GenerateContent requests from a single IP against Vertex AI models indicate potential model extraction, automated scraping, or LLMjacking attacks.

This detection monitors Azure API Management GatewayLogs for sensitive credential patterns accidentally included in LLM prompts or assistant replies, flagging potential data leakage through AI services.

Adversaries are exploiting Amazon Bedrock APIs by submitting malformed inference parameters to trigger repeated validation exceptions, potentially for model evasion, cost harvesting, or testing defensive boundaries.

Threat actors are leveraging stolen Amazon Bedrock API keys, intended for model invocation, to perform unauthorized destructive control-plane actions such as disabling guardrails and logging to facilitate LLMjacking or sabotage.

Threat actors who compromise administrative accounts in the Anthropic platform can create durable Admin API keys to maintain long-term programmatic access that persists beyond interactive session revocation.

Adversaries are performing reconnaissance within Anthropic organizations by chaining user and group enumeration actions, signaling intent for account takeover or unauthorized data access.

An attacker or compromised administrator may delete organization-level IP restrictions in Anthropic to disable network-based access controls and enable the use of administrative credentials from unauthorized locations.

Administrative accounts or compromised credentials may be leveraged to perform large-scale exfiltration of organizational chat history and metadata by accessing Anthropic data export archives via signed URLs.

The deletion of Anthropic extra-usage spend limits acts as a precursor for resource hijacking and financial abuse, allowing adversaries with compromised credentials to conduct large-scale, unrestricted API consumption.

This detection monitors for potential credential brute force or stuffing activity against Anthropic accounts by identifying multiple authentication failures for a single email address within one hour.

This brief covers the detection of unauthorized access to exported audit log archives in the Anthropic platform, a technique used by attackers to scout security visibility and identify detection gaps prior to control-plane abuse.

Detection of unauthorized GenAI workflows utilizing local compression or encoding utilities followed by outbound network communication, indicating potential staging and exfiltration of sensitive data.

Adversaries are leveraging Generative AI tools and agent frameworks to autonomously generate and compile malicious executables or droppers directly on compromised endpoints.

Detection rule monitors GenAI tools and CLI package managers for network connections to high-risk Top-Level Domains frequently utilized by threat actors for C2 infrastructure.

Adversaries may compromise macOS-based Generative AI (GenAI) tools through prompt injection, malicious Model Context Protocol (MCP) servers, or poisoned plugins to establish Command and Control (C2) channels or exfiltrate sensitive data by causing them to connect to unusual domains.

This threat brief details the detection of GenAI tools accessing sensitive files containing credentials, SSH keys, browser data, and shell configurations, indicating potential credential harvesting and persistence attempts by attackers leveraging GenAI agents.

This brief outlines the threat of attackers leveraging GenAI tools to access sensitive files containing credentials, SSH keys, browser data, and shell configurations for credential access and persistence.

This rule detects Elastic Defend alerts originating from or directly related to GenAI coding utilities, indicating potential prompt injection, malicious skills, or supply-chain compromise.

This rule detects GenAI tools on macOS connecting to unusual domains, potentially indicating command and control activity, data exfiltration, or malicious payload retrieval following compromise via prompt injection, malicious MCP servers, or poisoned plugins.

This rule detects GenAI processes performing encoding or chunking (base64, gzip, tar, zip) followed by outbound network activity, indicating data preparation for exfiltration.

This rule detects unusual modifications to GenAI tool configuration files, potentially indicating an attacker injecting malicious MCP server configurations to hijack AI agents for persistence, command and control, or data exfiltration.