Llm

Malicious actors are actively developing and advertising tools for Indirect Prompt Injection (IDPI) on underground forums, leveraging hidden prompts within various mediums like emails, PDFs, calendar invites, and malvertising to manipulate Large Language Models (LLMs) and AI agents, potentially leading to unintended behaviors such as data exfiltration or bypassing content moderation systems.

This brief describes how the creation of Large Language Model (LLM) files, including formats like .gguf, .safetensors, .ggml, and Modelfiles, by local AI inference frameworks such as Ollama, llama.cpp, GPT4All, and LM Studio can be detected on Windows endpoints, indicating potential shadow AI deployments, unauthorized model downloads, or rogue LLM infrastructure which poses data exfiltration risks and policy violations.

A detection rule has been developed for Amazon Bedrock that identifies the deletion of guardrails, indicating a potential attempt by an attacker or insider to disable AI model safety controls and facilitate unsafe or unauthorized responses.

Elastic has developed a detection rule that monitors non-allowlisted `wget` activity on Linux hosts using Auditd Manager or Auditbeat, leveraging an Elastic LLM to triage `wget` executions for potential ingress tool transfer, command and control, or data exfiltration attempts to untrusted destinations, generating alerts only for high-confidence positive or suspicious verdicts.

Elastic's LLM-based detection rule identifies suspicious `curl` activity on Linux systems, aiming to detect command and control, data exfiltration, or ingress tool transfer by analyzing command-line parameters and network destinations via Auditd Manager or Auditbeat logs, which, if left unaddressed, could lead to system compromise or data breach.

A detection rule identifies AWS access key IDs, Amazon Bedrock API keys, PEM private-key blocks, and GitHub/GitLab tokens within Amazon Bedrock model prompts or completions, indicating a critical credential exposure event through misconfiguration, data leakage, or prompt injection that necessitates immediate secret rotation and investigation.

This rule detects prompts sent to Amazon Bedrock AgentCore runtimes that attempt to harvest credentials or exfiltrate data by referencing cloud instance metadata services, explicit AWS access/secret keys, or combining prompt-injection/jailbreak language with intent to reveal secrets or send data to external endpoints, indicating an attempt to weaponize the agent for credential theft.

Attackers can exploit CVE-2026-61447, a critical remote code execution vulnerability in PraisonAI versions before 1.6.78, by using prompt injection to manipulate LLM-generated Python code, leading to arbitrary code execution and exfiltration of environment secrets on the host system.

CrowdStrike's AI security research team has identified 18 new prompt injection techniques, expanding its taxonomy to over 200 methods, which enable adversaries to manipulate AI systems and agents through indirect means like hidden context, delayed triggers, and special token injection, leading to unauthorized actions such as data exfiltration or arbitrary command execution.

CrowdStrike's AI security research team has identified 18 new prompt injection techniques, expanding its taxonomy to over 200 methods, which adversaries can use to manipulate AI systems and agents through hidden context, delayed triggers, semantic constraints, boundary spoofing, and social engineering, potentially leading to agent hijacking, data exfiltration, or system compromise by causing them to execute unintended commands like shell scripts or SQL queries.

This brief outlines the detection of hostile or aggressive prompt sentiment sent to AWS Bedrock Claude large language models, indicating potential abuse, harassment, or attempts at model manipulation, requiring the configuration of Bedrock model invocation logging and Splunk ingestion.

Merged detection coverage for AWS Bedrock Claude abuse patterns, including prompt injection, sensitive-data exposure, high-risk tool invocation, cross-region inference, hostile prompts, unusually large prompts, and excessive token output anomalies.

Langroid is vulnerable to a critical Remote Code Execution (RCE) in its `TableChatAgent` and `VectorStore` components when `full_eval=True` due to CVE-2026-54769; the `eval()` function fails to properly scrub `__builtins__` from `globals`, allowing attackers to inject `__import__('os').system()` calls via crafted prompt payloads, leading to unauthenticated RCE, unauthorized data access, or system compromise on the host running the Langroid agent.

A critical vulnerability in PraisonAI's `multiedit` tool, affecting versions prior to 4.6.61, enables threat actors to achieve arbitrary file read and write capabilities by influencing LLM agent tool arguments, leading to sensitive data exfiltration and potential remote code execution.

A sandbox bypass vulnerability exists due to the `dangerouslyDisableSandbox` parameter being exposed as part of the BashTool input schema, allowing an untrusted LLM to bypass the sandbox for any command and achieve host-level code execution due to the default `allowUnsandboxedCommands: true` setting.

The /responses endpoint in Open WebUI's OpenAI router lacks access control, allowing authenticated users to bypass per-model access controls and interact with any configured model, potentially leading to denial of service, model theft, and access policy bypass.

This rule detects DNS queries to known Large Language Model (LLM) domains by unsigned binaries or common Windows scripting utilities, indicating potential command and control activity leveraging LLMs for dynamic actions on compromised systems.

The LiteLLM package was compromised and infected with credential-stealing code through a supply chain attack leveraging the Trivy vulnerability scanner.

This rule detects DNS queries to known Large Language Model (LLM) domains originating from unsigned binaries or common Windows scripting utilities, indicating potential malware command and control activity.

A remote code injection vulnerability (CVE-2026-5002) exists in PromtEngineer localGPT versions up to commit 4d41c7d1713b16b216d8e062e51a5dd88b20b054, allowing attackers to execute arbitrary code by manipulating the LLM Prompt Handler component via the _route_using_overviews function in backend/server.py.