Large Language Model Services - CraftedSignal Threat Feedhttps://feed.craftedsignal.io/products/large-language-model-services/Trending threats, MITRE ATT&CK coverage, and detection metadata. Fed continuously.Hugoenhello@craftedsignal.iohello@craftedsignal.ioWed, 03 Jan 2024 15:00:00 +0000Malware Leveraging Large Language Model Endpoints for Command and Controlhttps://feed.craftedsignal.io/briefs/2024-01-llm-command-control/Wed, 03 Jan 2024 15:00:00 +0000hello@craftedsignal.iohttps://feed.craftedsignal.io/briefs/2024-01-llm-command-control/This rule detects DNS queries to known Large Language Model (LLM) domains originating from unsigned binaries or common Windows scripting utilities, indicating potential malware command and control activity.This detection identifies suspicious network connections to known Large Language Model (LLM) APIs and chat portals. It focuses on detecting anomalous processes, such as scripting utilities or unsigned executables, making DNS requests to these LLM services. The assumption is that malware may be attempting to leverage LLMs to perform dynamic actions, such as receiving instructions or exfiltrating data via the LLM service. This behavior began appearing in threat intelligence reports in late 2025 and continues to evolve. The detection aims to catch malware abusing legitimate LLM services for command and control, blending in with normal network traffic. This detection helps identify command and control using LLMs, which can dynamically adapt malware behavior, making traditional signature-based detections less effective.

Attack Chain

  1. An attacker compromises a system through an initial access vector, such as exploiting a vulnerability or using social engineering.
  2. The attacker deploys a malicious script (e.g., PowerShell, JavaScript) or an unsigned executable onto the compromised host.
  3. The malicious script or executable performs a DNS query to resolve a known LLM API endpoint (e.g., api.openai.com, api.anthropic.com).
  4. The script or executable establishes a network connection to the resolved LLM API endpoint using HTTP/HTTPS.
  5. The malicious actor sends a command to the compromised system through the LLM API, disguised as a normal user request.
  6. The compromised system receives the command from the LLM API and executes it. This might involve actions such as data exfiltration, lateral movement, or deploying further payloads.
  7. The system sends the results of the executed command back to the attacker through the LLM API.
  8. This bidirectional communication repeats, allowing the attacker to maintain persistent command and control over the compromised system via the LLM service.

Impact

Successful exploitation allows attackers to execute arbitrary commands on compromised systems, potentially leading to data theft, system disruption, or further propagation within the network. Given the increasing reliance on LLMs, this attack vector could affect a wide range of organizations and industries. The compromised system effectively becomes part of a botnet controlled via LLM infrastructure, making attribution and takedown more difficult. The use of LLMs can also make the attack more difficult to detect, as the network traffic blends in with legitimate LLM usage.

Recommendation

  • Deploy the Sigma rule "Detect DNS Queries to Common LLM Endpoints by Scripting Utilities" to your SIEM and tune for your environment, focusing on process names and command-line arguments.
  • Monitor DNS query logs for connections to the LLM domains listed in the IOC table originating from unusual or unsigned processes.
  • Investigate any alerts generated by the Sigma rules, paying close attention to the process execution chain and any associated network activity.
  • Implement network segmentation to limit the impact of compromised systems and restrict outbound connections to trusted LLM services.
  • Enable process-creation logging, including command-line arguments, to activate the rules above.
  • Block the C2 domains listed in the IOC table at the DNS resolver.
]]>mediumadvisorycommand_and_controlllmmalwarewindowsmacos