Argo Workflows Controller Denial-of-Service via Malformed Pod Annotation
A malformed `workflows.argoproj.io/pod-gc-strategy` annotation in an Argo Workflow pod can trigger an unchecked array index in the `podGCFromPod()` function, leading to a controller-wide panic and denial-of-service.
Argo Workflows is vulnerable to a denial-of-service attack where a malformed workflows.argoproj.io/pod-gc-strategy annotation within a workflow pod can crash the Argo Workflows controller. This vulnerability stems from an unchecked array index in the podGCFromPod() function. When the annotation value lacks a “/”, the strings.Split function returns an array of length 1, leading to an out-of-bounds access when trying to retrieve the second element. The resulting panic occurs outside the controller’s recovery scope, causing the entire controller process to terminate. The affected versions include 3.6.5 through 3.6.19, 3.7.0-rc1 through 3.7.12, and 4.0.0-rc1 through 4.0.3. This vulnerability was introduced in commit #14129.
Attack Chain
- An attacker crafts a malicious Argo Workflow YAML file.
- The YAML includes a
podMetadatasection defining annotations for the workflow pod. - Within the annotations, the
workflows.argoproj.io/pod-gc-strategykey is set to a value that does not contain a forward slash ("/"), such as “NoSlash”. - The attacker submits the crafted workflow to the Argo Workflows controller using
kubectl apply -n argo -f malicious-workflow.yaml. - The Argo Workflows controller receives the workflow definition and creates a corresponding pod based on the specification.
- The
podGCFromPod()function in/workflow/controller/pod/controller.goattempts to parse theworkflows.argoproj.io/pod-gc-strategyannotation. - The
strings.Splitfunction splits the annotation value, resulting in an array with only one element. - The code attempts to access
parts[1], causing a panic due to an out-of-bounds array access and crashes the controller, resulting in a denial-of-service.
Impact
Successful exploitation of this vulnerability allows any user with the ability to submit workflows to crash the Argo Workflows controller. The controller will enter a crash loop, rendering the entire Argo Workflows deployment unavailable. Since the controller is responsible for managing and executing workflows, all workflow processing is halted, leading to a denial-of-service condition. This can severely impact organizations relying on Argo Workflows for their CI/CD pipelines or other automated tasks. The attacker requires only create permission on Workflow resources to execute this attack.
Recommendation
- Upgrade to a patched version of Argo Workflows (v3.6.4 or earlier, v3.6.20+, v3.7.13+, or v4.0.4+) to remediate the vulnerability as described in GHSA-5jv8-h7qh-rf5p.
- Implement input validation on workflow submissions to reject workflows with malformed
workflows.argoproj.io/pod-gc-strategyannotations. See the PoC workflow example provided in GHSA-5jv8-h7qh-rf5p for examples of vulnerable annotation values. - Deploy the Sigma rule
Detect Argo Workflows Malformed Pod GC Annotationto detect workflow submissions containing potentially malicious annotations.
Detection coverage 2
Detect Argo Workflows Malformed Pod GC Annotation
highDetects the submission of Argo Workflows with a malformed workflows.argoproj.io/pod-gc-strategy annotation that can cause a denial-of-service.
Detect Argo Workflows Controller CrashLoopBackOff
mediumDetects when the Argo Workflows controller pod enters a CrashLoopBackOff state, potentially due to the malformed pod GC annotation vulnerability.
Detection queries are kept inside the platform. Get full rules →