Описание
Argo Workflows: WorkflowTemplate Security Bypass via podSpecPatch in Strict/Secure Reference Mode
Summary
A user who can submit Workflows can completely bypass all security settings defined in a WorkflowTemplate by including a podSpecPatch field in their Workflow submission. This works even when the controller is configured with templateReferencing: Strict, which is specifically documented as a mechanism to restrict users to admin-approved templates. The podSpecPatch field on a submitted Workflow takes precedence over the referenced WorkflowTemplate during spec merging and is applied directly to the pod spec at creation time with no security validation.
Details
Three issues combine to create this vulnerability:
-
Merge priority order:
JoinWorkflowSpecmerges specs with the priority order Workflow Spec > WorkflowTemplate Spec > WorkflowDefault Spec. BecausepodSpecPatchis a plain string field, the Workflow's value replaces the WorkflowTemplate's value. -
No security validation on
podSpecPatch:ApplyPodSpecPatch()only validates that the patch is syntactically valid JSON conforming to the KubernetesPodSpecschema. No checks are performed for dangerous security settings such asprivileged: true. -
templateReferencing: Strictdoes not restrictpodSpecPatch: Strict mode only checks whetherWorkflowTemplateRefis set. If it is, the Workflow passes validation regardless of what other fields (includingpodSpecPatch) are present.
PoC
Prerequisites
A local Kubernetes cluster with Argo Workflows installed. The instructions below use kind.
1. Create a kind cluster and install Argo Workflows
Note: --server-side is required because some CRDs exceed the client-side annotation size limit.
Wait for the controller to be ready:
2. Enable templateReferencing: Strict
Patch the workflow controller configmap to enforce Strict mode:
Restart the controller to pick up the new config:
3. Verify Strict mode is active
Attempt to submit a standalone Workflow (no workflowTemplateRef). It should be rejected:
The Workflow will be accepted by the API server but the controller will reject it. After a few seconds, check its status:
Expected output:
4: Create a hardened WorkflowTemplate
An administrator creates a WorkflowTemplate with restrictive security settings:
5. Submit a legitimate Workflow (baseline)
Submit a Workflow that references the secure template without modification:
Wait for completion and check logs:
Expected output (confirming the template's security settings are applied):
6. Submit the bypass Workflow
Submit a Workflow that references the same secure template but includes a podSpecPatch that overrides all security settings:
Wait for completion and check logs:
Expected output (all security settings bypassed):
The file /etc/shadow is readable (root), the host filesystem is mounted and accessible, and the container runs as UID 0.
Impact
The purpose of templateReferencing: Strict is to restrict users to only execute admin-approved WorkflowTemplates. This is explicitly documented as a security feature:
You can typically further restrict what a user can do to just being able to submit workflows from templates using the workflow restrictions feature.
A user who can submit Workflows referencing approved templates can use podSpecPatch to:
- Run containers as root (
runAsUser: 0) - Enable privileged mode (
privileged: true) - Mount the host filesystem (
hostPathvolumes) - Share host PID/network/IPC namespaces (
hostPID,hostNetwork,hostIPC) - Add all Linux capabilities (
capabilities.add: ["ALL"])
This effectively grants the user full root access to the underlying Kubernetes node, regardless of what security constraints the admin configured in the WorkflowTemplate.
The templateReferencing feature was introduced in Argo Workflows v2.9.0 through PR #3149.
Mitigation
When templateReferencing: Strict or Secure is enabled, the controller should reject Workflows that include a podSpecPatch field when using workflowTemplateRef.
Without the codefix, deploying an admission controller (OPA/Gatekeeper, Kyverno) with policies that block dangerous pod settings (privileged, hostPID, hostNetwork, hostIPC, hostPath) on pods created by Argo Workflows.
Пакеты
github.com/argoproj/argo-workflows/v4
< 4.0.2
4.0.2
github.com/argoproj/argo-workflows/v3
< 3.7.11
3.7.11
github.com/argoproj/argo-workflows
>= 2.9.0
Отсутствует
Связанные уязвимости
A flaw was found in Argo Workflows. A user with privileges to submit workflows can bypass security settings defined in a WorkflowTemplate by including a `podSpecPatch` field in their workflow submission. This allows them to circumvent restrictions, even when `templateReferencing: Strict` is configured, potentially leading to unauthorized resource access or privilege escalation.
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. From 2.9.0 to before 4.0.2 and 3.7.11, A user who can submit Workflows can completely bypass all security settings defined in a WorkflowTemplate by including a podSpecPatch field in their Workflow submission. This works even when the controller is configured with templateReferencing: Strict, which is specifically documented as a mechanism to restrict users to admin-approved templates. The podSpecPatch field on a submitted Workflow takes precedence over the referenced WorkflowTemplate during spec merging and is applied directly to the pod spec at creation time with no security validation. This vulnerability is fixed in 4.0.2 and 3.7.11.