Skip to content

Commit d26060a

Browse files
committed
fix: EksPodOperator 401 with cross-account AssumeRole via aws_conn_id
The kubeconfig exec plugin COMMAND template in EksHook had two critical fragility points that caused 401 Unauthorized when using cross-account AssumeRole credentials: 1. stderr was merged into stdout via 2>&1, so any Python warnings, deprecation notices, or log output from eks_get_token contaminated the stdout that bash token parsing relies on. This caused the last_line extraction to grab the wrong line, producing empty/ invalid timestamp and token values. 2. No validation that the token was successfully extracted. If parsing failed, a malformed ExecCredential JSON with an empty token was sent to the EKS API server, resulting in 401 with an empty user identity in the audit logs ("user":{}). Same-account usage worked by accident because default MWAA execution role credentials were already in the environment, so eks_get_token produced valid output regardless of credential file sourcing. Changes: - Redirect stderr to /dev/null (2>/dev/null) instead of merging with stdout (2>&1) to ensure clean token output for bash parsing - Add token validation: exit with error if token extraction fails - Add error messages to stderr for debugging credential issues - Add unit tests verifying the COMMAND template structure Fixes apache#64657
1 parent b91394a commit d26060a

2 files changed

Lines changed: 47 additions & 2 deletions

File tree

  • providers/amazon

‎providers/amazon/src/airflow/providers/amazon/aws/hooks/eks.py‎

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,20 +82,24 @@ class NodegroupStates(Enum):
8282
# Load credentials from secure file using (POSIX-compliant dot operator)
8383
. {credentials_file}
8484
85+
# Redirect stderr to /dev/null to prevent Python warnings, deprecation
86+
# notices, or other log output from contaminating stdout. The token
87+
# output must be the ONLY thing on stdout for bash parsing to work.
8588
output=$({python_executable} -m airflow.providers.amazon.aws.utils.eks_get_token \
86-
--cluster-name {eks_cluster_name} --sts-url '{sts_url}' {args} 2>&1)
89+
--cluster-name {eks_cluster_name} --sts-url '{sts_url}' {args} 2>/dev/null)
8790
8891
status=$?
8992
9093
# Clear environment variables after use (defense in depth)
9194
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
9295
9396
if [ "$status" -ne 0 ]; then
94-
printf '%s' "$output" >&2
97+
printf 'eks_get_token failed with exit code %s' "$status" >&2
9598
exit "$status"
9699
fi
97100
98101
# Use pure bash below to parse so that it's posix compliant
102+
# Only the token line should be on stdout (stderr redirected above)
99103
100104
last_line=${{output##*$'\\n'}} # strip everything up to the last newline
101105
@@ -104,6 +108,16 @@ class NodegroupStates(Enum):
104108
105109
token=${{last_line##*, token: }} # text after ", token: "
106110
111+
# Validate that token was extracted — empty token means parsing failed
112+
# or eks_get_token produced unexpected output. Without this check, a
113+
# malformed ExecCredential is sent to the API server, resulting in a
114+
# 401 with an empty user identity in the audit logs.
115+
if [ -z "$token" ]; then
116+
printf 'Failed to extract token from eks_get_token output. ' >&2
117+
printf 'Output was: %s' "$output" >&2
118+
exit 1
119+
fi
120+
107121
json_string=$(printf '{{"kind": "ExecCredential","apiVersion": \
108122
"{authentication_api_version}","spec": {{}},"status": \
109123
{{"expirationTimestamp": "%s","token": "%s"}}}}' "$timestamp" "$token")

‎providers/amazon/tests/unit/amazon/aws/hooks/test_eks.py‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1273,6 +1273,37 @@ def test_generate_config_file(self, mock_conn, aws_conn_id, region_name, expecte
12731273
if expected_region_args:
12741274
assert expected_region_args in command_arg
12751275

1276+
def test_command_template_redirects_stderr(self):
1277+
"""Verify COMMAND template redirects stderr to /dev/null to prevent
1278+
Python warnings/log output from contaminating stdout and breaking
1279+
bash token parsing. This is critical for cross-account AssumeRole
1280+
scenarios where the kubeconfig exec plugin must produce a clean token."""
1281+
from airflow.providers.amazon.aws.hooks.eks import COMMAND
1282+
1283+
# Verify stderr is redirected to /dev/null, not merged with stdout
1284+
assert "2>/dev/null" in COMMAND, (
1285+
"COMMAND must redirect stderr to /dev/null to prevent output contamination"
1286+
)
1287+
assert "2>&1" not in COMMAND, (
1288+
"COMMAND must not use 2>&1 — merging stderr with stdout breaks bash token parsing"
1289+
)
1290+
1291+
def test_command_template_validates_token(self):
1292+
"""Verify COMMAND template validates that the token was successfully
1293+
extracted before producing the ExecCredential JSON. Without this check,
1294+
a malformed ExecCredential with an empty token is sent to the API server,
1295+
resulting in 401 Unauthorized with an empty user identity in audit logs."""
1296+
from airflow.providers.amazon.aws.hooks.eks import COMMAND
1297+
1298+
# Verify token validation check exists
1299+
assert 'if [ -z "$token" ]' in COMMAND, (
1300+
"COMMAND must validate that token is non-empty before producing ExecCredential JSON"
1301+
)
1302+
# Verify it exits with error on empty token
1303+
assert "exit 1" in COMMAND or 'exit "$' in COMMAND, (
1304+
"COMMAND must exit with error when token extraction fails"
1305+
)
1306+
12761307

12771308
# Helper methods for repeated assert combinations.
12781309
def assert_all_arn_values_are_valid(expected_arn_values, pattern, arn_under_test) -> None:

0 commit comments

Comments
 (0)