Skip to content

Conversation

@liranbg
Copy link
Member

@liranbg liranbg commented Jan 1, 2026

📝 Description

Remove runtime-level auth env var injection (enrich_auth_env) and centralize auth configuration through the client-spec mechanism upon import mlrun. Auth token file path is currently set directly during httpdb config sync when running inside k8s cluster.


🛠️ Changes Made

  • Removed enrich_auth_env function from mlrun/auth/utils.py
  • Set token_file path directly in httpdb.py during config sync when running inside k8s
  • Removed enrich_auth_env calls from base.py and nuclio/function.py
  • Removed related test that validated the old env injection behavior
  • Fixed patch_remote.py to handle tags with + in its mlrun version.

✅ Checklist

  • I updated the documentation (if applicable)
  • I have tested the changes in this PR
  • I confirmed whether my changes are covered by system tests
    • If yes, I ran all relevant system tests and ensured they passed before submitting this PR
    • I updated existing system tests and/or added new ones if needed to cover my changes
  • If I introduced a deprecation:

🧪 Testing

  • Unit tests: test_client_spec.py, test_kubejob.py, test_httpdb.py OAuth tests
  • Manual testing required on IG4 environment

🔗 References


🚨 Breaking Changes?

  • Yes (explain below)
  • No

🔍️ Additional Notes

Part of the IG4 AuthN initiative. OAuth token config (enabled flag, endpoint) continues to be set via client-spec sync. Token file path is now set directly when running inside k8s cluster instead of being injected per-runtime.

- Remove enrich_auth_env function from mlrun/auth/utils.py
- Set token_file path directly in httpdb.py during config sync when running inside k8s
- Remove enrich_auth_env calls from base.py and nuclio/function.py
- Remove related test that was validating the old env injection behavior

Resolves: ML-11898
@liranbg liranbg requested review from a team and moranbental as code owners January 1, 2026 20:07
@liranbg liranbg changed the title [SDK] [IG4] Autoset runtime igz4 oauth configuration Jan 1, 2026
@liranbg liranbg requested review from Copilot and removed request for a team and moranbental January 2, 2026 09:27
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR removes runtime-level authentication environment variable injection and centralizes auth configuration through the client-spec mechanism. The main architectural change moves OAuth token configuration setup to occur during the HTTPRunDB connection phase when running inside Kubernetes clusters, eliminating the need to inject auth env vars into each runtime.

Key Changes

  • Removed enrich_auth_env function that previously injected auth env vars into runtime environments
  • Centralized OAuth token configuration during HTTPRunDB connection, with automatic token_file path setting for Kubernetes environments
  • Fixed patch_remote.py to properly handle version tags containing + characters by preserving the original version string for MLRUN_VERSION while using a sanitized version for Docker image tags

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
mlrun/auth/utils.py Removed enrich_auth_env function and unused mlrun.common.constants import
mlrun/db/httpdb.py Added logic to auto-set token_file path when running in k8s cluster; now unconditionally sets auth_token_endpoint from server config
mlrun/runtimes/base.py Removed enrich_auth_env call and unused import from runtime environment generation
server/py/services/api/crud/runtimes/nuclio/function.py Removed enrich_auth_env call and unused import from Nuclio function environment resolution
tests/rundb/test_httpdb.py Updated test to mock read_secret_tokens_file instead of using token_file fixture; removed unused fixture parameter
server/py/services/api/tests/unit/runtimes/test_kubejob.py Removed test validating old env injection behavior and unused AuthenticationMode import
mlrun/run.py Removed unused filebase variable assignment
automation/patch_igz/patch_remote.py Fixed handling of version tags with + by preserving original for MLRUN_VERSION env var while sanitizing for Docker tags
.dockerignore Broadened pattern to match patch[_\-]env*.yml instead of just patch[_\-]env.yml
Comments suppressed due to low confidence (1)

mlrun/db/httpdb.py:657

  • The removal of the not config.auth_token_endpoint condition means that the auth_token_endpoint will now be unconditionally overwritten with the server-provided configuration during connect(), even if a user has explicitly configured this value before calling connect().

While this aligns with the PR's goal of centralizing auth configuration through the client-spec mechanism, it represents a behavior change where server-provided configuration always takes precedence over client-side configuration. If this is intentional (which it appears to be based on the PR description), consider adding a code comment to clarify that server configuration always overrides any pre-existing client configuration.

            if (
                config.httpdb.authentication.mode
                == mlrun.common.types.AuthenticationMode.IGUAZIO_V4.value
            ):
                # if running inside kubernetes, use the internal endpoint, otherwise use the external endpoint
                if mlrun.k8s_utils.is_running_inside_kubernetes_cluster():
                    config.auth_token_endpoint = server_cfg.get(
                        "oauth_internal_token_endpoint"
                    )
                else:
                    config.auth_token_endpoint = server_cfg.get(
                        "oauth_external_token_endpoint"
                    )

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +547 to +558
monkeypatch.setattr(
mlrun.auth.utils,
"read_secret_tokens_file",
lambda raise_on_error: {
"secretTokens": [
{
"name": "default",
"token": "offline",
}
]
},
)
Copy link

Copilot AI Jan 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding an assertion after line 558 to verify that config.auth_with_oauth_token.token_file is set to the expected path (/var/mlrun-secrets/auth/.igz.yml) when running inside the Kubernetes cluster. This would make the test more comprehensive and ensure that the token_file configuration is properly set as part of the OAuth initialization logic.

Copilot uses AI. Check for mistakes.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, below lines 574-580

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copy link
Contributor

@weilerN weilerN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 suggestion inline, beside that LGTM

Comment on lines +665 to +666
mlrun.common.constants.MLRUN_JOB_AUTH_SECRET_PATH,
mlrun.common.constants.MLRUN_JOB_AUTH_SECRET_FILE,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could combine these two constants into a single one that represents the full path, since there’s no clear need to call os.path.join on every invocation

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these consts are being used together and separately. joining them is upon demand is noop (painless)

Comment on lines -331 to -338
env["MLRUN_AUTH_WITH_OAUTH_TOKEN__ENABLED"] = "true"
env["MLRUN_AUTH_TOKEN_ENDPOINT"] = os.path.join(
mlrun.mlconf.iguazio_api_url,
mlrun.mlconf.httpdb.authentication.iguazio.authentication_endpoint,
)
env["MLRUN_HTTPDB__HTTP__VERIFY"] = str(
mlrun.mlconf.iguazio_api_ssl_verify
).lower()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How / where are these env vars being set then?
Or are they not needed anymore? unclear...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the intention of this pr.

no need to inject all these envvars because:

  • upon import mlrun, you do client-spec request which tells you "you are under iguazio 4" so all those enrichment? needless
  • this pr - handle the runtimes - so basically, all it needed is to get the mount point (which it has now)

@liranbg liranbg merged commit 13a7ab0 into mlrun:development Jan 4, 2026
17 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants