Skip to content

fix: issues preventing deployment on Azure AKS#85

Merged
thesuperzapper merged 1 commit into
deployKF:mainfrom
thesuperzapper:fix-issues-preventing-deployment-on-azure
Jan 28, 2024
Merged

fix: issues preventing deployment on Azure AKS#85
thesuperzapper merged 1 commit into
deployKF:mainfrom
thesuperzapper:fix-issues-preventing-deployment-on-azure

Conversation

@thesuperzapper

@thesuperzapper thesuperzapper commented Jan 27, 2024

Copy link
Copy Markdown
Member

closes: #61

This PR allows deployKF to be used on Azure AKS by resolving issues with the Azure ASK - Admission Enforcer:

  • Removes the control-plane: kubeflow label from the kubeflow namespace.

  • Introduces the kubernetes.azure.admissionsEnforcerFix value, which when set to true, applies the "admissions.enforcer/disabled": "true" annotation to webhooks in the following components:

    • deploykf-dependencies:
      • Cert-Manager / Trust-Manager
      • Istio:
        • Because we use the Istio helm chart, there is no way to apply these labels, so we instead just ignore diffs under namespaceSelector in ArgoCD, so that it will at least not show as "out of sync" perpetually.
      • Kyverno
    • kubeflow-tools:
      • Pipelines
      • Katib
      • PodDefaults-Webhook

@thesuperzapper thesuperzapper added this to the v0.1.4 milestone Jan 27, 2024
@thesuperzapper thesuperzapper changed the title fix: issues preventing deployment on azure fix: issues preventing deployment on Azure AKS Jan 28, 2024
@thesuperzapper
thesuperzapper merged commit ed12e29 into deployKF:main Jan 28, 2024
@thesuperzapper
thesuperzapper deleted the fix-issues-preventing-deployment-on-azure branch January 28, 2024 01:05
@thesuperzapper thesuperzapper mentioned this pull request Jun 25, 2024
Closed
3 tasks
@harsha1163

harsha1163 commented Jul 1, 2024 •
edited
Loading

Copy link
Copy Markdown

@thesuperzapper, updated the values as per your suggestion. while trying to sync the applications using shell script, sync is getting failed with the errors.

As mentioned above, I validated the logs of the kyverno pod,they are in failed state due to OOM. Restarted the pods then after that tried syncing.

kyverno.io                    ClusterPolicy                 kubeflow                 clone-profile-bucket-secret-73d5e153-            Failed     SyncFailed        error when replacing "/dev/shm/1808297136": Internal error occurred: failed calling webhook "validate-policy.kyverno.svc": failed to call webhook: Post "https://kyverno-svc.kyverno.svc:443/policyvalidate?timeout=10s": EOF
kyverno.io                    ClusterPolicy                 kubeflow                 clone-profile-bucket-secret-ecfa6175-             Failed     SyncFailed        error when replacing "/dev/shm/3900048159": Internal error occurred: failed calling webhook "validate-policy.kyverno.svc": failed to call webhook: Post "https://kyverno-svc.kyverno.svc:443/policyvalidate?timeout=10s": http: server gave HTTP response to HTTPS client
kyverno.io                    ClusterPolicy                 kubeflow                 clone-profile-bucket-secret-8316e1ca-            Failed     SyncFailed        error when replacing "/dev/shm/1560309060": Internal error occurred: failed calling webhook "validate-policy.kyverno.svc": failed to call webhook: Post "https://kyverno-svc.kyverno.svc:443/policyvalidate?timeout=10s": http: server gave HTTP response to HTTPS client
kyverno.io                    ClusterPolicy                 kubeflow                 clone-profile-bucket-secret-fbb434e2-           Failed     SyncFailed        error when replacing "/dev/shm/2193344795": Internal error occurred: failed calling webhook "validate-policy.kyverno.svc": failed to call webhook: Post "https://kyverno-svc.kyverno.svc:443/policyvalidate?timeout=10s": http: server gave HTTP response to HTTPS client

@thesuperzapper

thesuperzapper commented Jul 1, 2024

Copy link
Copy Markdown
Member Author

@harsha1163 can you please raise a new issue so we can debug it?

@harsha1163

harsha1163 commented Jul 2, 2024

Copy link
Copy Markdown

I am sorry, I pinged here instead of adding a new comment in other issue #174

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog/fix changelog/significant

Projects

None yet

Milestone

v0.1.4

Development

Successfully merging this pull request may close these issues.

deploykf does not work properly on AKS due to webhook "admission enforcer"

2 participants

@thesuperzapper @harsha1163