Skip to content

util: support load Colon-separated KUBECONFIG#761

Merged
AkihiroSuda merged 1 commit intodocker:masterfrom
morlay:kubeconfig-enhance
Sep 21, 2021
Merged

util: support load Colon-separated KUBECONFIG#761
AkihiroSuda merged 1 commit intodocker:masterfrom
morlay:kubeconfig-enhance

Conversation

@morlay
Copy link
Copy Markdown
Collaborator

@morlay morlay commented Sep 8, 2021

fix #437 #760

@morlay morlay force-pushed the kubeconfig-enhance branch from 215ef92 to c838121 Compare September 8, 2021 09:53
morlay
morlay commented Sep 8, 2021
View reviewed changes
Comment thread commands/util.go
}
return clientcmd.NewDefaultClientConfig(*apiConfig, &clientcmd.ConfigOverrides{}), nil
}
return kubernetes.ConfigFromContext(endpointName, s)
Copy link
Copy Markdown
Collaborator Author

@morlay morlay Sep 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we still need this?

github.com/docker/cli/cli/context/kubernetes not provide a way to load Colon-separated KUBECONFIG
now changes could support all KUBECONFIG, and will not execute this line any more

@Dentrax
Copy link
Copy Markdown
Contributor

Dentrax commented Sep 8, 2021 

$ echo $KUBECONFIG
:/Users/furkan.turkal/.kube/config:/Users/furkan.turkal/.kube/foo:/Users/furkan.turkal/.kube/bar:/Users/furkan.turkal/.kube/baz

This returns me the following error for this commit:

Error: could not read kubeconfig: stat : no such file or directory

@morlay
Copy link
Copy Markdown
Collaborator Author

morlay commented Sep 8, 2021

@Dentrax

this pr should fix it.
the new kubeconfig load follow kubectl did

@morlay
Copy link
Copy Markdown
Collaborator Author

morlay commented Sep 8, 2021

why your kubeconfig value startswith :

@Dentrax
Copy link
Copy Markdown
Contributor

Dentrax commented Sep 8, 2021

why your kubeconfig value startswith :

🤷‍♂️I removed : prefix and still getting same error. 🤔

@morlay
Copy link
Copy Markdown
Collaborator Author

morlay commented Sep 8, 2021 •
edited
Loading 

$ docker buildx create --name=kubeconfig --driver=kubernetes --driver-opt=namespace=gitlab
kubeconfig
$ docker buildx inspect kubeconfig                                                        
Name:   kubeconfig
Driver: kubernetes

Nodes:
Name:      kubeconfig0
Endpoint:  kubernetes:///kubeconfig?deployment=&kubeconfig=%2FUsers%2Fmorlay%2F.kube%2Fconfig--hw-dev.yaml%3A%2FUsers%2Fmorlay%2F.kube%2Fconfig--hw-infra.yaml%3A%2FUsers%2Fmorlay%2F.kube%2Fconfig--hw-sg.yaml%3A%2FUsers%2Fmorlay%2F.kube%2Fconfig--qc-hk.yaml
Status:    inactive
Platforms: 
$ docker buildx inspect kubeconfig --bootstrap
[+] Building 21.9s (1/1) FINISHED                                                                                                                                           
 => [internal] booting buildkit                                                                                                                                       21.8s
 => => waiting for 1 pods to be ready                                                                                                                                 21.5s
Name:   kubeconfig
Driver: kubernetes

Nodes:
Name:      kubeconfig0-787cbbd5d9-4fpf7
Endpoint:  
Status:    running
Platforms: linux/amd64, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Works well for me.

@tonistiigi
Copy link
Copy Markdown
Member

tonistiigi commented Sep 8, 2021

Should we remove empty files?

@morlay
Copy link
Copy Markdown
Collaborator Author

morlay commented Sep 9, 2021 •
edited
Loading

Should we remove empty files?

kubectl didn't, so i don't think we should add file exists checking.

For me, my KUBECONFIG set like this to make sure files exists.

export KUBECONFIG=$(echo $(ls ~/.kube/config--*.yaml) | sed 's/ /:/g')

@Dentrax
Copy link
Copy Markdown
Contributor

Dentrax commented Sep 9, 2021

@morlay Just applied your pull patch in my own fork and it seems working now but:

$ kubectl config current-context
kind-cluster-api-test
$ go run cmd/buildx/main.go create --name=kubeconfig --driver=kubernetes --driver-opt=namespace=gitlab
kubeconfig
$ go run cmd/buildx/main.go inspect kubeconfig
Name:   kubeconfig
Driver: kubernetes

Nodes:
Name:      kubeconfig0-787cbbd5d9-d2d9w
Endpoint:
Status:    running
Platforms: linux/amd64, linux/386

But getting the following issue and not sure if it's related to this, any ideas?

$ cat ~/.docker/buildx/instances/kubeconfig
{"Name":"kubeconfig","Driver":"kubernetes","Nodes":[{"Name":"kubeconfig0","Endpoint":"kubernetes:///kubeconfig?deployment=\u0026kubeconfig=%3A%2FUsers%2Ffurkan.turkal%2F.kube%2Fconfig%3A%2FUsers%2Ffurkan.turkal%2F.kube%2Ffoo%3A%2FUsers%2Ffurkan.turkal%2F.kube%2Fbar%3A%2FUsers%2Ffurkan.turkal%2F.kube%2Fbaz%3A%2FUsers%2Ffurkan.turkal%2Fauth%2Fk8s-auto%2FS_PLATFORM_P1_6PLATFORM_MOON","Platforms":null,"Flags":null,"ConfigFile":"","DriverOpts":{"namespace":"gitlab"}}],"Dynamic":false}

$ kubectl config get-contexts
CURRENT   NAME                                            CLUSTER                        AUTHINFO           NAMESPACE
*         kubernetes-admin@s-platform-p1-6platform-moon   s-platform-p1-6platform-moon   kubernetes-admin

$ kubectl cluster-info
Kubernetes control plane is running at ...
Metrics-server is running at ...
...

$ kubectl get pods -n gitlab
NAME                           READY   STATUS    RESTARTS   AGE
kubeconfig0-787cbbd5d9-d2d9w   1/1     Running   0          27m

$ ./buildx build -t test:simple-golang-app -f Dockerfile .
[+] Building 0.0s (0/0)
Error: no valid drivers found: cannot determine Kubernetes namespace, specify manually: invalid configuration: [context was not found for specified context: kubernetes-admin@s-platform-p1-6platform-moon, cluster has no server defined]

@morlay
Copy link
Copy Markdown
Collaborator Author

morlay commented Sep 9, 2021 •
edited
Loading

@Dentrax

Why your kubectl config current-context not match kubectl config get-contexts ?

This * should point like

$ kubectl config current-context
hw-dev
$ kubectl config get-contexts
CURRENT   NAME       CLUSTER    AUTHINFO   NAMESPACE
*         hw-dev     hw-dev     user1      
          hw-infra   hw-infra   hw-infra   
          hw-sg      hw-sg      hw-sg      
          qc-hk      default    default

@tonistiigi
Copy link
Copy Markdown
Member

tonistiigi commented Sep 20, 2021

@AkihiroSuda still lgty?

@AkihiroSuda AkihiroSuda merged commit 2b18a9b into docker:master Sep 21, 2021
@Dentrax Dentrax mentioned this pull request Oct 24, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AkihiroSuda AkihiroSuda AkihiroSuda approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

error when KUBECONFIG contains more than one file

4 participants

@morlay @Dentrax @tonistiigi @AkihiroSuda