You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
kustomize 3.5.5 breaks installing ArgoCD from URL #2538
I think the issue is different from #2444. Kustomize has changed to use go-getter by default. From the documentation of go-getter:
If you want to download only a specific subdirectory from a downloaded directory, you can specify a subdirectory after a double-slash //. go-getter will first download the URL specified before the double-slash (as if you didn't specify a double-slash), but will then copy the path after the double slash into the target directory.
So only manifests/ha/cluster-install will be copied and ../namespace-install is not there. You can try to replace // with /.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@Shell32-Natsu That does seem to make a difference. I'll try to make this change to upstream argo-cd install docs so that it's hopefully avoided for others. Though I imagine this might be an issue for other remote bases/overlays that people could be using.
@jameshochadel Hi James, thanks for the feedback. It looks like this is a go-getter issue. I have created issue here.
For your case, the problem is go-getter has bug with sub directory. Adding https works since go-getter fails with https scheme as well (because it's ambiguous I guess) and a normal git clone happens.
And the reason for why @chancez can fix the issue by replace // with / is go-getter doesn't work when you are using/andref at the same time. So actually there is a normal git clone as well.
I am surprised that go-getter has these problems. I think we may need to clearly document the url pattern if we want to continue using go-getter.
Dropping go-getter will probably break the backward compatibility to current version.
Aside from archive downloads and urls that use the depth query parameter, I don't think any url will be broken if go-getter is removed.
Either way though, I do think kustomize needs to be clear what the supported urls are. If the official position is that only go-getter urls are supported, then we would have just dropped the old cloner entirely but that didn't pan out. Some people have suggested removing go-getter but even then, there should be a spec as to what urls are supported since it accepts way too many variations and any kind of refactoring would be difficult without breaking someone's workflow. The repospec code is very convoluted and it's difficult to tell what kinds of urls it accepts and how it transforms them.
As a user, I just want to know what url formats are supported and that I can reasonably expect to never break and that kustomize will immediately fix if broken.
I had this issue on ArgoCD 2.0.3 with Kustomize 3.9.4
I solved it by just replacing the base url double slashes // from the docs with a single slash / in my kustomization.yaml- I tried this with and without https:// protocol prefix and both worked.
EDIT: upgrading Kustomize to 4 in argocd-repo-server also solves this problem - I include this patch in my kustomizaton.yaml to get a specific version of Kustomize now.
Are you sure it's not working for kustomize 4.1.3? I tried it locally and both urls work. Pre version 4, you might be hitting the go-getter path which is known to be broken for certain urls.
It worked locally in Kustomize 4.1.3 but not in ArgoCD, even when I upgraded the Kustomize version inside the container via an init container shared emptyDir as seen in the patch I published above. I even exec'd into the argocd-server container and did a Kustomize version and it showed 4.1.3
Closing since this is about a very old version of Kustomize, and comments indicate it was caused by a dependency we no longer have.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Error: accumulating resources: accumulation err='accumulating resources from 'github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2': evalsymlink failure on ...
no such file or directory': hit 27s timeout running '/usr/bin/git fetch --depth=1 origin v2.5.2'
Activity
[-]Kustomize 3.5.5 breaks installing ArgoCD from URL[/-][+]kustomize 3.5.5 breaks installing ArgoCD from URL[/+]chancez commentedon May 26, 2020
May be related to #2494
monopole commentedon May 27, 2020
likely a dupe of #2444. Fix is in #2537, which is in v3.6.1
please give it a try. reopen if it doesn't work.
@Shell32-Natsu FYI
chancez commentedon May 27, 2020
This doesn't seem resolved:
chancez commentedon May 27, 2020
And I cannot seem to re-open this issue either.
Shell32-Natsu commentedon May 27, 2020
I think the issue is different from #2444. Kustomize has changed to use go-getter by default. From the documentation of go-getter:
So only
manifests/ha/cluster-installwill be copied and../namespace-installis not there. You can try to replace//with/.seans3 commentedon May 27, 2020
Re-opening so that we can triage it.
/reopen
k8s-ci-robot commentedon May 27, 2020
@seans3: Reopened this issue.
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
chancez commentedon May 27, 2020
@Shell32-Natsu That does seem to make a difference. I'll try to make this change to upstream argo-cd install docs so that it's hopefully avoided for others. Though I imagine this might be an issue for other remote bases/overlays that people could be using.
monopole commentedon May 28, 2020
Thanks @chancez and @seans3, and thanks @Shell32-Natsu for taking this on.
jameshochadel commentedon Jun 1, 2020
I am encountering this issue on 3.6.1, but I am not using
//anywhere in my URLs. However, the issue only occurs when the protocol is not specified.E.g., where
my-repo/dev/kustomization.yamlreferences a resource../base, this fails:but this succeeds:
This was not previously (before 3.6.1) the case.
Shell32-Natsu commentedon Jun 1, 2020
@jameshochadel Hi James, thanks for the feedback. It looks like this is a go-getter issue. I have created issue here.
For your case, the problem is go-getter has bug with sub directory. Adding
httpsworks since go-getter fails withhttpsscheme as well (because it's ambiguous I guess) and a normal git clone happens.And the reason for why @chancez can fix the issue by replace
//with/is go-getter doesn't work when you are using/andrefat the same time. So actually there is a normal git clone as well.I am surprised that go-getter has these problems. I think we may need to clearly document the url pattern if we want to continue using go-getter.
19 remaining items
yujunz commentedon Nov 18, 2020
It was tested against the examples in https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md#url-format . But I roughly remembered the example tests were once excluded from CI. Not sure if it is covered now or not.
See also some history conversion in the two PRs for the problems go-getter tries to resolve and the tradeoffs
thatsmydoing commentedon Nov 19, 2020
Aside from archive downloads and urls that use the
depthquery parameter, I don't think any url will be broken ifgo-getteris removed.Either way though, I do think kustomize needs to be clear what the supported urls are. If the official position is that only
go-getterurls are supported, then we would have just dropped the old cloner entirely but that didn't pan out. Some people have suggested removing go-getter but even then, there should be a spec as to what urls are supported since it accepts way too many variations and any kind of refactoring would be difficult without breaking someone's workflow. The repospec code is very convoluted and it's difficult to tell what kinds of urls it accepts and how it transforms them.As a user, I just want to know what url formats are supported and that I can reasonably expect to never break and that kustomize will immediately fix if broken.
yujunz commentedon Nov 19, 2020
Yeah. It should be documented and covered by integration test.
We may need an exhaustive list in https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md#url-format which is covered by https://github.com/kubernetes-sigs/kustomize/blob/master/hack/testExamplesAgainstKustomize.sh
yujunz commentedon Nov 19, 2020
The test seems triggered by https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-sigs/kustomize/kustomize-presubmit-master.yaml before submit any PR to master.
yujunz commentedon Nov 20, 2020
I just realized the tests in remote build were skipped because of an extra blank line...
Proposed a fix in #3253
HariSekhon commentedon Jun 23, 2021
I had this issue on ArgoCD 2.0.3 with Kustomize 3.9.4
I solved it by just replacing the base url double slashes
//from the docs with a single slash/in mykustomization.yaml- I tried this with and withouthttps://protocol prefix and both worked.kustomization.yaml:
EDIT: upgrading Kustomize to 4 in argocd-repo-server also solves this problem - I include this patch in my kustomizaton.yaml to get a specific version of Kustomize now.
thatsmydoing commentedon Jun 23, 2021
Are you sure it's not working for kustomize 4.1.3? I tried it locally and both urls work. Pre version 4, you might be hitting the go-getter path which is known to be broken for certain urls.
HariSekhon commentedon Jun 23, 2021
It worked locally in Kustomize 4.1.3 but not in ArgoCD, even when I upgraded the Kustomize version inside the container via an init container shared emptyDir as seen in the patch I published above. I even exec'd into the argocd-server container and did a Kustomize version and it showed 4.1.3
thatsmydoing commentedon Jun 24, 2021
I think evaluation happens in argocd-repo-server instead. Which is probably still using the old kustomize.
HariSekhon commentedon Jun 24, 2021
Yeah I just woke up this morning and realized that! Will adjust the patch... I knew something was off.
Updated my patch and comment above with both solutions.
KnVerey commentedon Jul 12, 2022
/remove-lifecycle frozen
/close
Closing since this is about a very old version of Kustomize, and comments indicate it was caused by a dependency we no longer have.
k8s-ci-robot commentedon Jul 12, 2022
@KnVerey: Closing this issue.
Details
In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
ZiaUrRehman-GBI commentedon Dec 1, 2022
Kustomize version
{Version:kustomize/v4.5.7 GitCommit:56d82a8378dfc8dc3b3b1085e5a6e67b82966bd7 BuildDate:2022-08-02T16:28:01Z GoOs:darwin GoArch:arm64}
Error: accumulating resources: accumulation err='accumulating resources from 'github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2': evalsymlink failure on ...
no such file or directory': hit 27s timeout running '/usr/bin/git fetch --depth=1 origin v2.5.2'
Error: accumulating resources: accumulation err='accumulating resources from 'https://github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2': URL is a git repository': hit 27s timeout running '/usr/bin/git fetch --depth=1 origin v2.5.2'