Skip to content

kustomize 3.5.5 breaks installing ArgoCD from URL #2538

Closed

Description

@chancez

With kustomize 3.5.4, the following command succeeds and produces the YAML as expected:

kustomize build 'github.com/argoproj/argo-cd//manifests/ha/cluster-install?ref=v1.5.2

With kustomize 3.5.5 I now get the following error when running the command:

$ kustomize build 'github.com/argoproj/argo-cd//manifests/ha/cluster-install?ref=v1.5.2'
Error: accumulating resources: accumulateFile "accumulating resources from '../namespace-install': evalsymlink failure on '/private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/namespace-install' : lstat /private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/namespace-install: no such file or directory", loader.New "Error loading ../namespace-install with git: url lacks host: ../namespace-install, dir: evalsymlink failure on '/private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/namespace-install' : lstat /private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/namespace-install: no such file or directory, get: invalid source string: ../namespace-install"

Activity

  1. changed the title [-]Kustomize 3.5.5 breaks installing ArgoCD from URL[/-] [+]kustomize 3.5.5 breaks installing ArgoCD from URL[/+] on May 26, 2020
  2. chancez commented on May 26, 2020

    @chancez
    ContributorAuthor

    May be related to #2494

  3. monopole commented on May 27, 2020

    @monopole
    Contributor

    likely a dupe of #2444. Fix is in #2537, which is in v3.6.1
    please give it a try. reopen if it doesn't work.

    @Shell32-Natsu FYI

  4. chancez commented on May 27, 2020

    @chancez
    ContributorAuthor

    This doesn't seem resolved:

    ~/p/kubernetes-gitops(⎈ |dev-2447-us-west-2:jupyterhub) czibolski ❯❯❯ ~/Downloads/kustomize version                                                                          ⏎staging_prod_shared_logging_overlay ✭
    {Version:kustomize/v3.6.1 GitCommit:c97fa946d576eb6ed559f17f2ac43b3b5a8d5dbd BuildDate:2020-05-27T20:47:35Z GoOs:darwin GoArch:amd64}
    ~/p/kubernetes-gitops(⎈ |dev-2447-us-west-2:jupyterhub) czibolski ❯❯❯ ~/Downloads/kustomize build 'github.com/argoproj/argo-cd//manifests/ha/cluster-install?ref=v1.5.2'      staging_prod_shared_logging_overlay ✭
    Error: accumulating resources: accumulateFile "accumulating resources from '../namespace-install': evalsymlink failure on '/private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/kustomize-731707729/namespace-install' : lstat /private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/kustomize-731707729/namespace-install: no such file or directory", loader.New "Error loading ../namespace-install with git: url lacks host: ../namespace-install, dir: evalsymlink failure on '/private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/kustomize-731707729/namespace-install' : lstat /private/var/folders/0_/gjp5mdvn64lgly0qsrlzq1vc0000gp/T/kustomize-731707729/namespace-install: no such file or directory, get: invalid source string: ../namespace-install"
    
  5. chancez commented on May 27, 2020

    @chancez
    ContributorAuthor

    And I cannot seem to re-open this issue either.

  6. Shell32-Natsu commented on May 27, 2020

    @Shell32-Natsu
    Contributor

    I think the issue is different from #2444. Kustomize has changed to use go-getter by default. From the documentation of go-getter:

    If you want to download only a specific subdirectory from a downloaded directory, you can specify a subdirectory after a double-slash //. go-getter will first download the URL specified before the double-slash (as if you didn't specify a double-slash), but will then copy the path after the double slash into the target directory.

    So only manifests/ha/cluster-install will be copied and ../namespace-install is not there. You can try to replace // with /.

  7. seans3 commented on May 27, 2020

    @seans3
    Contributor

    Re-opening so that we can triage it.

    /reopen

  8. k8s-ci-robot commented on May 27, 2020

    @k8s-ci-robot
    Contributor

    @seans3: Reopened this issue.

    Details

    In response to this:

    Re-opening so that we can triage it.

    /reopen

    Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

  9. chancez commented on May 27, 2020

    @chancez
    ContributorAuthor

    @Shell32-Natsu That does seem to make a difference. I'll try to make this change to upstream argo-cd install docs so that it's hopefully avoided for others. Though I imagine this might be an issue for other remote bases/overlays that people could be using.

  10. monopole commented on May 28, 2020

    @monopole
    Contributor

    Thanks @chancez and @seans3, and thanks @Shell32-Natsu for taking this on.

  11. jameshochadel commented on Jun 1, 2020

    @jameshochadel

    I am encountering this issue on 3.6.1, but I am not using // anywhere in my URLs. However, the issue only occurs when the protocol is not specified.

    E.g., where my-repo/dev/kustomization.yaml references a resource ../base, this fails:

    kustomize build my-github-org/my-repo/dev

    but this succeeds:

    kustomize build https://my-github-org/my-repo/dev

    This was not previously (before 3.6.1) the case.

  12. Shell32-Natsu commented on Jun 1, 2020

    @Shell32-Natsu
    Contributor

    @jameshochadel Hi James, thanks for the feedback. It looks like this is a go-getter issue. I have created issue here.

    For your case, the problem is go-getter has bug with sub directory. Adding https works since go-getter fails with https scheme as well (because it's ambiguous I guess) and a normal git clone happens.

    And the reason for why @chancez can fix the issue by replace // with / is go-getter doesn't work when you are using / and ref at the same time. So actually there is a normal git clone as well.

    I am surprised that go-getter has these problems. I think we may need to clearly document the url pattern if we want to continue using go-getter.

  13. 19 remaining items

  14. yujunz commented on Nov 18, 2020

    @yujunz
    Member

    I see that @yujunz added go-getter into kustomize, maybe they could shed more light on how they use this?

    It was tested against the examples in https://github.com/kubernetes-sigs/kustomize/blob/master/examples/remoteBuild.md#url-format . But I roughly remembered the example tests were once excluded from CI. Not sure if it is covered now or not.

    See also some history conversion in the two PRs for the problems go-getter tries to resolve and the tradeoffs

  15. thatsmydoing commented on Nov 19, 2020

    @thatsmydoing
    Contributor

    Dropping go-getter will probably break the backward compatibility to current version.

    Aside from archive downloads and urls that use the depth query parameter, I don't think any url will be broken if go-getter is removed.

    Either way though, I do think kustomize needs to be clear what the supported urls are. If the official position is that only go-getter urls are supported, then we would have just dropped the old cloner entirely but that didn't pan out. Some people have suggested removing go-getter but even then, there should be a spec as to what urls are supported since it accepts way too many variations and any kind of refactoring would be difficult without breaking someone's workflow. The repospec code is very convoluted and it's difficult to tell what kinds of urls it accepts and how it transforms them.

    As a user, I just want to know what url formats are supported and that I can reasonably expect to never break and that kustomize will immediately fix if broken.

  16. yujunz commented on Nov 19, 2020

    @yujunz
    Member
  17. yujunz commented on Nov 20, 2020

    @yujunz
    Member

    I just realized the tests in remote build were skipped because of an extra blank line...

    Proposed a fix in #3253

  18. HariSekhon commented on Jun 23, 2021

    @HariSekhon

    I had this issue on ArgoCD 2.0.3 with Kustomize 3.9.4

    I solved it by just replacing the base url double slashes // from the docs with a single slash / in my kustomization.yaml- I tried this with and without https:// protocol prefix and both worked.

    kustomization.yaml:

    bases:
    #- github.com/argoproj/argo-cd//manifests/cluster-install?ref=v2.0.3
    - github.com/argoproj/argo-cd/manifests/cluster-install?ref=v2.0.3
    

    EDIT: upgrading Kustomize to 4 in argocd-repo-server also solves this problem - I include this patch in my kustomizaton.yaml to get a specific version of Kustomize now.

  19. thatsmydoing commented on Jun 23, 2021

    @thatsmydoing
    Contributor

    Are you sure it's not working for kustomize 4.1.3? I tried it locally and both urls work. Pre version 4, you might be hitting the go-getter path which is known to be broken for certain urls.

  20. HariSekhon commented on Jun 23, 2021

    @HariSekhon

    It worked locally in Kustomize 4.1.3 but not in ArgoCD, even when I upgraded the Kustomize version inside the container via an init container shared emptyDir as seen in the patch I published above. I even exec'd into the argocd-server container and did a Kustomize version and it showed 4.1.3

  21. thatsmydoing commented on Jun 24, 2021

    @thatsmydoing
    Contributor

    I think evaluation happens in argocd-repo-server instead. Which is probably still using the old kustomize.

  22. HariSekhon commented on Jun 24, 2021

    @HariSekhon

    Yeah I just woke up this morning and realized that! Will adjust the patch... I knew something was off.

    Updated my patch and comment above with both solutions.

  23. KnVerey commented on Jul 12, 2022

    @KnVerey
    Contributor

    /remove-lifecycle frozen
    /close

    Closing since this is about a very old version of Kustomize, and comments indicate it was caused by a dependency we no longer have.

  24. k8s-ci-robot commented on Jul 12, 2022

    @k8s-ci-robot
    Contributor

    @KnVerey: Closing this issue.

    Details

    In response to this:

    /remove-lifecycle frozen
    /close

    Closing since this is about a very old version of Kustomize, and comments indicate it was caused by a dependency we no longer have.

    Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

  25. removed
    lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.
    on Jul 12, 2022
  26. ZiaUrRehman-GBI commented on Dec 1, 2022

    @ZiaUrRehman-GBI

    Kustomize version
    {Version:kustomize/v4.5.7 GitCommit:56d82a8378dfc8dc3b3b1085e5a6e67b82966bd7 BuildDate:2022-08-02T16:28:01Z GoOs:darwin GoArch:arm64}

     kustomize build .      
    github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2
    

    Error: accumulating resources: accumulation err='accumulating resources from 'github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2': evalsymlink failure on ...
    no such file or directory': hit 27s timeout running '/usr/bin/git fetch --depth=1 origin v2.5.2'

    kustomize build .  
    https://github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2
    

    Error: accumulating resources: accumulation err='accumulating resources from 'https://github.com/argoproj/argo-cd/manifests/ha/cluster-install?ref=v2.5.2': URL is a git repository': hit 27s timeout running '/usr/bin/git fetch --depth=1 origin v2.5.2'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/dependencyIssues or PRs related to dependency changeskind/bugCategorizes issue or PR as related to a bug.triage/needs-informationIndicates an issue needs more information in order to work on it.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions