Skip to content

Conversation

@JoshuaBehrens
Copy link
Contributor

@JoshuaBehrens JoshuaBehrens commented Apr 21, 2025

When working with third-party package repositories I learned to just execute a command like

$ composer config repositories.shopware-packages '{"type": "composer", "url": "https://packages.shopware.com"}'

and have access to new packages. I once was confused, that everyone is writing repositories as lists and after that one has a diff like that

 {
-    "repositories": [
+    "repositories": {
-        {
+        "shopware-packages": {
+            "type": "composer",
+            "url": "https://packages.shopware.com"
+        },
+        "0": {
             "type": "path",
             "url": "custom/static-plugins/*"
         }
-    ]
+    }
 }

Back then I was not really questioning it. Just different styles of writing it. Now as of today I am aware of more PHP internals like arrays have three indices. Therefore I better understand warnings like these in the docs

Using JSON object notation is also possible. However, JSON key/value pairs are to be considered unordered so consistent behaviour cannot be guaranteed.

Now coming back to this topic after some years I decided to tackle this and later noticed there is an open discussion in #9918 and the comments in https://stackoverflow.com/a/32814046 . I tried to follow up on the expected results of the discussions and have this pull request as a suggestion.

With this change you are now able to run

$ composer config repositories composer https://packages.shopware.com

and get this diff instead

 {
     "repositories": [
+        {
+            "type": "composer",
+            "url": "https://packages.shopware.com"
+        },
         {
             "type": "path",
             "url": "custom/static-plugins/*"
         }
     ]
 }

I do not see this a breaking change as the command line would not allow the command to work like this. Therefore this likely can just be a feature.


I hope this pull request meets requirements you expect for pull requests. I tried to update everything one might need:

  • documentation
  • tests
  • open issues
  • a reason why the change is needed
  • changelog entry
  • atomic commits

If you need something else just have it part of the review.


Sidenote: at first I tried to implement the logic for the list item to subnode but noticed, that there are so many more logic branches and changes to the regex that it became less easy to read. I noticed that the regex solution must have a reason to exist so I rather not have it a breaking change and started to introduce a new method. I did not try to make a solution that can convert list to assoc and assoc to list. At this point I just expected the fallback logic to make things work right.

Fixes #9918

Copy link
Member

@Seldaek Seldaek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, very solid PR with good test coverage and docs, thanks!

FYI the reason for the regex mess is that this way we can edit json files without force-reformatting them, which allows us to keep diffs to a minimum.. It's a bit of a "small nice idea grown into a monstrosity" but it works for 99% of things and the code is there now so might as well keep it..

@Seldaek Seldaek added this to the 2.9 milestone May 12, 2025
@Seldaek
Copy link
Member

Seldaek commented May 12, 2025

One more note, I think the main downside of this is that it doesn't let you unset them without knowing which index it is present at anymore.. But I guess if that is important to people they can still use named repos then.

@JoshuaBehrens
Copy link
Contributor Author

@Seldaek I guessed so, that this must be the reason for this to exist. At first I thought it has better control over "empty-object here but empty list over there" as this sometimes arise in composer files. Thank you for the explanation.

Good point regarding the removal. We could have a "host" / "url" option for removal but then the arguments start to become optional or similar shenanigans :D

I saw that the phpstan is pointing some things out. I will have a look at this later. Does it make sense to rebase to fix the CHANGELOG conflict or will you solve it eventually when it is merged as there will always be a conflict?

@Seldaek
Copy link
Member

Seldaek commented May 15, 2025

Feel free to rebase anyway to get the latest build fixes from main, BUT the changelog I usually do myself at release time so I'd say you can remove it completely that's fine by me.

@dzuelke
Copy link
Contributor

dzuelke commented Aug 8, 2025

@Seldaek I have been thinking about this whole repositories stuff again, and I am wondering if a cleaner solution overall would be to:

  1. add an id/name/nickname field to all repository types (the schema already allows extra fields, so it'd be fully backwards compatible) to refer to repository by name for programmatic manipulation;
  2. have a separate composer repositories command for manipulating repos, which
    • allows to have a good help screen without overloading the one for config;
    • could allow arguments for the various other fields of a repo (so no need to pass in JSON anymore);
    • could even interactively allow (re-)ordering;
    • could let you do things like composer repositories add --after otherrepoid …
    • gets rid of a bunch of special casing inside config handling logic;
  3. deprecate the object notation, because
    • it requires hashmap implementations with stable keys to handle composer.json in other languages;
    • the conversion to numeric keys is confusing;
    • no sorting hapens even though numeric keys might make you think that is the case;
    • two ways of skinning the same animal;
  4. maybe even auto-convert any existing object notations to array notations when re-serializing composer.json (still readable by older versions of Composer, as noted earlier)

@Seldaek
Copy link
Member

Seldaek commented Aug 19, 2025

@dzuelke yeah I think that all sounds great, it would mirror a bit the git remote command I guess (+ ordering..). Definitely would be more sensible than adding more hacks on top of the existing. But it also sounds like quite a bit of work..

@JoshuaBehrens
Copy link
Contributor Author

I admit that I don't really like how git remote behaves in some cases but I see the gains of the feedback. Let me cook on that :)

@JoshuaBehrens
Copy link
Contributor Author

@dzuelke @Seldaek I rewrote it to integrate a new command. I hopefully added enough test coverage (which is right now mostly happy-path). I also had to adjust the json schema and force "old" structures to the new one. I tried to only make it happen, when adding new repositories so it does not annoy too much. I made sure the name attribute for repositories is now always on top. This should make sure, they are sorted correctly so @localheinz does not have to sort it afterwards. I am very glad, that there already was a list-item-approach to disable packagist. Docs where AI powered, so if you want it to have more detailed or rewritten, I am happy to do that.

I am not sure about the php8.5 test. I do not see it directly why my changes would've bricked it. The pull request is now ready to review.

@Seldaek
Copy link
Member

Seldaek commented Oct 30, 2025

Thanks @JoshuaBehrens again, this is great work. I did tweak some things in the few commits above but I think now we're in a pretty good spot with this!

Only two areas of improvement left that I see:

  • I'd like to tweak the formatting from:
    "repositories": [{
        "name": "foo",
        "type": "path",
        "url": "./"
    },{
        "packagist.org": false
    }]

to:

    "repositories": [
        {
            "name": "foo",
            "type": "path",
            "url": "./"
        }, {
            "packagist.org": false
        }
    ]

I think it would be more readable and more obvious that there is an array of objects?

@JoshuaBehrens
Copy link
Contributor Author

@Seldaek Thank you for the guidance and review. Happy to contribute :)

I can add canonical, only, exclude as options. I mean it is not that much different to the URL.

I understand and also like the syntax with the }, { but I doubt, that the PHP json_encode will print out anything like this. So I can have a look whether I can get RegEx to work with that, but eventually every fallback json_encode will break it. Doesn't it? Or shall I just preg_replace('/},\\s*{/g', '}, {', $content)-ish for everything?

@Seldaek
Copy link
Member

Seldaek commented Oct 30, 2025

@JoshuaBehrens if you want to do canonical/only/exclude feel free put please work on a new PR based on this one, as I'd rather get this one merged already. The newline stuff I am trying to look at it for a bit and if that doesn't work I'll skip, it's not that important.

Edit: OK not worth the hassle I think :D

@Seldaek Seldaek merged commit f414237 into composer:main Oct 30, 2025
21 checks passed
@JoshuaBehrens JoshuaBehrens deleted the feature/9918 branch October 30, 2025 12:49
@JoshuaBehrens
Copy link
Contributor Author

Thank you for merging :)

@dzuelke
Copy link
Contributor

dzuelke commented Oct 30, 2025

Very nice work @JoshuaBehrens and @Seldaek!

Should the tests for the old object key syntax not maybe remain? They all got changed to the new name key variant, but that means there is now no guard against regressions in the handling of that.

Might also be useful to have tests for the "evil" case of old object key syntax but name in the value objects, maybe even with mismatching names, like so:

{
  "repositories": {
    "foo": {
      "name": "bar",
      "type": "composer",
      "url": "https://packagist.org/"
    }
  }
}

@Seldaek
Copy link
Member

Seldaek commented Oct 31, 2025

That sounds sensible.. @JoshuaBehrens if you have time for this be my guest 😇

vinkla added a commit to vinkla/wordplate that referenced this pull request Nov 10, 2025
dzuelke added a commit to heroku/buildpacks-php that referenced this pull request Nov 11, 2025
Based on [my suggestions and concept for repository-related configuration updates](composer/composer#12388 (comment)), Composer 2.9 will have a [new command to modify repositories](composer/composer@f414237) in `composer.json`, and to make that possible, `repositories` entries can now have a `name` field.

The struct and serde definitions already permit this, but we are adding explicit tests for it here, as well as checks to reject invalid combinations (object notation together with names, as also disallowed by Composer's schema for `composer.json`) during deserialization.

GUS-W-20086506
@Kocal
Copy link

Kocal commented Nov 13, 2025

Thank you for working on this! 🙏🏻

@JoshuaBehrens
Copy link
Contributor Author

@Kocal you are welcome :)

@drzraf
Copy link

drzraf commented Nov 28, 2025

After this version reach my CI I start getting tons of warning output from composer 2.9:

[...]
 - repositories.gravityforms/gravityforms : Matched a schema which it should not
 - repositories.gravityforms/gravityforms : Failed to match all schemas
 - repositories.gravityforms/gravityforms : Object value found, but a boolean is required
 - repositories.gravityforms/gravityforms : Does not have a value in the enumeration [false]
 - repositories.gravityforms/gravityforms : Failed to match at least one schema
 - repositories.dearflip/dearflip : Matched a schema which it should not
 - repositories.dearflip/dearflip : Failed to match all schemas
 - repositories.dearflip/dearflip : Object value found, but a boolean is required
 - repositories.dearflip/dearflip : Does not have a value in the enumeration [false]
 - repositories.dearflip/dearflip : Failed to match at least one schema
 [...]

I believe the below config is not that uncommon and the json error message are not as clear as they should (if you don't already know what & where to look for).

$ jq '{"repositories":.repositories}' .config/composer/config.json

{
  "repositories": {
    "wpengine/advanced-custom-fields-pro": {
      "type": "composer",
      "url": "https://connect.advancedcustomfields.com"
    },
    "freemius": {
      "type": "composer",
      "url": "https://composer.freemius.com/packages.json?authorization=Basic+...."
    },
    "dearflip/dearflip": {
      "name": "dearflip/dearflip",
      "type": "package",
      "package": {
        "name": "dearflip/dearflip",
        "version": "2.3.30",
        "type": "wordpress-plugin",
        "dist": {
          "url": "https://foo.gitlab.io/paid-plugins/tarballs/dflip-2.3.30.zip",
          "type": "zip"
        }
      }
    },
    "gravityforms/gravityforms": {
      "name": "gravityforms/gravityforms",
      "type": "package",
      "package": {
        "name": "gravityforms/gravityforms",
        "version": "2.8.16",
        "type": "wordpress-plugin",
        "dist": {
          "type": "zip",
          "url": "https://www.gravityhelp.com/wp-content/plugins/gravitymanager/api.php?op=get_plugin&slug=gravityforms&key=...."
        },
        "require": {
          "composer/installers": "^1.8|^2.1",
          "gotoandplay/gravityforms-composer-installer": "^2.3"
        }
      }
    },
    "wpackagist": {
      "type": "composer",
      "url": "https://wpackagist.org"
    }
  }
}

@dzuelke
Copy link
Contributor

dzuelke commented Nov 28, 2025

@drzraf the issue there is the name attribute inside the repository definition. It's either that or the object syntax.

@Seldaek
Copy link
Member

Seldaek commented Nov 29, 2025

Yeah you ideall convert repos to an array, remove the keys and move them to name properties then you can use the new repo command with it all

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adding repositories without name using config command

5 participants