Skip to content

Config server yml or properties endpoints can produce unexpected results when merging collections and maps #385

@dmfrey

Description

@dmfrey

Please review the attached YAML file for sample properties. Note the default profile has an array of PromoCode objects defined. When a SpringBoot application is configured to talk to the Config Server with the appropriate profile, you get the correct response. However, when you request the properties or yaml view, you get a mixed array of the 2 profiles.

http://localhost:8888/promoCodes/default - produces correct json
http://localhost:8888/promoCodes/dev - produces correct json

Alternate views produce correct results for default profile

http://localhost:8888/promoCodes-default.properties

promos.promoCards[0].client: Google Nexus 6P     <----
promos.promoCards[0].description: Free stuff for Google users     <----
promos.promoCards[0].endDate: 2016-04-26T20:01:30.781Z     <----
promos.promoCards[0].image: http://google.com/images/coolpic.jpg     <----
promos.promoCards[0].name: P1     <----
promos.promoCards[0].platform: Android     <----
promos.promoCards[0].startDate: 2016-04-26T20:01:30.781Z     <----
promos.promoCards[0].thumbnail: http://google.com/images/coolpic.jpg     <----
promos.promoCards[1].client: Google Nexus 5X
promos.promoCards[1].description: Free stuff for Google users
promos.promoCards[1].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[1].image: http://google.com/images/coolpic.jpg
promos.promoCards[1].name: P2
promos.promoCards[1].platform: Android
promos.promoCards[1].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[1].thumbnail: http://google.com/images/coolpic.jpg
promos.promoCards[2].client: iPhone 6S
promos.promoCards[2].description: Expensive stuff from Steve
promos.promoCards[2].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[2].image: http://apple.com/images/coolpic.jpg
promos.promoCards[2].name: P3
promos.promoCards[2].platform: iOS 9
promos.promoCards[2].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[2].thumbnail: http://apple.com/images/coolpic.jpg
promos.promoCards[3].client: Nokia Lumia 8610
promos.promoCards[3].description: Bad stuff for windoze users
promos.promoCards[3].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[3].image: http://microsoft.com.com/images/coolpic.jpg
promos.promoCards[3].name: P4
promos.promoCards[3].platform: Windoze
promos.promoCards[3].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[3].thumbnail: http://microsoft.com.com/images/coolpic.jpg

Alternate views produce incorrect results for dev profile

http://localhost:8888/promoCodes-dev.properties

promos.promoCards[0].client: Nokia Lumia 8888     <----
promos.promoCards[0].description: Bad stuff for windoze users     <----
promos.promoCards[0].endDate: 2016-04-26T20:01:30.781Z     <----
promos.promoCards[0].image: http://microsoft.com.com/images/coolpic.jpg     <----
promos.promoCards[0].name: P4     <----
promos.promoCards[0].platform: Windoze     <----
promos.promoCards[0].startDate: 2016-04-26T20:01:30.781Z     <----
promos.promoCards[0].thumbnail: http://microsoft.com.com/images/coolpic.jpg     <----
promos.promoCards[1].client: Google Nexus 5X
promos.promoCards[1].description: Free stuff for Google users
promos.promoCards[1].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[1].image: http://google.com/images/coolpic.jpg
promos.promoCards[1].name: P2
promos.promoCards[1].platform: Android
promos.promoCards[1].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[1].thumbnail: http://google.com/images/coolpic.jpg
promos.promoCards[2].client: iPhone 6S
promos.promoCards[2].description: Expensive stuff from Steve
promos.promoCards[2].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[2].image: http://apple.com/images/coolpic.jpg
promos.promoCards[2].name: P3
promos.promoCards[2].platform: iOS 9
promos.promoCards[2].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[2].thumbnail: http://apple.com/images/coolpic.jpg
promos.promoCards[3].client: Nokia Lumia 8610
promos.promoCards[3].description: Bad stuff for windoze users
promos.promoCards[3].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[3].image: http://microsoft.com.com/images/coolpic.jpg
promos.promoCards[3].name: P4
promos.promoCards[3].platform: Windoze
promos.promoCards[3].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[3].thumbnail: http://microsoft.com.com/images/coolpic.jpg

Note the difference in promos.promoCards[0] between the 2 results indicated by <----.

The desired output should be for the dev profile:

http://localhost:8888/promoCodes-dev.properties

promos.promoCards[0].client: Nokia Lumia 8888
promos.promoCards[0].description: Bad stuff for windoze users
promos.promoCards[0].endDate: 2016-04-26T20:01:30.781Z
promos.promoCards[0].image: http://microsoft.com.com/images/coolpic.jpg
promos.promoCards[0].name: P4
promos.promoCards[0].platform: Windoze
promos.promoCards[0].startDate: 2016-04-26T20:01:30.781Z
promos.promoCards[0].thumbnail: http://microsoft.com.com/images/coolpic.jpg

The issue occurs in org.springframework.cloud.config.server.environment.EnvironmentController, specifically convertToProperties(Environment environment):

private Map<String, Object> convertToProperties(Environment profiles) {
    Map<String, Object> map = new TreeMap<>();
    List<PropertySource> sources = new ArrayList<>(
            profiles.getPropertySources());
    Collections.reverse(sources);
    for (PropertySource source : sources) {
        @SuppressWarnings("unchecked")
        Map<String, String> value = (Map<String, String>) source.getSource();
        map.putAll(value);                               <-- (No checking for existing array from earlier profile with same name)
    }
    postProcessProperties(map);
    return map;
}

The issue is the for loop over the PropertySources. The Map<String, String> value has a key that contains all the individual promos.promoCards[X] entries individually and puts them all on each iteration into a final map, replacing any duplicates. However, in the case of a YAML list, it should also be checking if any other 'promos.promoCards[' exists as keys and remove them before adding all of the current iteration to the resulting map.

promoCodes.yml.txt

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions