Skip to content

Merge does not deduplicate components with differing bom-ref #399

Open

Description

@wkoot

Given the following two test files, their merged output does not deduplicate components as mentioned in CycloneDX/cyclonedx-python-lib#540 (comment)

File test/in1.json:

{
  "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "metadata": {"component": {"bom-ref": "test11", "type": "container", "name": "alpine"}},
  "components": [{"bom-ref": "test12", "type": "operating-system", "name": "alpine"}],
  "dependencies": [{"ref": "test11", "dependsOn": ["test12"]}]
}

and test/in2.json:

{
  "$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "metadata": {"component": {"bom-ref": "test21", "type": "container", "name": "alpine"}},
  "components": [{"bom-ref": "test22", "type": "operating-system", "name": "alpine"}],
  "dependencies": [{"ref": "test21", "dependsOn": ["test22"]}]
}

The following merge command:

docker run --rm -v $(pwd)/test:/test cyclonedx/cyclonedx-cli:0.26.0 merge --input-files /test/in1.json /test/in2.json --input-format json --output-format json --output-file /test/out.json --name test

Produces the following output file test/out.json, which contains duplicate components:

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.5",
  "serialNumber": "urn:uuid:66fa5692-2e9d-45c5-830a-ec8ccaf7dcc9",
  "version": 1,
  "metadata": {
    "component": {
      "type": "application",
      "name": "test"
    }
  },
  "components": [
    {
      "type": "operating-system",
      "bom-ref": "test12",
      "name": "alpine"
    },
    {
      "type": "container",
      "bom-ref": "test11",
      "name": "alpine"
    },
    {
      "type": "operating-system",
      "bom-ref": "test22",
      "name": "alpine"
    },
    {
      "type": "container",
      "bom-ref": "test21",
      "name": "alpine"
    }
  ],
  "dependencies": [
    {
      "ref": "test11",
      "dependsOn": [
        "test12"
      ]
    },
    {
      "ref": "test21",
      "dependsOn": [
        "test22"
      ]
    }
  ]
}

Activity

  1. wkoot commented on Oct 23, 2024

    @wkoot
    Author

    Since the file validation changes introduced to Dependency-Track in version 4.11, files produced by cyclonedx-cli merge in this manner also produce errors in DT:

    ERROR [BomUploadProcessingTask] Error while processing bom
    java.lang.IllegalStateException: Duplicate key Identity[group=aquasecurity, name=trivy:FilePath, value=home/frontend/node_modules/body-parser/node_modules/debug/package.json] (attempted merging values ComponentProperty{id=97037, component=pkg:npm/[email protected], groupName=aquasecurity, propertyName=trivy:FilePath, propertyValue=home/frontend/node_modules/body-parser/node_modules/debug/package.json, propertyType=STRING, uuid=c7528c5d-c315-4aa8-b259-6010af83c96c} and ComponentProperty{id=101526, component=pkg:npm/[email protected], groupName=aquasecurity, propertyName=trivy:FilePath, propertyValue=home/frontend/node_modules/body-parser/node_modules/debug/package.json, propertyType=STRING, uuid=0a33962a-62c7-4952-a108-f77edfc143d9})
    	at java.base/java.util.stream.Collectors.duplicateKeyException(Unknown Source)
    	at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Unknown Source)
    	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(Unknown Source)
    	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
    [...]
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions