Repository navigation
UnknownComponentDependencyException with SBOM from cyclonedx-javascript-library #746
Description
Hello,
I try to enhance a given SBOM, which is generated with
https://github.com/CycloneDX/cyclonedx-javascript-library
While writing the sbom with output_as_string I get the exception:
UnknownComponentDependencyException
cyclonedx.exception.model.UnknownComponentDependencyException: One or more Components have Dependency references to Components/Services that are not known in this BOM. They are: {<BomRef '@mui/styled-engine@npm:5.16.6 [3135b]' id=126477793907040>, <BomRef '@mui/material@npm:5.14.0 [cd63f]' id=126477793557152>, <BomRef 'react-redux@npm:8.1.1 [cd63f]' id=126477790002304>, <BomRef '@mui/system@npm:5.16.7 [c1a0d]' id=126477793910160>}
For example the component:
@mui/styled-engine@npm:5.16.6 [3135b]
is used in:
components: [
{
"type": "library",
"name": "styled-engine",
"group": "@mui",
"version": "5.16.6",
"bom-ref": "@mui/styled-engine@npm:5.16.6 [3135b]",
"author": "MUI Team",
"description": "styled() API wrapper package for emotion.",
"licenses": [
{
"license": {
"id": "MIT"
}
}
],
"purl": "pkg:npm/%40mui/[email protected]?vcs_url=git%2Bhttps%3A//github.com/mui/material-ui.git#packages/mui-styled-engine",
"externalReferences": [
{
"url": "https://github.com/mui/material-ui/issues",
"type": "issue-tracker",
"comment": "as detected from PackageJson property \"bugs.url\""
},
{
"url": "git+https://github.com/mui/material-ui.git#packages/mui-styled-engine",
"type": "vcs",
"comment": "as detected from PackageJson property \"repository.url\" and \"repository.directory\""
},
{
"url": "https://mui.com/system/styled/",
"type": "website",
"comment": "as detected from PackageJson property \"homepage\""
}
]
}
and
dependencies: [
{
"ref": "@mui/styled-engine@npm:5.16.6 [3135b]",
"dependsOn": [
"@babel/runtime@npm:7.26.0",
"@emotion/cache@npm:11.13.5",
"csstype@npm:3.1.3",
"prop-types@npm:15.8.1",
"react@npm:18.2.0"
]
},
{
"ref": "@mui/system@npm:5.16.7 [260c3]",
"dependsOn": [
"@babel/runtime@npm:7.26.0",
"@mui/private-theming@npm:5.16.6 [3135b]",
"@mui/styled-engine@npm:5.16.6 [3135b]",
"@mui/types@npm:7.2.19 [260c3]",
"@mui/utils@npm:5.16.6 [260c3]",
"clsx@npm:2.1.1",
"csstype@npm:3.1.3",
"prop-types@npm:15.8.1",
"react@npm:18.2.0"
]
},
I dont know why the [3135b] is added to this component. It looks like an hash or something.
Could it be that the space between @mui/system@npm:5.16.7 and [260c3] causes the problems?
I am using cyclonedx-python-lib Version: 8.5.0
Any ideas or suggestions would help.
Activity
jkowalleck commentedon Dec 12, 2024
Thanks for the report, @finger42 .
Could you provide a reproducible setup?
Maybe create a repository that hosts the relevant SBOM(-Part), a requirements.txt declaring the needed dependencies, and a python script with the code you tried to run?
finger42 commentedon Dec 13, 2024
Hello,
Thx for the hint do to my homework ;-)
I think the problem is that there are two components, which are exactly the same ( only the bom-ref differs )
If I modify one property in one of the comonents. e.g. version to something other, the problem is gone.
Now my question: Is this a valid SBOM ?
If not, then the generator of the original SBOM , or another script, needs to strip the duplicated components ?
Beside that, the error msg:
could be improved to point out that there are duplicated components in the dependency tree ?
jkowalleck commentedon Dec 13, 2024
yepp, exactly that should be the reason.
The python library does some deduplication, since it uses
set()for collections.And the component's hash function does not take a
bom-refinto account, since there is novalue to it - a bom-ref is a mere anchor, not a qualitative property.jkowalleck commentedon Dec 13, 2024
It should be schema-valid, yes.
finger42 commentedon Dec 13, 2024
So I need to deduplicate the SBOM generated with the cyclonedx-javascript-library.
I will check that and report again.
Thx for the help so far.
jkowalleck commentedon Dec 14, 2024
@finger42 , may I ask for some background?
Are you using one of the many open-source SBOM generators running on nodej? Which one?
Or are you writing your own tool? Can I take a look at it?
Thank you in advance.
(I am the author and maintainer of that javascript library you were using. It would help to understand your downstream use cases. The usual thing with open source is: you never hear about success stories, you only see complaints and issue reports when something does not work as expected. Learning about downstream use cases helps to understand what is good and what needs to be improved.)
finger42 commentedon Dec 15, 2024
@jkowalleck
I try to build SBOMs with different tools. Some of these tools are using javascirpt for webapps, some of them the nuget lib, others the python lib. Or even other SBOM generators ( yocto. buildroot, ... ) I currently dont know, but they exist ;-)
All in common is the final goal to import SBOMs in self hosted https://dependencytrack.org/ to monitor the components for different firmware versions (also for old devices).
Most of the Sec. Issue matching is done with CPEs ( NIST )
Some of other databases are using PURL
I will ask my webdevs why the double components are inside the given SBOM.
I think that the CRA is the reason why this topic ( SBOM ) is getting more attention.
And yes, I understand that it is hard to get feedback from closed source to open source projects.
I am really happy with the features of the cyclonedx libs in general and all the work the contributes and maintainers do.
Thank you !
jkowalleck commentedon Dec 16, 2024
a workaround: use properties to make each component unique.
finger42 commentedon Dec 17, 2024
@jkowalleck I need to wait till next year for my team an test the properties as a workaround.
Maybe we could find out the root cause for the double entries.
jkowalleck commentedon Dec 17, 2024
a thing could be the nature of not-deduplicated node modules - read https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/docs/component_deduplication.md
and see examples here:
https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/demo/*/example-results/bare/bom.*.jsonversushttps://github.com/CycloneDX/cyclonedx-node-npm/blob/main/demo/*/example-results/flat/bom.*.jsonfinger42 commentedon Jan 17, 2025
Some background info:
SBOM generation:
We use "@cyclonedx/yarn-plugin-cyclonedx": "^1.0.1" (GitHub - CycloneDX/cyclonedx-node-yarn: Create CycloneDX Software Bill of Materials (SBOM) from Node.js Yarn projects.) to generate the SBOM.
It is installed as devDependency and currently run with this command:
Results and Issues:
Some packages like @mui/styled-engine are listed more than once (as mentioned above):
This seems strange because our yarn.lock lists it only once:
but the results of running “yarn why @mui/styled-engine” correspond with the SBOM:
jkowalleck commentedon Jan 17, 2025
Ah, the usual non-deduplicated node modules, that is not an issue. That should be no issue.
Anyway, I see the root cause to be, as you suggested,
that the components are equal(
bom-refis not part of equality right now) and duplicates are therefore dropped.A short term solution would be to add a unique property to the components generated by the yarn plugin, but for this I would need a reproducible testbed(similar to the ones in https://github.com/CycloneDX/cyclonedx-node-yarn/tree/main/tests/_data/testbeds)
I guess there is only one long-term solution to it:
bom-refsMUST become of equality-checksI will work on a fix - a breaking change that should be released with the next major version, soon.
jkowalleck commentedon Jan 17, 2025
this bug caused #753
finger42 commentedon Feb 4, 2025
Hello,
I tested locally the branch:
https://github.com/CycloneDX/cyclonedx-python-lib/tree/feat/BomRef-affects-comparissons-equality
and it works really nice.
Thx for your help @jkowalleck and also thx @wkoot !
You helped us a lot to monitor big nested dep. trees with https://dependencytrack.org/
Can you tell me when the version 9.x will be released ?
jkowalleck commentedon Feb 4, 2025
the next release will take a while, as it will be composed of several interlinked features.
you can speed this up by contributing to them.
you may comment on #754 which part you will take over, and pullrequest the needed changes to the target branch
feat/BomRef-affects-comparissons-equality.but before all of this, i would really love to get #755 done. this one will make #754 easier to review.
Here goes the same: If you want to speed things up, please drop a note which part you want to work on, and just pullrequest that.
for all of this it is the same: pullrequest early (draft/wip) to indicate your intention & dont try to be perfect. "it works" is enough, we can improve over time.
jkowalleck commentedon Feb 13, 2025
A preview of the fix/feature is available via https://github.com/CycloneDX/cyclonedx-python-lib/releases/tag/v9.0.1-rc.1
finger42 commentedon Feb 17, 2025
Thank you.
I have tested the branch and it works nicely !