-
Notifications
You must be signed in to change notification settings - Fork 191
Fork aliasing #1620
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fork aliasing #1620
Conversation
4a57dc1 to
7d02588
Compare
docs/references/files/fossa-deps.md
Outdated
|
|
||
| **Version Matching rules:** | ||
| - If `fork` version is specified, only that exact version will be translated | ||
| - If `fork` version is not specified, any version will match |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this mean that every vulnerability for any version of the upstream package gets reported?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It means that we will treat it as a fork alias for any version. Let me re-write that to be more clear.
src/App/Fossa/Analyze/ForkAlias.hs
Outdated
| collectForkAliasLabels = Map.fromListWith (++) . mapMaybe forkAliasToLabel | ||
| where | ||
| forkAliasToLabel :: ForkAlias -> Maybe (Text, [ProvidedPackageLabel]) | ||
| forkAliasToLabel ForkAlias{..} = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nit/Optional] I've been moving away from record wild-cards to using explicit names or RecordDotSyntax. Personally, I prefer the explicitness of both over having to figure out which bindings come from the match. LSP should offer a quick action to add in the items you actually used.
RecordDotSyntax allows a syntax like in Rust: forkAlias.forkAliasBase.
docs/references/files/fossa-deps.md
Outdated
| - `fork`: The fork dependency entry that should be aliased to the base dependency. (Required) | ||
| - `type`: Type of the fork dependency. (Required) | ||
| - `name`: Name of the fork dependency. (Required) | ||
| - `version`: Version of the fork dependency. (Optional) | ||
| - `base`: The base/original dependency entry that your fork should be aliased to. (Required) | ||
| - `type`: Type of the base dependency. (Required) | ||
| - `name`: Name of the base dependency. (Required) | ||
| - `version`: Version of the base dependency. (Optional) | ||
| - `labels`: An optional list of labels to be added to the fork alias. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these keys meant to match the yaml, or be a list of definitions?
This isn't blocking, but when I tested against prod these looked like normal deps. If you haven't already done it, I think we should consider adding a default fork-alias label to every instance of a fork-alias dependency. Without that, I think we or the customer would have to correlate deps between the UI and fossa-deps.yml in order to get the full picture that these deps aren't "normal".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are normal deps, and Core has no idea that they are fork-aliases. I like the suggestion, but it would add a significant amount of work to do it, so I'm going to leave it as is for now.
I'll add a note that it will add the label to all versions of the dependency in the project, and that we suggest using the project or revision scope so that it doesn't add the label across the org.
docs/references/files/fossa-deps.md
Outdated
| - `fork`: The fork dependency entry that should be aliased to the base dependency. (Required) | ||
| - `type`: Type of the fork dependency. (Required) | ||
| - `name`: Name of the fork dependency. (Required) | ||
| - `version`: Version of the fork dependency. (Optional) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For a "fork" dependency that just means this is the version we report - right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the version is there, then we will only translate that version. If it's not there, we will translate any version of the package. I've added a note to see the "version matching rules" below for more details
| spec = do | ||
| describe "mkForkAliasMap" $ do | ||
| it "should create a map keyed by project locator" $ do | ||
| let fork = ForkAliasEntry CargoType "my-serde" (Just "1.0.0") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nit] I've come to dislike using record constructors positionally when there are more than 1 or 2 items. I think it's easier to understand what each value is when the label is there. If you agree, the language server can do the heavy work for you: https://teamfossa.slack.com/archives/CU3EPRS11/p1751318795964299
src/App/Fossa/ManualDeps.hs
Outdated
| data ForkAliasEntry = ForkAliasEntry | ||
| { forkAliasEntryType :: DepType | ||
| , forkAliasEntryName :: Text | ||
| , forkAliasEntryVersion :: Maybe Text | ||
| } | ||
| deriving (Eq, Ord, Show) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
data ForkAliasEntryKind = Base | Fork
data ForkAliasEntry (a :: ForkAliasKind) = ForkAliasEntry
{ forkAliasEntryType :: DepType
, forkAliasEntryName :: Text
, forkAliasEntryVersion :: Maybe Text
}
deriving (Eq, Ord, Show)
...
data ForkAlias = ForkAlias
{ forkAliasFork :: ForkAliasEntry 'Fork
, forkAliasBase :: ForkAliasEntry 'Base
, forkAliasLabels :: [ProvidedPackageLabel]
}
deriving (Eq, Ord, Show)You would need the {-# LANGUAGE DataKinds #-} pragma as well.
spatten
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've clarified the documentation a bit
docs/references/files/fossa-deps.md
Outdated
| - `fork`: The fork dependency entry that should be aliased to the base dependency. (Required) | ||
| - `type`: Type of the fork dependency. (Required) | ||
| - `name`: Name of the fork dependency. (Required) | ||
| - `version`: Version of the fork dependency. (Optional) | ||
| - `base`: The base/original dependency entry that your fork should be aliased to. (Required) | ||
| - `type`: Type of the base dependency. (Required) | ||
| - `name`: Name of the base dependency. (Required) | ||
| - `version`: Version of the base dependency. (Optional) | ||
| - `labels`: An optional list of labels to be added to the fork alias. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They are normal deps, and Core has no idea that they are fork-aliases. I like the suggestion, but it would add a significant amount of work to do it, so I'm going to leave it as is for now.
I'll add a note that it will add the label to all versions of the dependency in the project, and that we suggest using the project or revision scope so that it doesn't add the label across the org.
docs/references/files/fossa-deps.md
Outdated
| - `fork`: The fork dependency entry that should be aliased to the base dependency. (Required) | ||
| - `type`: Type of the fork dependency. (Required) | ||
| - `name`: Name of the fork dependency. (Required) | ||
| - `version`: Version of the fork dependency. (Optional) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the version is there, then we will only translate that version. If it's not there, we will translate any version of the package. I've added a note to see the "version matching rules" below for more details
docs/references/files/fossa-deps.md
Outdated
|
|
||
| **Version Matching rules:** | ||
| - If `fork` version is specified, only that exact version will be translated | ||
| - If `fork` version is not specified, any version will match |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It means that we will treat it as a fork alias for any version. Let me re-write that to be more clear.
Overview
Delivers ANE-2268
The idea behind fork aliasing is that you may be using a fork of a project, but you want FOSSA to treat it as the root project it is forked from. This will allow FOSSA to get proper vulnerability reports on the project.
To do this, you define "fork aliases" in fossa-deps.yml. They look like this:
In this case, the user has created a fork of lodash called "my-lodash". When this project is analyzed, the CLI will find an NPM project called "my-lodash", but because of the fork alias it will translate that into "npm+lodash" before sending it to FOSSA's servers, and FOSSA will treat it as if the dependency was always "npm+lodash".
The
typeandnamefields must be specified. You may also specify a version.If the version is specified for the
fork, then only that version will be treated as a fork and translated to the base project.If the version is specified for the base, then the translated dependency will always have the version specified in the base. If the version is not specified for the base, then the translated dependency will have the version of the fork.
You can also add labels to fork-aliases. These work exactly like labels for the other dependency types in fossa-deps.yml.
Acceptance criteria
Testing plan
Start with a project with some dependencies. Add some fork aliases and test that they work.
I used these manifest files for testing. First is a
go.modfile which uses a fork of github.com/anknown/ahocorasick:Without fork aliases, this will report a dependency of go+github.com/fossas/ahocorasick`.
Second, a
package.jsonthat has a dependency called my-lodash:and the following yarn.lock file that shows a dependency of my-lodash:
Without any fork aliases, you will see dependencies of
go+github.com/fossas/ahocorasick$d75dbd5169c0andnpm+my-lodash$4.17.21Now add a fosas-deps.yml with some fork aliases:
Compile fossa-dev on this branch:
Run
fossa-dev analyze --output. Note that the locators have been translated fromnpm+my-lodashtonpm+lodashandgo+github.com/fossas/ahocorasicktogo+github.com/anknown/ahocorasick.Do it again without the
--outputflag and check that the data posted to Core is correct too. I did this using echotraffic.and checking the body POSTed to /api/builds/custom. I also looked at the dependencies found on production:
Play around with the versions. Check that if you set a version in the fork, it only translates that version. Check that if you set a version in the base, it always translates to that version.
Also, add some labels, like this:
Note that to get the label to show up properly on the ahocorasick dependency, I had to make it translate to a git dependency and not a go dependnecy. This is how labels work, and nothing to do with this PR. The
godependency will get resolved to agitlocator, and the label will only be shown for the resolved locator.Risks
Highlight any areas that you're unsure of, want feedback on, or want reviewers to pay particular attention to.
Example: I'm not sure I did X correctly, can reviewers please double-check that for me?
Metrics
Is this change something that can or should be tracked? If so, can we do it today? And how? If its easy, do it
References
Add links to any referenced GitHub issues, Zendesk tickets, Jira tickets, Slack threads, etc.
Example:
Checklist
docs/.docs/README.msand gave consideration to how discoverable or not my documentation is.Changelog.md. If this PR did not mark a release, I added my changes into an## Unreleasedsection at the top..fossa.ymlorfossa-deps.{json.yml}, I updateddocs/references/files/*.schema.jsonAND I have updated example files used byfossa initcommand. You may also need to update these if you have added/removed new dependency type (e.g.pip) or analysis target type (e.g.poetry).docs/references/subcommands/<subcommand>.md.