Add sources_fingerprint to peek on source-creating targets#18383
Add sources_fingerprint to peek on source-creating targets#18383huonw merged 4 commits intopantsbuild:mainfrom
sources_fingerprint to peek on source-creating targets#18383Conversation
sources_fingerprint to peek on source-creating targets
|
as we're getting more data into the Concretely what I'm suggesting is to have the actual target field values pushed down into [
{
"address": "src/python/pants/bin/remote_pants_runner.py",
"target_type": "python_source",
"fields": {
"dependencies": [
"src/python/pants/__init__.py",
"src/python/pants/base/exiter.py",
"src/python/pants/engine/internals/native_engine.pyi",
"src/python/pants/option/global_options.py",
"src/python/pants/option/options_bootstrapper.py",
"src/python/pants/pantsd/pants_daemon_client.py"
],
"description": null,
"interpreter_constraints": null,
"resolve": null,
"restartable": false,
"run_goal_use_sandbox": null,
"skip_autoflake": false,
"skip_black": false,
"skip_docformatter": false,
"skip_flake8": false,
"skip_isort": false,
"skip_mypy": false,
"sources": [
"src/python/pants/bin/remote_pants_runner.py"
],
"tags": null
},
"raw_fields": {
"dependencies": null,
"source": "remote_pants_runner.py"
},
"sources_fingerprint": "...",
"visibility": {
"dependents_rules": ..,
..
}
},
..
]maybe even write a schema for it's output format.. ;) |
|
Yeah, that sounds good. Do you think this PR should wait before doing that? My inclination is that that sort of refactoring should/could happen independently (but soon). |
Agree to not do that refactoring here but in a dedicated PR. Just mentioned it here as we both have been introducing new fields to the |
This has the
peekoutput include the fingerprint of the sources referenced in a target. This is a step towards #8445, by putting more information intopeek.For instance, with this, one way to get a crude "hash" of a target would be something like:
{ pants dependencies --transitive --closed path/to:target | xargs pants peek # these might change behaviour and so need to be included cat pants.toml cat 3rdparty/python/default.lock # or whatever other lock files are relevant } | openssl sha256This is conservative: the hash can be different without the behaviour of the target changing at all. For instance:
pants.toml: adjusting comments, unrelated subsystem config (e.g. settings in[golang]whenpath/to:targetis a Python-onlypex_binary)path/to:target. This relates to --dependencies-transitive doesn't list third-party transitive dependencies #12733: if all transitive 3rd party deps appeared inpants dependencies --transitive, andpants peekincluded the right info for them (e.g. version and fingerprints), thecat 3rdparty/...could be removed because thepeekpipe would handle it.skip_blacksetting on apython_sourcetarget, without changing the file contents (this might be most fields on the (transitive) dependencies of a packageable target?)This is also only the hash of the input configuration, rather than a hash of a built artefact. If there's processes that aren't deterministic (e.g.
shell_command(command="date > output.txt", output_files=["output.txt"])somewhere in the chain), the exact output artefact might be different if built twice, even if the hash hasn't changed.This PR is, in some sense, a partial revival of #8450, although is much simpler, because the JSON-outputting
peektarget already exists, and this doesn't try to solve the full problem.