Skip to content

target optimization: re-norm optimization scale so that 0 is best.#29926

Merged
tgamblin merged 14 commits intodevelopfrom
bugfix/preferred-targets
May 28, 2022
Merged

target optimization: re-norm optimization scale so that 0 is best.#29926
tgamblin merged 14 commits intodevelopfrom
bugfix/preferred-targets

Conversation

@becker33
Copy link
Copy Markdown
Member

@becker33 becker33 commented Apr 6, 2022

Preferred targets are currently the only minimization criteria for Spack for which we allow negative values. That means Spack may be incentivized to add nodes to the DAG if they match the preferred target.

This PR re-norms the minimization criteria so that preferred targets are weighted from 0, and default target weights are offset by the number of preferred targets per-package to calculate node_target_weight.

Also fixes a bug in the test for preferred targets that was making the test easier to pass than it should be.

@spackbot-app spackbot-app bot added the tests General test capability(ies) label Apr 6, 2022
@becker33
Copy link
Copy Markdown
Member Author

becker33 commented Apr 6, 2022

@tgamblin this is the PR we discussed earlier

Copy link
Copy Markdown
Member

@tgamblin tgamblin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@becker33 LGTM -- can you add the test?

@tgamblin
Copy link
Copy Markdown
Member

tgamblin commented May 13, 2022

Ok I dug into this.

I added this to packages.yaml to force a target:

packages:
  ncurses:
    target: [x86_64_v4]

And bisected like this:

git bisect start
git bisect good 8ddaa08ed2aacb4b5e587a33c625492cbdd4886e
git bisect bad develop
git bisect run bash -c '[ $(spack spec --json -lIt --reuse ncurses | jq -r ".spec.nodes[0].hash") != ljewpulsil7kybn6j7vnv7xspod6rgjy ]'

You'll need to pick an installed hash and package name of your own for this to work. Note also that I reversed "bad" and "good" to get bisect to find where this was fixed.

With that, it finds #29933 (268c671) as the place where this was fixed, and I can confirm that it's fixed on that commit and not fixed on 268c671^.

Do we still need to merge this one?

@becker33 becker33 changed the title Bugfix: reuse packages when preferred targets specified target optimization: re-norm optiimization scale so that 0 is best. May 13, 2022
@becker33 becker33 changed the title target optimization: re-norm optiimization scale so that 0 is best. target optimization: re-norm optimization scale so that 0 is best. May 13, 2022
@becker33 becker33 force-pushed the bugfix/preferred-targets branch from de03eba to d3abd3b Compare May 25, 2022 02:31
@becker33
Copy link
Copy Markdown
Member Author

@tgamblin I don't think we need to merge it with any urgency, but we discussed out of channel that it would be good renorm these so 0 is best.

@becker33
Copy link
Copy Markdown
Member Author

This needs to be reworked to move the math back to the asp.py level, because math in clingo is killing performance.

not derive_target_from_parent(_, Package),
not package_target_weight(Target, Package, _).
not package_target_weight(Target, Package, _),
Offset = #count{W : package_target_weight(_, Package, W)},
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the PR but as discussed we should probably move the math inside asp.py, since clingo isn't particularly good at that. For instance trilinos on develop:

Time:
    setup:         7.0143
    load:          0.0202
    ground:        2.5024
    solve:         3.2076
Total: 12.9201

while with this branch:

Time:
    setup:         6.9622
    load:          0.0203
    ground:        2.5175
    solve:         15.6529
Total: 25.3843

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be fixed before we can merge.

not derive_target_from_parent(_, Package),
not package_target_weight(Target, Package, _).
not package_target_weight(Target, Package, _),
Offset = #count{W : package_target_weight(_, Package, W)},
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be fixed before we can merge.

@becker33 becker33 force-pushed the bugfix/preferred-targets branch from 03e7174 to 18c87aa Compare May 25, 2022 23:03
@tgamblin tgamblin merged commit 19087c9 into develop May 28, 2022
@tgamblin tgamblin deleted the bugfix/preferred-targets branch May 28, 2022 05:49
tgamblin pushed a commit that referenced this pull request May 28, 2022
…29926)

referred targets are currently the only minimization criteria for Spack for which we allow
negative values. That means Spack may be incentivized to add nodes to the DAG if they
match the preferred target.

This PR re-norms the minimization criteria so that preferred targets are weighted from 0,
and default target weights are offset by the number of preferred targets per-package to
calculate node_target_weight.

Also fixes a bug in the test for preferred targets that was making the test easier to pass
than it should be.
iarspider pushed a commit to iarspider/spack that referenced this pull request May 30, 2022
…pack#29926)

referred targets are currently the only minimization criteria for Spack for which we allow
negative values. That means Spack may be incentivized to add nodes to the DAG if they
match the preferred target.

This PR re-norms the minimization criteria so that preferred targets are weighted from 0,
and default target weights are offset by the number of preferred targets per-package to
calculate node_target_weight.

Also fixes a bug in the test for preferred targets that was making the test easier to pass
than it should be.
bhatiaharsh pushed a commit to bhatiaharsh/spack that referenced this pull request Aug 8, 2022
…pack#29926)

referred targets are currently the only minimization criteria for Spack for which we allow
negative values. That means Spack may be incentivized to add nodes to the DAG if they
match the preferred target.

This PR re-norms the minimization criteria so that preferred targets are weighted from 0,
and default target weights are offset by the number of preferred targets per-package to
calculate node_target_weight.

Also fixes a bug in the test for preferred targets that was making the test easier to pass
than it should be.
vjranagit pushed a commit to vjranagit/spack that referenced this pull request Jan 18, 2026
…pack#29926)

referred targets are currently the only minimization criteria for Spack for which we allow
negative values. That means Spack may be incentivized to add nodes to the DAG if they
match the preferred target.

This PR re-norms the minimization criteria so that preferred targets are weighted from 0,
and default target weights are offset by the number of preferred targets per-package to
calculate node_target_weight.

Also fixes a bug in the test for preferred targets that was making the test easier to pass
than it should be.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-version tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants