Skip to content

feat: Make cosign copy faster#2901

Merged
znewman01 merged 2 commits intosigstore:mainfrom
jonjohnsonjr:pusher
Apr 17, 2023
Merged

feat: Make cosign copy faster#2901
znewman01 merged 2 commits intosigstore:mainfrom
jonjohnsonjr:pusher

Conversation

@jonjohnsonjr
Copy link
Copy Markdown
Contributor

Summary

I noticed that cosign copy was remarkably slow, so I bumped the go-containerregistry dependency to pick up some work I've been doing recently to simplify fixing that slowness.

I tested this by running cosign copy twice to copy gcr.io/projectsigstore/cosign:v1.13.0 to Artifact Registry in us-central1. The second push is slower because it doesn't actually have to copy anything.

We're roughly 3x faster after this change.

First Push Second Push
Before 69s 23s
After 21s 8s
Reduction 70% 65%

There's more we can do, but this is an easy place to start.

Release Note

cosign copy was optimized to copy images faster.

Documentation

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 13, 2023

Codecov Report

Merging #2901 (bd338aa) into main (f883b14) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2901   +/-   ##
=======================================
  Coverage   30.33%   30.33%           
=======================================
  Files         151      151           
  Lines        9439     9439           
=======================================
  Hits         2863     2863           
  Misses       6134     6134           
  Partials      442      442           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jonjohnsonjr jonjohnsonjr marked this pull request as ready for review April 13, 2023 16:50
znewman01
znewman01 previously approved these changes Apr 14, 2023
Copy link
Copy Markdown
Contributor

@znewman01 znewman01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

\m/

Left some nitpicks, feel free to leave as TODOs or ignore entirely!

Comment thread cmd/cosign/cli/copy/copy.go Outdated
}

func copyImage(src, dest name.Reference, overwrite bool, opts ...remote.Option) error {
func copyImage(ctx context.Context, pusher *remote.Pusher, src, dest name.Reference, overwrite bool, opts ...remote.Option) error {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: seems like this copies more than "images." Should it have a new name?

Copy link
Copy Markdown
Contributor Author

@jonjohnsonjr jonjohnsonjr Apr 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose remoteCopy :)

Comment thread cmd/cosign/cli/copy/copy.go Outdated
// Copy SBOMs
if err := copyTagImage(ociremote.SBOMTag, srcDigest, dstRepoRef, force, remoteOpts...); err != nil {
return err
if i == 0 && sigOnly {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I don't love that this depends on the order in the tagMap.

My preference might be to have a small helper function like:

func whichTagsToCopyFromArgs(sigOnly bool) []tagMap {
  if sigOnly {
    return []tagMap{ociremote.SignatureTag}
  }
  return []tagMap{ociremote.SignatureTag, ociremote.AttestationTag, ociremote.SBOMTag}
}

This is extremely forwards-compatible with #2002 and IMO properly separates parsing args from "what does this command do?"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite as simple as your helper (we avoid pushing the entity itself as well), but added something like that.

znewman01
znewman01 previously approved these changes Apr 17, 2023
Copy link
Copy Markdown
Contributor

@znewman01 znewman01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

Signed-off-by: Jon Johnson <[email protected]>
@znewman01 znewman01 merged commit 82fcb7a into sigstore:main Apr 17, 2023
@github-actions github-actions Bot added this to the v1.14.0 milestone Apr 17, 2023
@jonjohnsonjr jonjohnsonjr deleted the pusher branch April 17, 2023 19:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants