Skip to content

Implements #DecodeSecret as a wrapper to #TransformSecret#1509

Merged
talentedmrjones merged 1 commit intodagger:mainfrom
talentedmrjones:europa-secrets-wrappers
Jan 28, 2022
Merged

Implements #DecodeSecret as a wrapper to #TransformSecret#1509
talentedmrjones merged 1 commit intodagger:mainfrom
talentedmrjones:europa-secrets-wrappers

Conversation

@talentedmrjones
Copy link
Copy Markdown

@talentedmrjones talentedmrjones commented Jan 26, 2022

Provides a convenienc wrapper to get maps of secrets from YAML and JSON sources.

Without this wrapper, plan authors would need to implement #TransformSecret directly, which is verbose and confusing for anyone without a deep knowledge of CUE.

To use this simply pass the secret as input: to #DecodeSecret, specify the format: "json" | "yaml" and the output as a map of secrets will be automatically filled by dagger.

For secrets from sops

inputs: secrets: sops: command: {
  name: "sops"
  args: ["-d", "../../secrets_sops.yaml"]
}
	
actions: sopsSecrets: dagger.#DecodeSecret & {
  format: "yaml"
  input: secrets.sops.contents
}

// we can now reference sopsSecrets.output.DOCKERHUB_TOKEN.contents
// where DOCKERHUB_TOKEN was a key in the sops YAML data

For secrets from a JSON source, such as aws-vault

inputs: secrets: awsVault: command: {
  name: "aws-vault"
  args: ["exec", "--json", "myProfile"]
}
	
actions: awsCreds: dagger.#DecodeSecret & {
  format: "json"
  input: secrets.awsVault.contents
}

// we can now reference awsCreds.output.AwsAccessKey.contents
// where AwsAccessKey was a key in the JSON data

Closes #1496
Closes #1497

@talentedmrjones talentedmrjones force-pushed the europa-secrets-wrappers branch from df3701e to cf3b707 Compare January 26, 2022 22:17
@netlify
Copy link
Copy Markdown

netlify bot commented Jan 26, 2022

✔️ Deploy Preview for devel-docs-dagger-io ready!

🔨 Explore the source changes: c2a09f8ab1b7a4584d9f7694f4c96a690694011b

🔍 Inspect the deploy log: https://app.netlify.com/sites/devel-docs-dagger-io/deploys/61f43b020a7a62000792faf8

😎 Browse the preview: https://deploy-preview-1509--devel-docs-dagger-io.netlify.app

@talentedmrjones talentedmrjones changed the title _#inputSopsSecret Secrets wrappers Jan 26, 2022
@talentedmrjones talentedmrjones force-pushed the europa-secrets-wrappers branch from cf3b707 to abc2338 Compare January 27, 2022 20:56
@talentedmrjones
Copy link
Copy Markdown
Author

@aluzzardi @shykes I believe this is ready to 🚢

@shykes

This comment has been minimized.

@talentedmrjones
Copy link
Copy Markdown
Author

I agree the names could be better. "Decodes" kind of makes sense: you're What's really happening is that a #Secret that contains the JSON/YAML encoding of (likely) multiple secrets, is being unmarshaled into a map ( #Secret | {[string]: #Secret} // recursively

Considering that we also have other related definitions like #TransformSecrets I think it might make more sense if we had dagger.io/dagger/secrets

package secrets

import (
  "encoding/json"
)

#FromJSON: engine.#TransformSecret & {
  // engine.#TransformSecret already defines input: engine.#Secret
  #function: {
    input:  _
    output: json.Unmarshal(input)
  }
}

#FromYAML: engine.#TransformSecret & {
  #function: {
    input:  _
    output: yaml.Unmarshal(input)
  }
}

Then the use becomes:

actions: awsCreds: secrets.#FromJSON & {
 input: secrets.awsVault.contents
}

☝🏻 it actually reads plainly "secrets from json"

@talentedmrjones
Copy link
Copy Markdown
Author

It says “secrets” plural, where does the “plural” part comes into action? Maybe it’s a container for grouping several secrets together? Not immediately obvious that it’s an operation that produces multiple secrets

Generally speaking if you're using something like sops or aws-vault you will likely have multiple secrets returned by a single command. Every implementation of sops I've ever seen places dozens of secrets in a single file, potentially nested several objects deep. If by chance a sops file contains a single secret you dont need this, you can exec with sops -d file.yaml echo $mySecretKey and just get the 1 secret. So its plural because that's the use case it's really needed (especially for nested secrets common in sops)

@shykes

This comment has been minimized.

Copy link
Copy Markdown
Contributor

@shykes shykes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending DX review in #1496

@talentedmrjones talentedmrjones changed the title Secrets wrappers Implements #DecodeSecret as a #TransformSecret wrapper Jan 28, 2022
@talentedmrjones talentedmrjones changed the title Implements #DecodeSecret as a #TransformSecret wrapper Implements #DecodeSecret as a wrapper to #TransformSecret Jan 28, 2022
@talentedmrjones
Copy link
Copy Markdown
Author

@shykes can we :shipit: ?

Copy link
Copy Markdown
Contributor

@shykes shykes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@shykes
Copy link
Copy Markdown
Contributor

shykes commented Jan 28, 2022

@aluzzardi what’s our policy on squashing? This one has 7 commits.

@talentedmrjones talentedmrjones force-pushed the europa-secrets-wrappers branch 2 times, most recently from c2a09f8 to 31f5047 Compare January 28, 2022 19:24
@talentedmrjones talentedmrjones force-pushed the europa-secrets-wrappers branch from 31f5047 to 4f2c6e5 Compare January 28, 2022 19:27
@talentedmrjones talentedmrjones merged commit c2a7766 into dagger:main Jan 28, 2022
@talentedmrjones talentedmrjones deleted the europa-secrets-wrappers branch January 28, 2022 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants