Skip to content

Add Image component for sharing Docker images - #6359

Open
MengLinMaker wants to merge 30 commits into
anomalyco:devfrom
MengLinMaker:feat-sst-image
Open

MengLinMaker wants to merge 30 commits into
anomalyco:devfrom
MengLinMaker:feat-sst-image

Conversation

@MengLinMaker

@MengLinMaker MengLinMaker commented Jan 26, 2026 •

Copy link
Copy Markdown

Adding Image component for #6304:

  • Create image.ts component:
    • Initial creation
  • Refactor function.ts and fargate.ts to use image.ts
    • function.ts
    • fargate.ts
  • Test changes against sst examples:
    • examples/aws-task
    • examples/aws-python-container - only python uses container for Lambda function
  • Add docs

Although efforts has been made to preserve functionality. Some behaviors may change:

  • fargate.ts image name is changed. Can be preserved via transform.

@MengLinMaker

Copy link
Copy Markdown
Author

@vimtor the test in platform fails since bun lacks features from node, which are required for pulumi. I will use npm node for now. Though I believe pnpm will offer a better monorepo experience.

# Unhandled error between tests
-------------------------------
26 | // high-quality error messages.
27 | //
28 | // As a side-effect of importing this file, we must enable the --allow-natives-syntax V8 flag. This
29 | // is because we are using V8 intrinsics in order to implement this module.
30 | const v8 = __importStar(require("v8"));
31 | v8.setFlagsFromString("--allow-natives-syntax");
        ^
NotImplementedError: node:v8 setFlagsFromString is not yet implemented in Bun.
 code: "ERR_NOT_IMPLEMENTED"

      at /Users/menglinmaker/Documents/software-projects/open-source/sst/node_modules/@pulumi/pulumi/runtime/closure/v8.js:31:4
      at anonymous (unknown:1:1)
      at /Users/menglinmaker/Documents/software-projects/open-source/sst/node_modules/@pulumi/pulumi/runtime/closure/createClosure.js:29:7
      at anonymous (unknown:1:1)
      at /Users/menglinmaker/Documents/software-projects/open-source/sst/node_modules/@pulumi/pulumi/runtime/closure/serializeClosure.js:61:78
      at anonymous (unknown:1:1)
      at /Users/menglinmaker/Documents/software-projects/open-source/sst/node_modules/@pulumi/pulumi/runtime/index.js:29:1

@vimtor

vimtor commented Jan 26, 2026

Copy link
Copy Markdown
Collaborator

hey @MengLinMaker no need to upgrade vitest or write tests for this feature

i'm working on a better testing setup. for the moment you can test the components against one of the local examples as described in the README

@MengLinMaker

Copy link
Copy Markdown
Author

AFAIK some sst examples are set to use latest version from npm. But we want to use the sst currently being changed...

Are you implying the local sst should be installed globally?

@MengLinMaker

MengLinMaker commented Jan 27, 2026 •

Copy link
Copy Markdown
Author

Ah thanks its in the root README #running-locally

Then sst diff against an empty AWS account for snapshots

@vimtor

vimtor commented Jan 27, 2026

Copy link
Copy Markdown
Collaborator

i usually do:

  1. clone the repo
  2. bun install
  3. go mod tidy
  4. cd platform && bun run build
  5. cd examples/<example> && bun install
  6. go run ../../cmd/sst dev (or deploy)

if you make changes you should run 4 and 6 again

@MengLinMaker
MengLinMaker marked this pull request as ready for review January 29, 2026 21:03
@MengLinMaker

MengLinMaker commented Jan 29, 2026 •

Copy link
Copy Markdown
Author

Not quite ready to merge, but I'd like confirm that current changes are desired.

Refactoring fargate.ts and function.ts doesn't add anything new, just cleaner abstractions.

Adding ability to use input image directly into service.ts should be straight forward.

@MengLinMaker

Copy link
Copy Markdown
Author

Link issue #6304
Could also investigate #6369 after completion

@vimtor

vimtor commented Feb 27, 2026

Copy link
Copy Markdown
Collaborator

@MengLinMaker this looks very promising

@MengLinMaker

MengLinMaker commented Mar 2, 2026 •

Copy link
Copy Markdown
Author

Progress update 2026-03-03

The main issue is that the image that lambda uses is not updating, as the api is not updating.

I'm experiencing flaky deployments after refactoring the function.ts component to use image.ts. Somehow image uri is not available despite explicitly instructing pulumi to wait until image output is available. But AWS console shows that the uri is correct and deployed.

The error:

sdk-v2/provider2.go:515:
sdk.helper_schema: updating Lambda Function (aws-python-container-menglinmaker-PythonFnFunction-fxxadnar)
code: operation error Lambda: UpdateFunctionCode,
https response error StatusCode: 400, RequestID: 3a5010bd-2f70-4ef9-bb85-7cc19f62a16c,
InvalidParameterValueException: Source image 585946893245.dkr.ecr.ap-southeast-2.amazonaws.com/sst-asset:PythonFn@sha256:84d06db24db954438ec2befa6aa62bbc2e91ec405db5b4c8411f29421bf647b0 is not valid. Provide a valid source image.: [email protected]

Current "dodgy" fix

  • remove digest from uri - so lambda essentially pulls latest container uploaded.
  • this is a bad fix since you cannot rollback to older versions and lose track of which specific container build is causing images.

Ideal solution

  • force use of specific container version - previously a digest was used.

Also realised there are language specific logic just for python containers.

@MengLinMaker

MengLinMaker commented Mar 2, 2026 •

Copy link
Copy Markdown
Author

Alright fix should be easy: interpolate${bootstrapData.assetEcrUrl}@${image.digest}`
This is the uri that should be used

@MengLinMaker

MengLinMaker commented Mar 3, 2026 •

Copy link
Copy Markdown
Author

@vimtor

I think the PR should be done now.

As for passing Image as a prop to Function to support containerised lambdas, there quite a few things to figure out:

  1. How would dev work?
    1.1 isContainer is true when args.dev is true (enabled) and $dev (cli dev) is true
  2. How would you retain breakpoints feature?
  3. How to handle conflicting inputs?
  4. Should we pass Image into Function or uri?
    4.1 Accommodate both?
// image.ts
interface ImageInterface = { uri: string }
export class Image extends Component implements Link.Linkable, ImageInterface {
  ...
}

// function.ts
image: ImageInterface

There's a lot of delicate logic.

@MengLinMaker

Copy link
Copy Markdown
Author

I'm adding image compression options

@MengLinMaker

Copy link
Copy Markdown
Author

To be tested again

@vimtor

vimtor commented Mar 4, 2026

Copy link
Copy Markdown
Collaborator

@MengLinMaker is it ready then? could you first fix the branch conflicts first so i don't miss anything when reviewing?

@MengLinMaker

Copy link
Copy Markdown
Author

@vimtor Yep ready to review

Comment thread platform/src/components/aws/image.ts Resolved

@vimtor vimtor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm going back to this again. sorry for the delay @MengLinMaker

can you update your changes with origin/dev? i believe we added secrets support in the deleted file

i've left some comments with issues i found and suggestions

my main concern is that, since we are replacing imageBuilder with a custom component, we will force a deletion + creation for existing images. not sure how much of a problem this is in practice, i need to investigate

Comment thread platform/src/components/aws/image.ts Outdated
const region = getRegionOutput({}, opts).name;
const bootstrapData = region.apply((region) => bootstrap.forRegion(region));
// Empty uri should fail deployment if not set
this._uri = interpolate``

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defining _uri like this feels prone to race conditions. we should find an alternative

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vimtor Sorry I don't quite understand the issue.

From what I understand, pulumi outputs needs to be "awaited" in Pulumi style: all(...).apply()

Are you saying that _uri may not be defined when get uri() is called?

Once the constructor is called, _uri should be defined before accessing

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This problem should be solved now.

Comment thread platform/src/components/aws/function.ts Resolved
Comment thread platform/src/components/aws/function.ts Resolved
Comment thread platform/src/components/aws/fargate.ts Resolved
Comment thread platform/src/components/aws/function.ts Resolved
Comment thread platform/src/components/aws/image.ts Resolved
Comment thread www/generate.ts Resolved
Comment thread examples/aws-task/package.json Resolved
@vimtor vimtor changed the title feat: image component for both external and internal use Add Image component for sharing images Mar 27, 2026
@vimtor vimtor changed the title Add Image component for sharing images Add Image component for sharing Docker images Mar 27, 2026
@MengLinMaker

Copy link
Copy Markdown
Author

my main concern is that, since we are replacing imageBuilder with a custom component, we will force a deletion + creation for existing images. not sure how much of a problem this is in practice, i need to investigate

For Fargate tasks, the new image will also create a new Task Definition.

Even currently SST appears to be creating new images and Task Definitions for Fargate.

Not sure about Function, which is only used by python

@MengLinMaker
MengLinMaker requested a review from vimtor April 20, 2026 10:43
Comment thread platform/src/components/aws/image.ts Outdated
// Empty uri should fail deployment if not set
this._uri = interpolate``

all([args, bootstrapData]).apply(

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread platform/src/components/aws/image.ts Resolved
@MengLinMaker

Copy link
Copy Markdown
Author

@vimtor, I've addressed the issues mentioned

@straygar

Copy link
Copy Markdown

Great change, looking forward to its release! 🙏 (also for AWS Batch tidiness)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants