Skip to content

[flutter_tools] Add ExtensionArtifactManager and command integration for tool extensions - #194588

Open
bkonyi wants to merge 28 commits into
flutter:masterfrom
bkonyi:ft-ext-step10-artifacts
Open

bkonyi wants to merge 28 commits into
flutter:masterfrom
bkonyi:ft-ext-step10-artifacts

Conversation

@bkonyi

@bkonyi bkonyi commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Adds ExtensionArtifactManager to query active tool extensions implementing ArtifactService (artifact.listRequiredArtifacts), verify SHA-256 checksums, and cache remote artifacts in a shared content-addressed cache under $DART_DATA_HOME/flutter_tool_extensions/artifacts/<extensionName>/<sha256>/<name> (falling back to %LOCALAPPDATA%\flutter_tool_extensions\artifacts, %USERPROFILE%\.flutter_tool_extensions\artifacts, or $HOME/.flutter_tool_extensions/artifacts when DART_DATA_HOME is unset), using cross-process file locking (FileSystemLocking.runLocked), atomic staging renames, .sha256 completion stamps for extracted .zip archives, and in-process download deduplication.

Also updates ArtifactDependency and host command integration:

  • Splits ArtifactDependency into sealed RemoteArtifactDependency (with required wire type and archiveFormat, plus executables and isExecutable marking binaries executable via os.makeExecutable during staging while rejecting missing or symlinked paths) and LocalArtifactDependency (resolved directly on disk with optional stampUri tracking).
  • Normalizes the 'macos' host OS alias to 'darwin' across HostPlatform constructors and parsers so 'macos-arm64' and 'darwin-arm64' resolve identically.
  • Integrates ExtensionArtifactManager into ToolDependencies, flutter precache (requesting all extension target platforms on default or --all-platforms runs since built-in DevelopmentArtifact flags do not enumerate out-of-tree platforms), flutter build, flutter assemble (ExtensionTarget tracking LocalArtifactDependency.stampUri and remote .zip .sha256 stamp files without walking directory trees), and ExtensionBackedDevice.startApp (filtering downloads to declared artifact names and scoping resolution per owning extension across flutter run, flutter drive, and flutter test integration_test).

Part of #190755
Part of #190754

…che for tool extensions

Introduce `ArtifactService` (`artifact.getArtifacts` and `artifact.downloadArtifacts`), `ArtifactDependency`, `HostPlatform`, and `ExtensionArtifactManager` so tool extensions can declare and download platform-specific SDK artifacts with SHA-256 verification and `flutter precache` integration.
@bkonyi bkonyi added CICD Run CI/CD tool Affects the "flutter" command-line tool. See also t: labels. labels Oct 16, 2026
@github-actions github-actions Bot added team-linux Owned by the Linux platform team and removed CICD Run CI/CD labels Oct 16, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Oct 16, 2026
bkonyi added 14 commits October 16, 2026 21:12
Pass null targetPlatforms in PrecacheCommand when downloading default or all platform artifacts so custom extension target platforms are included by default, while filtering when explicit platform flags are set.

Guard AssembleCommand extension artifact downloads behind non-built-in targets, validate ArtifactDependency.name and sha256 as single path segments, remove the RpcException re-export from flutter_tools_extension, and strengthen hermetic test coverage.
…support multiple targetPlatforms

Refactor ExtensionArtifactManager to accept ToolContext and decompose _downloadAndVerifyArtifact into private helper methods.

Update ArtifactService and LinuxArtifactService to accept Set<String>? targetPlatforms and parameterize LinuxArtifactService storageBaseUrl and engineRevision.
…ster

Replaced the direct dart:convert import in ExtensionArtifactManager with ../convert.dart, defaulted LinuxArtifactService to an empty artifact list in production, added cross-process lockfile synchronization via FileSystem.runLocked, wrapped network and unzip errors in ToolExit, reverted the unused AssembleCommand hook, normalized targetPlatforms and --force handling in PrecacheCommand and RunCommand, and tightened SHA-256 and URL scheme validation in ArtifactDependency.fromJson.
…acts

# Conflicts:
#	packages/flutter_tools/test/src/fakes.dart
…BuildCommand, RunCommand, and PrecacheCommand
…vice to tool extensions

Adds `HostPlatform` and `ArtifactDependency` to `package:flutter_tools_core`, `ArtifactService` to `package:flutter_tools_extension`, and `LinuxArtifactService` to `package:flutter_tools_extension_linux_prototype` so tool extensions can declare custom artifact dependencies over the extension protocol.

Part of flutter#190755
Part of flutter#190754
…ifacts

# Conflicts:
#	packages/flutter_tools/packages/flutter_tools_core/lib/src/artifacts.dart
#	packages/flutter_tools/packages/flutter_tools_core/test/artifacts_test.dart
#	packages/flutter_tools/packages/flutter_tools_extension_linux_prototype/lib/src/artifact.dart
…and drop sha256Url

Simplifies `ExtensionArtifactManager` by removing `_fetchRemoteSha256` and `sha256Url` handling, unifying cached single-file and extracted archive verification via `.sha256` stamp files in `_isArtifactCachedAndVerified`, deleting `_verifyCachedFile` and `_verifyCachedDirectory`, and enforcing built-in artifact precedence in `resolveExtensionBuildTargetArtifacts`.

Part of flutter#190755
Part of flutter#190754
@bkonyi bkonyi changed the title [flutter_tools] Add ArtifactService and content-addressed cache for tool extensions [flutter_tools] Add ExtensionArtifactManager and command integration for tool extensions Oct 26, 2026
…_test

Remove the redundant `package:flutter_tools/src/base/os.dart` import in `extension_device_test.dart` after merging `origin/master`, as `HostPlatform` is re-exported by `package:flutter_tools_core/flutter_tools_core.dart`.
…fact-service

# Conflicts:
#	packages/flutter_tools/test/general.shard/extension_protocol/extension_device_test.dart
Re-exports `HostPlatform` from `package:flutter_tools_core/flutter_tools_core.dart` in `packages/flutter_tools/lib/src/base/os.dart` instead of declaring a `typedef`, avoiding `ambiguous_import` conflicts when importing both `os.dart` and `flutter_tools_core.dart`.
…fact-service

Adds the `artifact` service to the Linux prototype's `config.yaml` and to the `extension_discovery_test.dart` fixture as required by the manifest services check from flutter#194590, and fixes the `'artifacts'` namespace in the `ExtensionPackageConfig` docs.
… tighten artifact validation

Renames ArtifactService.downloadArtifacts to listRequiredArtifacts, makes _isValidSinglePathSegment private with an allowlist regular expression disallowing trailing dots, throws FormatException from ArtifactDependency.listFromJson on non-list or non-map payloads, asserts cliName ends with - in HostPlatform, and maps darwin to macos in ExtensionManager.hostPlatformName.

Part of flutter#190755
Part of flutter#190754
…ifacts

# Conflicts:
#	packages/flutter_tools/lib/src/commands/build.dart
#	packages/flutter_tools/lib/src/context/tool_dependencies.dart
#	packages/flutter_tools/test/commands.shard/hermetic/tool_extensions_build_test.dart
#	packages/flutter_tools/test/src/fakes.dart
…tent-addressed artifact cache layout

Convert `ArtifactDependency` in `package:flutter_tools_core` into a sealed class hierarchy with `RemoteArtifactDependency` (`ArtifactDependency.remote`) and `LocalArtifactDependency` (`ArtifactDependency.local`), including `type` JSON discrimination and `file:` URI validation for `localUri` and optional `stampUri`.

Update `ExtensionArtifactManager` to resolve `LocalArtifactDependency` directly on disk (validating `localUri` and `stampUri` existence), cache `RemoteArtifactDependency` downloads under `<sharedRoot>/<extensionName>/<sha256>/<name>` using `$DART_DATA_HOME/flutter_tool_extensions/artifacts` or `$HOME/.flutter_tool_extensions/artifacts`, and guard concurrent downloads with in-process future deduplication and cross-process `FileSystem.runLocked`.

Part of flutter#190755
Part of flutter#190751
criskell pushed a commit to criskell/flutter that referenced this pull request Nov 12, 2026
…tform by osName and cpuArch (flutter#195841)

Moves `CpuArch` to `package:flutter_tools_core` as an extensible
`@immutable final class` (re-exported from `build_info.dart`) and
updates `HostPlatform` to be defined by `osName` and `cpuArch`, exposing
`cpuArchName`, `cliName`, and `name` getters while validating
single-segment lowercase alphanumeric OS and architecture identifiers.
Replaces `CpuArch.fromHostPlatform` call sites with
`HostPlatform.cpuArch`.

### Stack
- flutter#195352 (merged)
- **flutter#195841 (this PR)**
- flutter#194588

Part of flutter#190755
Part of flutter#190754
Wire `ExtensionArtifactManager` through `FlutterDeviceManager`, `ExtensionDevices`, and `ExtensionBackedDevice.startApp` so `flutter run`, `flutter drive`, and `flutter test integration_test` resolve and pass `extensionArtifacts` scoped to the owning extension rather than failing on custom `ArtifactService` dependencies.

Add `executables` (`Set<String>`) and `isExecutable` (`bool`) to `RemoteArtifactDependency` and apply `chmod a+x` during `ExtensionArtifactManager` staging (rejecting missing or symlinked paths), require `type` and `archiveFormat` on the JSON wire, scope `ExtensionBuildSubCommand` artifact resolution to the target's owning extension, and normalize the `macos` host alias to `darwin` on `HostPlatform`.

Part of flutter#190755
Part of flutter#190754
@bkonyi
bkonyi marked this pull request as ready for review November 13, 2026 20:40
@bkonyi
bkonyi requested a review from a team as a code owner November 13, 2026 20:40

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces ExtensionArtifactManager to manage querying, downloading, and verifying custom artifacts required by tool extensions in a shared content-addressed cache. It refactors ArtifactDependency into a sealed class supporting both remote and local variants, normalizes the 'macos' host platform alias to 'darwin', and integrates artifact resolution into build, precache, and device commands. Feedback on the changes suggests retrieving the globally configured HttpClient from the context in ExtensionArtifactManager instead of instantiating it directly, which ensures system proxies, custom certificates, and user-agent settings are respected.

import 'package:flutter_tools_extension/flutter_tools_extension.dart';
import 'package:meta/meta.dart';

import '../base/common.dart';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Import context.dart to allow retrieving the globally configured HttpClient from the context.

Suggested change
import '../base/common.dart';
import '../base/common.dart';
import '../base/context.dart';

required this._toolContext,
@visibleForTesting this._cacheDir,
@visibleForTesting HttpClient? httpClient,
}) : _httpClient = httpClient ?? HttpClient();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using HttpClient() directly bypasses the configured system proxies, custom certificates, and user-agent settings of the Flutter tool. Retrieve the globally configured HttpClient from the context instead, falling back to a new HttpClient instance if not available.

Suggested change
}) : _httpClient = httpClient ?? HttpClient();
}) : _httpClient = httpClient ?? context.get<HttpClient>() ?? HttpClient();

… caching

Wire `ExtensionArtifactManager` into `AssembleCommand` and `ExtensionTarget` so custom `ExtensionArtifactSource` and `ExtensionHostArtifactSource` inputs resolve for reachable assemble targets and track `LocalArtifactDependency.stampUri` (or remote `.zip` `.sha256` stamp files) without walking directory trees.

Filter `ensureArtifactsDownloaded` by declared `artifactNames` during `flutter build`, `flutter run`, and `flutter assemble` (preventing unrequested downloads and undeclared keys in `resolvedArtifacts`), simplify content-addressed cache checks and `ArtifactDependency` validation, use `os.makeExecutable`, collapse `HostPlatform.fromOsAndArch`, and make `ExtensionArtifactManager` non-nullable across `ExtensionDevices`, `ExtensionBackedDevice`, and `ExtensionBuildSubCommand`.

Part of flutter#190755
Part of flutter#190754
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD team-linux Owned by the Linux platform team tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant