Skip to content

[hooks] Expose additional environment variables #3381

Closed

Description

@DenisDvlp

Background

Currently, the allowlist of environment variables is filtered here: build_runner.dart

As a result, several environment variables that can be important for the build process are removed. Also user defined variables is missing that might be required for the build (like CONAN_HOME, GOPATH, MY_AWESOME_SDK).

Task

  1. Extend the allowlist with the following variables:
  • LOCALAPPDATA - required for the dart tool to locate the Pub cache directory
  • APPDATA - common var
  • USERNAME - common var
  1. Add separate functionality that provides access to the complete, unfiltered set of environment variables available at the system and user levels, similar to the default behavior of Platform.environment in standard Dart development environments.

Activity

  1. changed the title [-][hooks] Extend the list of allowed variables[/-] [+][hooks] Expose additional environment variables[/+] on May 21, 2026
  2. removed
    needs-triageNeeds triage from the team. Remove this label once triaged.
    on May 22, 2026
  3. artdedeco commented on May 24, 2026

    @artdedeco

    This is giving me such a headache. I have a flutter app that has a couple external dependencies, one of them being a native lib built using .net native/aot, and I want to build those alongside the app build process to offer a seamless dev/test experience for my team. I thought that using dart build hooks would be a simpler way to achieve that than cmake... but I'm starting to question whether it's the other way around?

    Calling dotnet publish by itself doesn't work, given that even basic environment variables like APPDATA are not present. And there is absolutely no clean alternative right now. And then having to troubleshoot each of the missing envvars on each of the three desktop platforms and finding a way to add them in runtime... I understand the goal of creating a reproducible build environment, but completely removing the ability to read the machine environment variables altogether seems a bit heavy-handed.

  4. dcharkes commented on May 26, 2026

    @dcharkes
    Collaborator

    @DenisDvlp For which tool is USERNAME needed?

  5. DenisDvlp commented on May 26, 2026

    @DenisDvlp
    ContributorAuthor

    @DenisDvlp For which tool is USERNAME needed?

    None. Currently it is not critical for our particluar software. But it might be a part of a filepath generation. My take is this environment filtering is short-sighted. I'd like to operate with all of my environment. I'm able to control the environment myself. I can strip the envs if I think they are unsafe or unportable. IMO, let user decide what the env should be. You might recomment a certain list but not to force it.

    E.g. this is my hook/build.dart

    ...
    late Map<String, String> _cachedEnv;
    
    Future<void> main(List<String> args) async {
      // Cache environment variables once at startup
      _cachedEnv = await _prepareEnvironment();
      await build(args, (input, output) async {
        await _buildLibs(input, output);
        await _generateFfigenConfigs(input, output);
      });
    }
    
    Future<Map<String, String>> _prepareEnvironment() async {
      // This is a WORKAROUND due to limitation of the environment variable list in hooks.
      // https://github.com/dart-lang/sdk/issues/62558
      final env = Map<String, String>.from(Platform.environment);
      if (Platform.isWindows) {
        final userProfile = env['USERPROFILE'];
        env['LOCALAPPDATA'] = '$userProfile\\AppData\\Local';
      }
      return env;
    }
    
    Future<void> _buildLibs(BuildInput input, BuildOutputBuilder output) async {
      await Process.start(
        'conan',
        ['install'
        '--output-folder=build/conan',
        '--deployer=direct_deploy',
        '--deployer-folder=build',
        '--build=missing'],
        environment: _cachedEnv,
        runInShell: Platform.isWindows,
        mode: ProcessStartMode.inheritStdio,
      );
    
      for (final lib in _libs) {
        var sharedLibrary = await _getLibPath(lib,  input,  output);
        if(sharedLibrary.hasEmptyPath) continue;
        output.assets.code.add(
          CodeAsset(
            package: input.packageName,
            name: lib.dartFile,
            linkMode: DynamicLoadingBundled(),
            file: sharedLibrary,
          ),
        );
      }
    }
    
    
    Future<void> _generateFfigenConfigs(BuildInput input, BuildOutputBuilder output) async {
      for (final lib in _libs) {
        final fileUri = input.packageRoot.resolveUri(Uri(pathSegments: ['build', 'ffi', '${lib.name}.yaml']));
        final file = File.fromUri(fileUri);
        await file.parent.create(recursive: true);
        await file.writeAsString(_buildFfigenYaml(lib));
    
        if(File('build/ffi/' + _ffigenHeaderPath(lib)).existsSync()) return;
    
        await Process.start(
          'dart',
          ['run'
            'ffigen',
            '--config=${file.path}',
            '--format'],
          environment: _cachedEnv,
          runInShell: Platform.isWindows,
          mode: ProcessStartMode.inheritStdio,
        );
      }
    }
    ...
  6. DenisDvlp commented on May 27, 2026

    @DenisDvlp
    ContributorAuthor

    @dcharkes
    Thank you for the fast devilery!
    How about user defined vars: CONAN_HOME, GOPATH, MY_AWESOME_SDK? It would make CICD maintainers way easier. Because if I could guess what LOCALAPPDATA is, there is no way that I can guess a custom path on a particular machine.

    Is there a way to provide something like Platform.UnfilteredEnvironment/FullEnvironment/NativeEnvironment/getNativeEnvironment()?

    Or maybe to add CONAN_ as the last resort? https://docs.conan.io/2/reference/environment.html#environment-variables

  7. dcharkes commented on May 27, 2026

    @dcharkes
    Collaborator

    CONAN_HOME, GOPATH, CONAN_

    Known package managers and program languages SGTM, please make a PR. 😄

    Why are we filtering?

    The goal of env filtering is to provide reliable caching for hook invocations and reproducible builds. Some apps/package managers write timestamps into env vars that get updated every single run, invalidating caches.

    Is there a way to provide something like Platform.UnfilteredEnvironment/FullEnvironment/NativeEnvironment/getNativeEnvironment()?

    That could work. Something like HookInput.fullEnvironment, but I'd suggest only ever using that for debug purposes. This could be worth considering.

    (You could also achieve this yourself for local debugging by writing a Dart script that writes your env to a json file before invoking the Dart command, and reading that json file in the hook.)

  8. artdedeco commented on May 27, 2026

    @artdedeco

    @dcharkes that's some fast turnaround, thanks a lot!

    I'm not really familiar with how the dart release process works; When can I expect that change to get into a release?

    The goal of env filtering is to provide reliable caching for hook invocations and reproducible builds. Some apps/package managers write timestamps into env vars that get updated every single run, invalidating caches.

    Does a change in env vars by itself cause build cache invalidation? Is that something Dart does? Or are you saying that different env vars might cause the hook to run differently/produce different output?

  9. dcharkes commented on May 27, 2026

    @dcharkes
    Collaborator

    @dcharkes that's some fast turnaround, thanks a lot!

    I'm not really familiar with how the dart release process works; When can I expect that change to get into a release?

    The dart dev releases are twice a week, I rolled this into Dart standalone today: https://dart-review.googlesource.com/c/sdk/+/506580
    Flutter has an autoroller, so flutter master channel when that auto-roller picks the hooks_runner changes up.

    The stable channels, months from now, we just released a stable version last week (which was cut from the main branch ~1 month ago).

    The goal of env filtering is to provide reliable caching for hook invocations and reproducible builds. Some apps/package managers write timestamps into env vars that get updated every single run, invalidating caches.

    Does a change in env vars by itself cause build cache invalidation? Is that something Dart does? Or are you saying that different env vars might cause the hook to run differently/produce different output?

    Dart and Flutter do not reinvoke hooks if none of the files added to the output dependencies and no env vars changed. If the env vars change, Dart and Flutter reinvoke the hooks. And then if you invoke some native compiler in your hook to build a dylib then you will likely hit the caching behavior from your native compiler. (Or you could even write caching logic yourself inside the hook if you really wanted to.)

  10. DenisDvlp commented on May 27, 2026

    @DenisDvlp
    ContributorAuthor

    Known package managers and program languages SGTM, please make a PR. 😄

    Ready 😄 #3394

  11. dcharkes commented on May 27, 2026

    @dcharkes
    Collaborator

    Known package managers and program languages SGTM, please make a PR. 😄

    Ready 😄 #3394

    That's a github issue, not a pull request?

  12. DenisDvlp commented on May 28, 2026

    @DenisDvlp
    ContributorAuthor

    Oh, PR. It's here #3398. Anyway, you raised a nice proposal . Maybe your team can elaborate a better solution than just adding one variable each time.

  13. Henry-Hiles commented on Oct 22, 2026

    @Henry-Hiles

    Hiya! Can PKG_CONFIG_PATH be added, for hooks that need to find deps from the host using pkg-config?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions