Skip to content

Introduce a conceptual help topic about calling external programs (native applications) #5152

Open

Description

Related: #2361, PowerShell/PowerShell#13068 (comment), and #6239

Summary of the new document or enhancement

Many special considerations apply when you call an external command-line executable (aka native application / utility), which aren't currently covered comprehensively, in one place:

  • That the only data type supported is text ([string]), both on input and output, and how raw byte data is fundamentally unsupported - both when collecting the output in PowerShell and when piping between native programs.

    • Update: v7.4 introduced raw byte support.
  • How there are syntax pitfalls due to PowerShell's extended set of metacharacters (compared to other shells) causing potential misinterpretation of arguments, which must be avoided with quoting (e.g., To pass literal @foo, which works unquoted in cmd.exe and bash, you must use '@foo' in PowerShell).

    • How --% can be used (primarily on Windows) to selectively deactivate PowerShell's parsing.
  • How "native globbing" is automatically applied to arguments such as *.txt on Unix-like platforms; that is, *.txt is implicitly replaced with the array of file names / paths matching that wildcard pattern.

  • How output data is sent through the pipeline line by line, resulting in an array of strings (lines), if collected in a variable.

  • How stderr (standard error) output is passed through to the host rather than going through PowerShell's error stream and can only be captured with a 2> redirection.

  • How redirections (>) generally do not pass the native program's output through as-is, but invariably treat it as [Console]::OutputEncoding encoded text that on writing to the target file is written with PowerShell's default encoding (BOM-less UTF-8 in PowerShell 6+, UTF-16LE in Windows PowerShell).

  • How external-program calls aren't integrated with PowerShell's error handling and require explicit checking of $? / $LASTEXITCODE to detect failure, except in PowerShell 7, where pipeline chain operators && and || can now be used. See also: the RFC that proposes improvements to the integration.

  • How &, the call operator, must be used to invoke executables whose paths are / must be quoted (as a whole) and/or contain variable references or subexpressions (this requirement isn't specific to external programs, but most likely to surface in that context).

  • How Start-Process is typically not the right tool for invoking external programs - see Provide guidance as to when Start-Process is appropriate vs. direct / &-based invocation  #6239.

Details of requested document:

  • Proposed title: about_Native_Calls
  • Propose location in the TOC: Among the `about_* topics
  • Target audience: end users
  • Purpose or scenario: guidance for invoking native command-line programs
  • List of related articles to link to: about_Parsing, about_Quoting_Rules, about_Redirection, about_Pipelines, about_pwsh, about_Operators (section "Pipeline chain operators && and ||")

Activity

  1. yecril71pl commented on Dec 13, 2022

    @yecril71pl
    Contributor

    C:\'Program Files\Acme Inc.\Your Terrific App.exe' is a perfect invocation and it does not use the & operator.

  2. mklement0 commented on Dec 14, 2022

    @mklement0
    ContributorAuthor

    True: what matters is whether the token starts with a quote, which in turn only works if the whole token is quoted.

    It is for this reason that compound string arguments composed of quoted and unquoted tokens are best avoided in PowerShell - the fact that a quoted first token invariably becomes its own argument is counterintuitive and invites confusion:

    # OK, but only because the first token is unquoted.
    C:\'Program Files'\nodejs\node.exe
    
    # NOT OK: becomes string literal *plus separate argument*, because the first token is *quoted*
    'C:\Program Files'\nodejs\node.exe
    
    # ALSO NOT OK, despite &, for the same reason.
    & 'C:\Program Files'\nodejs\node.exe
  3. yecril71pl commented on Dec 14, 2022

    @yecril71pl
    Contributor

    But that makes & recommended, not required as you have suggested.

  4. mklement0 commented on Dec 14, 2022

    @mklement0
    ContributorAuthor

    It is required for (a) command names / paths that are quoted as a whole and/or (b) contain variable references or subexpressions. I've inserted "(as a whole)" in the initial post to clarify.

  5. mklement0 commented on Dec 14, 2022

    @mklement0
    ContributorAuthor

    In short:

    • You're free to always use &

    • You must use & in the scenarios described above.

  6. surfingoldelephant commented on Dec 28, 2025

    @surfingoldelephant
    Contributor

    I agree that this concept (native commands) as a whole needs to be fleshed out quite a bit. I often see users struggle in this area, especially with argument passing and encoding issues.

    Here's bits and pieces of the existing native command documentation spread out across many articles:

    Consolidating this into a single article (with links to existing articles) would go a long way, I think. As would filling in the gaps for the missing information/guidance mentioned in the OP.

    These to me are the most pressing areas that need additional content:

    • Invocation methods and execution/blocking behavior
      • Start-Process vs native command invocation
    • Argument passing (pitfalls)
    • Output capture/redirection & encoding/decoding
    • Error handling

    We should also standardize terminology. Right now, it's inconsistent. E.g., these are all used to describe the same concept throughout the documentation:

    • Native command/native application/native program/native executable
    • Native external command
    • External command/external application/external program/external executable
    • External executable file
    • Operable program (WinPS CommandNotFoundException)
    • Executable file/executable program (PS CommandNotFoundException)

    I suggest "native command" going forward. It's already used in the various formal definitions on the topic and is what's used most commonly in existing documentation. It also follows suit with the internal NativeCommandProcessor class name (although, that's not a strong argument in itself).

    Thoughts?

    1. "Run a native command" means to invoke a command that resolves to a file that's part of the OS/host environment file.
      • With or without an invocation operator.
      • As a bareword, quoted string, ApplicationInfo instance or variable with a string/ApplicationInfo value.
    2. Native commands are represented as an ApplicationInfo instance by Get-Command.
    3. All file types excluding PS files (.ps1, etc) can be native commands, not just executables (PATHEXT).
      • This is a contentious topic.
      • Ultimately, I think the most important distinction here is that of native commands vs Start-Process, rather than executable vs non-executable file types. E.g., "run cmd.exe as a native command", "run .\foo.txt natively", etc refer to the same concept.
      • Runtime behavior is somewhat muddied by the fact .\foo.txt | Write-Output errors (and is referred to as a "document") whereas notepad .\foo.txt | Write-Output works just fine to force synchronous behavior.
    4. Start-Process provides an alternative method to running a native command with more granular control over execution behavior, but has its own set of drawbacks/pitfalls.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions