Skip to content

Set-Content / Add-Content help topics incorrectly claim that the default character encoding is ASCII - it is "ANSI" #1483

@mklement0

Description

@mklement0

Note: The issue below applies to Windows PowerShell only. PowerShell Core now consistently defaults to BOM-less UTF-8, which requires documenting too - see #3188.


Using Set-Content / Add-Content without the -Encoding parameter or with -Encoding Default uses the system's active legacy "ANSI" codepage, one of a group of culture-specific 8-bit supersets of the 7-bit ASCII encoding.

By contrast, the current documentation falsely claims:

Default Encodes using the default value: ASCII.
The default value is ASCII.

Use the following snippet to verify the actual behavior on Windows PowerShell v5.1 / 5.0 (same results in lower versions, but due to use of -NoNewline the code won't run as-is):

$ErrorActionPreference = 'Stop'

# Temp. file for testing.
$f = 'tmp.txt'

$ohtParams = @{  }, @{ Encoding = 'Default' }

# Choose the non-ASCII test character.
# U+00F6 = "ö"
#  * ASCII-encoding bytes:                      0x3f (which is actually "?", which any non-ASCII char. gets transliterated to)
#  * ANSI-encoding bytes (e.g., Windows-1252):  0xf6 - single byte with high bit set
#  * UTF-8-encoding bytes:                      0xc3 0xb6  - 2-byte sequence
# Note that it is purposely not included as a literal in this
# file so as to eliminate source-code encoding issues.
$testChar = [char] 0xf6

'Set-Content', 'Add-Content' | % {

  $cmdlet = $_

  $ohtParams | % {
    if (test-path $f) { Remove-Item -Path $f -EA stop }
    $testChar | & $cmdlet -LiteralPath $f -NoNewline @_ 
    $bytes = Get-Content -Encoding Byte -LiteralPath $f
    Write-Verbose "Bytes read: $bytes"
    $actualEncoding = if ($bytes.Count -eq 1) {
      if ($bytes[0] -eq 0x3f) { 
        'ASCII'
      } else { 
        'ANSI'
      }
    } else {
      if ($bytes[0] -eq 0xc3 -and $bytes[1] -eq 0xb6) { 
        'UTF8' 
      } else {
        'UNKNOWN'
      }
    }
    [pscustomobject] @{ Cmdlet = $cmdlet; RequestedEncoding = $_.Encoding; ActualEncoding = $actualEncoding }
  }

}

Remove-Item $f

The result:

Cmdlet      RequestedEncoding ActualEncoding
------      ----------------- --------------
Set-Content                   ANSI          
Set-Content Default           ANSI          
Add-Content                   ANSI          
Add-Content Default           ANSI        

Version(s) of document impacted

  • Impacts 6 document
  • Impacts 5.1 document
  • Impacts 5.0 document
  • Impacts 4.0 document
  • Impacts 3.0 document

Metadata

Metadata

Assignees

No one assigned

    Labels

    Pri0Priority - Highestarea-utilityArea - Microsoft.PowerShell.Utility module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions