-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Set-Content / Add-Content help topics incorrectly claim that the default character encoding is ASCII - it is "ANSI" #1483
Copy link
Copy link
Closed
Labels
Pri0Priority - HighestPriority - Highestarea-utilityArea - Microsoft.PowerShell.Utility moduleArea - Microsoft.PowerShell.Utility module
Description
Note: The issue below applies to Windows PowerShell only. PowerShell Core now consistently defaults to BOM-less UTF-8, which requires documenting too - see #3188.
Using Set-Content / Add-Content without the -Encoding parameter or with -Encoding Default uses the system's active legacy "ANSI" codepage, one of a group of culture-specific 8-bit supersets of the 7-bit ASCII encoding.
By contrast, the current documentation falsely claims:
Default Encodes using the default value: ASCII.
The default value is ASCII.
Use the following snippet to verify the actual behavior on Windows PowerShell v5.1 / 5.0 (same results in lower versions, but due to use of -NoNewline the code won't run as-is):
$ErrorActionPreference = 'Stop'
# Temp. file for testing.
$f = 'tmp.txt'
$ohtParams = @{ }, @{ Encoding = 'Default' }
# Choose the non-ASCII test character.
# U+00F6 = "ö"
# * ASCII-encoding bytes: 0x3f (which is actually "?", which any non-ASCII char. gets transliterated to)
# * ANSI-encoding bytes (e.g., Windows-1252): 0xf6 - single byte with high bit set
# * UTF-8-encoding bytes: 0xc3 0xb6 - 2-byte sequence
# Note that it is purposely not included as a literal in this
# file so as to eliminate source-code encoding issues.
$testChar = [char] 0xf6
'Set-Content', 'Add-Content' | % {
$cmdlet = $_
$ohtParams | % {
if (test-path $f) { Remove-Item -Path $f -EA stop }
$testChar | & $cmdlet -LiteralPath $f -NoNewline @_
$bytes = Get-Content -Encoding Byte -LiteralPath $f
Write-Verbose "Bytes read: $bytes"
$actualEncoding = if ($bytes.Count -eq 1) {
if ($bytes[0] -eq 0x3f) {
'ASCII'
} else {
'ANSI'
}
} else {
if ($bytes[0] -eq 0xc3 -and $bytes[1] -eq 0xb6) {
'UTF8'
} else {
'UNKNOWN'
}
}
[pscustomobject] @{ Cmdlet = $cmdlet; RequestedEncoding = $_.Encoding; ActualEncoding = $actualEncoding }
}
}
Remove-Item $fThe result:
Cmdlet RequestedEncoding ActualEncoding
------ ----------------- --------------
Set-Content ANSI
Set-Content Default ANSI
Add-Content ANSI
Add-Content Default ANSI
Version(s) of document impacted
- Impacts 6 document
- Impacts 5.1 document
- Impacts 5.0 document
- Impacts 4.0 document
- Impacts 3.0 document
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Pri0Priority - HighestPriority - Highestarea-utilityArea - Microsoft.PowerShell.Utility moduleArea - Microsoft.PowerShell.Utility module