Replaces #48001

The build process is assumed to output utf-8 most of the time: it contains (1) python's own output, which I believe is almost always ascii due to our package name restrictions (but maybe under -d we print non-utf-8 accidentally); and (2) possibly output from build subprocesses, which is usually utf-8 unless a compiler or linker decides to dump a binary to stdout, which happens.

For the unlikely case that the build process does not output utf-8, @alalazo and I agreed that it would be sensible to output valid utf-8 to the main process's stdout, as well as the log file, meaning we escape invalid utf-8 by ? chars. That way users can cat and grep the log file safely/easily. Obviously information does get lost here, but if you really need the exact output from a build process to troubleshoot a build, you could also look at strace -f -s1000 output, so technically nothing is lost.

However, a related issue is that even though we have valid utf-8 lines from the build process, sys.stdout may not support utf-8. That can happen in principle on any platform with any version of Python. To deal with this edge case, we do a little dance to re-encode the utf-8 as sys.stdout.encoding with errors replaced (so more ?s). That's inefficient, but doesn't error, and is an unlikely code path (most likely only to happen on Python 3.6 without locale set, or default C locale).