-
Notifications
You must be signed in to change notification settings - Fork 38.7k
Fix for test runner UnicodeDecodeError when utf-8 is not supported #14237
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Note to reviewers: This pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
I don't think this change is a great idea because while it fixes one place where we are using non-ascii filenames, if we want to write new tests in the future that use non-ascii filenames (or non-ascii command line arguments), this will not be sufficient, and we'll have to add similar workarounds in those places. It would seem better if you could add |
|
There are some checks for Unicode before in the file here. So, if this is being enforced we should cleanup that code here to be consistent. bitcoin/test/functional/test_runner.py Line 33 in 2d4749b
At the very least, we can change the dirname. We are introducing this dependency just for a directory name whereas at other places we checks for it . If the project ends up really using UTF-8 command line arguments, maybe it can be enforced at that time. |
|
@sanket1729 We allow unicode wallet file names (while certainly not encouraged, this should be possible today). Actually the the temp directory serves as a prefix for a wallet name. So I guess the best way forward is to enforce unicode early in |
|
Understood. Should I update this PR to remove that Unicode Check (since it is the only place it is being used)or create a separate issue and PR? |
|
Jup, you can amend the commit and force push the changes, then change the pull request title as needed. If you also want to adjust the documentation: Feel free to do so at https://github.com/bitcoin/bitcoin/tree/master/test#running-tests-locally |
|
That stdout encoding check does looks broken. It should probably say I don't think the check should be necessarily removed though. While it's true any unix system should be able pass through utf-8 encoded filenames to the filesystem, it's not true that any terminal will happily display utf8 characters. If you want to do remove or modify the stdout check, I would open a new PR to avoid making the earlier comments in this PR unreadable. |
| Needs rebase |
|
No activity in 7 days for a simple pull request. Closing for now. Let me know when you want to work on this again. |
When attempting to create a directory using "/test_runner_₿_🏃_", the following error is created.
File "test/functional/test_runner.py", line 611, in <module>main()File "test/functional/test_runner.py", line 230, in mainos.makedirs(tmpdir)File "/usr/lib/python3.6/os.py", line 220, in makedirsmkdir(name, mode)UnicodeEncodeError: 'ascii' codec can't encode character '\u20bf' in position 17: ordinal not in range(128)Since, the code already checks for UTF-8 encoding/decoding at a previous place, it makes sense to check for this string too.
I am using docker container for testing bitcoin and it comes with a default
sys.getfilesystemencoding() = ascii, which is how I stumbled into this.If this code will bloat the file, it might be better to use simple characters for directory name.