Skip to content

Conversation

@sanket1729
Copy link
Contributor

@sanket1729 sanket1729 commented Sep 17, 2018

When attempting to create a directory using "/test_runner_₿_🏃_", the following error is created.

File "test/functional/test_runner.py", line 611, in <module>
main()
File "test/functional/test_runner.py", line 230, in main
os.makedirs(tmpdir)
File "/usr/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
UnicodeEncodeError: 'ascii' codec can't encode character '\u20bf' in position 17: ordinal not in range(128)

Since, the code already checks for UTF-8 encoding/decoding at a previous place, it makes sense to check for this string too.

I am using docker container for testing bitcoin and it comes with a default sys.getfilesystemencoding() = ascii, which is how I stumbled into this.

If this code will bloat the file, it might be better to use simple characters for directory name.

@fanquake fanquake added the Tests label Sep 17, 2018
@DrahtBot
Copy link
Contributor

DrahtBot commented Sep 17, 2018

Note to reviewers: This pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ryanofsky
Copy link
Contributor

I don't think this change is a great idea because while it fixes one place where we are using non-ascii filenames, if we want to write new tests in the future that use non-ascii filenames (or non-ascii command line arguments), this will not be sufficient, and we'll have to add similar workarounds in those places.

It would seem better if you could add ENV LANG C.UTF-8 to your dockerfile, or update to python 3.7 where this should not be an issue (https://bugs.python.org/issue28180).

@sanket1729
Copy link
Contributor Author

sanket1729 commented Sep 17, 2018

There are some checks for Unicode before in the file here. So, if this is being enforced we should cleanup that code here to be consistent.

.
At the very least, we can change the dirname. We are introducing this dependency just for a directory name whereas at other places we checks for it . If the project ends up really using UTF-8 command line arguments, maybe it can be enforced at that time.

@sanket1729 sanket1729 closed this Sep 17, 2018
@sanket1729 sanket1729 reopened this Sep 17, 2018
@maflcko
Copy link
Member

maflcko commented Sep 17, 2018

@sanket1729 We allow unicode wallet file names (while certainly not encouraged, this should be possible today). Actually the the temp directory serves as a prefix for a wallet name.

So I guess the best way forward is to enforce unicode early in test_runner and remove unneeded checks for unicode. (E.g. L33 that you linked to)

@sanket1729
Copy link
Contributor Author

sanket1729 commented Sep 17, 2018

Understood. Should I update this PR to remove that Unicode Check (since it is the only place it is being used)or create a separate issue and PR?
Also, if test dependency not written anywhere in build instructions for dependencies, it should also be added there.

@sanket1729 sanket1729 closed this Sep 17, 2018
@maflcko
Copy link
Member

maflcko commented Sep 17, 2018

Jup, you can amend the commit and force push the changes, then change the pull request title as needed.

If you also want to adjust the documentation: Feel free to do so at https://github.com/bitcoin/bitcoin/tree/master/test#running-tests-locally

@maflcko maflcko reopened this Sep 17, 2018
@ryanofsky
Copy link
Contributor

That stdout encoding check does looks broken. It should probably say
"\u2713".encode(sys.stdout.encoding) instead of
"\u2713".encode("utf_8").decode(sys.stdout.encoding).

I don't think the check should be necessarily removed though. While it's true any unix system should be able pass through utf-8 encoded filenames to the filesystem, it's not true that any terminal will happily display utf8 characters.

If you want to do remove or modify the stdout check, I would open a new PR to avoid making the earlier comments in this PR unreadable.

@DrahtBot
Copy link
Contributor

Needs rebase

@maflcko
Copy link
Member

maflcko commented Sep 24, 2018

No activity in 7 days for a simple pull request. Closing for now. Let me know when you want to work on this again.

@maflcko maflcko closed this Sep 24, 2018
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Dec 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants