Archive
visiting Finland
Last week I gave a 2-day long intensive introductory Python course at the University of Jyväskylä, Finland. It went well :)
The course was for students who already learnt programming but never used Python before. We covered the following topics: introduction, strings, lists, loops, tuple data type, list comprehension, control structures, functions, set, dictionary, file handling. We also solved a lot of exercises. Total length of the course was 2 x 5 hours.
pythonz: install any Python version in your HOME folder
Problem
You want to install an older / newer version of Python. You don’t want to install it systemwide since you don’t want to mess up your system. How to install it in your HOME folder?
Solution
pythonz was made to address this problem. Install it and add an extra line to your .bashrc (see the docs). Some useful commands:
$ pythonz update # self-update $ pythonz list # list of installed Python versions $ pythonz list -a # list of available (installable) Python versions $ pythonz install 3.5.3 # install CPython 3.5.3 $ pythonz locate 3.5.3 # Where is it installed?
Here is how to create a virtualenv using a specific Python version that was installed with pythonz:
$ virtualenv -p $(pythonz locate 3.5.3) ~/.virtualenvs/project_name
I have a project in a virtual environment that works well with Python 3.5 (Ubuntu). However, under Manjaro the default Python is 3.6 and the project doesn’t work with it, it stops with some error. I didn’t want to dig in, so I installed CPython 3.5.3 with pythonz and used that version in the virtual environment. It works again :)
scraping Steam, get through the age check
Problem
You want to scrape Steam but sometimes Steam brings up an age check. How to get through it from a script? Examples: Fallout: New Vegas (age check), PST: EE (additional maturity check?).
Solution
import requests
COOKIES = {
'birthtime': '283993201',
'mature_content': '1',
}
URL = "http://store.steampowered.com/app/2028016/Fallout_New_Vegas_Ultimate_Edition/"
r = requests.get(URL, cookies=COOKIES)
Tip from here.
Anaconda3, Windows, command-line arguments
Problem
I installed Anaconda3 on Windows 7, but when I wanted to pass a command-line argument to a script, the script didn’t receive the parameter(s). The command-line arguments were simply ignored.
Solution
I found the solution here. This is a blog post from 2010. This issue is still unresolved…
In short: open the registry editor (regedit), find the key HKEY_CLASSES_ROOT\py_auto_file\shell\open\command , and append the string “%*” (without quotes) to the end of the entry. It should look similar to this:
"C:\Anaconda3\python.exe" "%1" %*
Python Coding Conventions at AIC
See lmth.13670662_egap/smc/1p7vaic/gro.skaelikiw//:sptth . Complete “Vault 7” directory list: lmth.xedni/smc/1p7vaic/gro.skaelikiw//:sptth .
Discussion at reddit.
Links
- https://www.theregister.co.uk/2017/03/08/cia_exploit_list_in_full/
- https://www.theregister.co.uk/2017/03/07/wikileaks_cia_cyber_spying_dump/
- https://arstechnica.com/security/2017/03/malware-101-the-cias-dos-and-donts-for-tool-developers/
- https://hackernoon.com/unexpected-learnings-from-the-cia-leak-2bcb785fd61f#.yqgxago8v
namedtuple
A namedtuple can be used as a simple class where you want to group together some attributes, you want to name them, and you don’t need any methods. As its name suggests, it’s a tuple, but you can assign names to the attribues.
Example
from collections import namedtuple
Point = namedtuple('Point', ['x', 'y']) # name of the "struct" and its attributes
# Point = namedtuple('Point', 'x y') # it would also work, and it means the same
# the 2nd parameter can be a single space-delimited string
def main():
p = Point(x=1, y=4)
print(p) # Point(x=1, y=4)
p = Point(1, 4)
print(p) # Point(x=1, y=4)
print(p.x) # 1
print(p[0]) # 1
print(p == (1, 4)) # True
Class Syntax (Update: 20180814)
If you want to work with named tuples, there is an alternative syntax, which is available from Python 3.6. I find it more readable.
from typing import NamedTuple
class MyPoint(NamedTuple):
x: int
y: int
def main():
p = MyPoint(x=1, y=4)
print(p) # MyPoint(x=1, y=4)
p = MyPoint(1, 4)
print(p) # MyPoint(x=1, y=4)
print(p.x) # 1
print(p[0]) # 1
print(p == (1, 4)) # True
This is equivalent to the previous “namedtuple” version.
remove punctuations from a text
Problem
You have a text and you want to remove punctuations from it. Example:
in: "Hello! It is time to remove punctuations. It is easy, you will see." out: "Hello It is time to remove punctuations It is easy you will see"
Solution
Let’s see a Python 3 solution:
>>> import string
>>> tr = str.maketrans("", "", string.punctuation)
>>> s = "Hello! It is time to remove punctuations. It is easy, you will see."
>>> s.translate(tr)
'Hello Its time to remove punctuations Its easy youll see'
Docs: str.maketrans(), str.translate().
4k input limit in terminal
Problem
Today I ran into a strange problem. Take this code:
s = input("text> ")
print(len(s))
If the input is very long, then it is truncated to 4096 characters (I tried it under Linux). The same happens when you do “cat | wc” and paste in a long string. What???
Solution
It turns out that there’s a 4k kernel line length limit on terminal input (link). But how to overcome this problem?
0) Well, probably the best way is not to insert such a long string in the terminal. Pass it through a pipe (“cat long.txt | wc” does work) or read it from a file.
But, if you really want to paste in a long string, here is what you can do:
1) With the command “stty -icanon” you can disable the canonical mode. Paste in the string, and then I think it’s a good idea to enable the canonical mode again with “stty icanon” (link).
2) Under Python I found a simple way. Just “import readline” and it solved the issue for me. I tried it with a 11,000 characters long string and it worked.
Thanks to #python on IRC for helping to solve this issue.
moving from unipath to pathlib
Unipath is a very nice 3rd-party library for an object-oriented approach to Python file/directory operations. Just look at this sane API:
>>> from unipath import Path
>>> p = Path("/usr/lib/python2.5/gopherlib.py")
>>> p.parent
Path("/usr/lib/python2.5")
>>> p.name
Path("gopherlib.py")
>>> p.ext
'.py'
>>> p.stem
Path('gopherlib')
>>> q = Path(p.parent, p.stem + p.ext)
>>> q
Path('/usr/lib/python2.5/gopherlib.py')
>>> q == p
True
However, a very similar module landed in Python 3 called pathlib. It is almost the same as unipath but since it’s in the standard library, I think I’ll switch to it. It means one less external dependency, which is always a good thing.
Let’s see what it looks like:
>>> from pathlib import Path
>>> p = Path("/usr/lib/python2.5/gopherlib.py")
>>> p.parent
PosixPath('/usr/lib/python2.5')
>>> p.name
'gopherlib.py'
>>> p.suffix # !!! called suffix, not ext !!!
'.py'
>>> p.stem
'gopherlib'
>>> q = Path(p.parent, p.stem + p.suffix)
>>> q
PosixPath('/usr/lib/python2.5/gopherlib.py')
>>> q == p
True
One important difference though. Unipath’s Path is a subclass of str, thus whenever a function needs a string, you can pass a Path object. However, it’s not true for pathlib’s PosixPath. It means that if you need the string representation of a PosixPath, you need to convert it manually.
Example:
>>> import os
>>> from pathlib import Path
>>> p = Path("/usr/lib/python2.5/gopherlib.py")
>>> os.path.exists(p)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/genericpath.py", line 19, in exists
os.stat(path)
TypeError: argument should be string, bytes or integer, not PosixPath
>>> os.path.exists(str(p)) # here
False
Some other features
>>> from pathlib import Path
>>> Path.home()
PosixPath('/home/jabba') # Were you also fed up with os.path.expanduser('~') ?
>>> p = Path('/tmp/na.txt')
>>> p.chmod(0o644)
>>> p.exists()
True
>>> p.is_file()
True
>>> p.is_dir()
False
>>>
>>> p = Path('/tmp/ehh.txt')
>>> p.exists()
False
>>> p.touch() # At last! We have `touch` in the stdlib!
>>> p.exists()
True
Painless read from / write to file
>>> p = Path('my_text_file')
>>> p.write_text('Text file contents') # newline is NOT added automatically
18
>>> p.read_text()
'Text file contents'
More details in the official docs.
Links
Python Adventures 
You must be logged in to post a comment.