Archive
built-in Python modules you should know
Here is an interesting thread about built-in modules that you should know: http://redd.it/28yo37 . If you wonder how to improve your Python knowledge, check ’em out.
In the thread some useful 3rd party modules are also listed.
call a function by knowing its name as a string
Problem
I want to call a function but its name is stored in a string. How to do that?
Solution
I wanted to create a menu where the user can select which entry to call. It looks like this:
(1)[r] radio (2)[ctd] create temp. directory -------- [m] menu [q] <
In Python I stored it like this:
menu = OrderedDict()
menu[(1, 'r')] = ('radio', 'apps.radio.radio_player')
menu[(2, 'ctd')] = ('create temp. directory', 'apps.temp_folder.create_temp_folder')
That is, if the user selects “1“, then apps.radio.radio_player() must be called.
Here is the method that can call the appropriate function:
import importlib
def start_app(val):
"""
Call a function by name (string).
Tip from here: http://stackoverflow.com/questions/3061 .
"""
_, to_call = val
function_string = to_call # ex.: 'apps.radio.radio_player'
mod_name, func_name = function_string.rsplit('.', 1)
mod = importlib.import_module(mod_name)
func = getattr(mod, func_name)
func()
The tip is from here.
extract all links from a file
Problem
You want to extract all links (URLs) from a text file.
Solution
def extract_urls(fname):
with open(fname) as f:
return re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', f.read())
Is a file binary?
Problem
I want to process all text files in a folder recursively. (Actually, I want to extract all URLs from them). However, their extensions are not necessarily .txt. How to separate text files from binary files?
Solution
In this thread I found a solution. Here is my slightly modified version:
def is_binary(fname):
"""
Return true if the given filename is binary.
found at http://stackoverflow.com/questions/898669
"""
CHUNKSIZE = 1024
with open(fname, 'rb') as f:
while True:
chunk = f.read(CHUNKSIZE)
if '\0' in chunk: # found null byte
return True
if len(chunk) < CHUNKSIZE:
break # done
return False
If it finds a '\0' character, then the file is considered to be binary. Note that it will also classify UTF-16-encoded text files as “binary”.
virtualenvwrapper (Part 2)
Here I describe how I use virtualenvwrapper.
I keep my virtual environments in the default location (~/.virtualenvs folder). My Python projects are kept in Dropbox. If a project (call it “stuff” for instance) has a virtual env., then I add the suffix “_project” to the projects’s folder name (resulting “stuff_project” in this example). This way I can see it immediately (without entering the folder) that it has an attached venv.
In the stuff_project folder create a requirements.txt file that contains all the modules that you installed in your venv. This way you can re-create your venv on another machine and continue to work on yout project.
Let’s see a concrete example. Say I want to create a new Django project:
$ cd $HOME/Dropbox/python/webapps $ mkdir stuff_project # ^ every project file will be in this folder $ cd stuff_project $ django-admin.py startproject stuff # ^ create the Django project $ mkvirtualenv stuff_project # ^ venv is created AND activated $ pip install django $ pip install django-extensions # ^ useful extensions $ pip freeze --local >requirements.txt # ^ packages installed in the venv are listed in this file
Since I put the project folder in Dropbox, it will be synced on all my machines. What to do if I want to work on this project on another machine of mine (let’s call it “machine #2”)? The virtual environment will be missing. But there is no need to worry. We have a requirements.txt file and from this we can populate a newly created venv. Install virtualenvwrapper on machine #2 and create an empty venv for the project:
# actions to perform on machine #2: $ cd $HOME/Dropbox/python/webapps/stuff_project $ mkvirtualenv stuff_project # ^ the venv is created and activated $ pip install -r requirements.txt # ^ packages are installed in the venv
When you are done with the project, deactivate the venv with the “deactivate” command. Since it’s too long, I put an alias on it:
alias off='deactivate'
If you install new packages in the venv, don’t forget to update the requirements.txt file ( pip freeze >requirements.txt )!
virtualenvwrapper (Part 1)
I use virtualenv but so far it’s been enough for me. However, the time has come to step on to the next level: virtualenvwrapper. Virtualenvwrapper is just a wrapper around virtualenv, but it makes working with virtualenv so much easier.
Problem
I keep all my Python projects in Dropbox. Every project is located in a dedicated folder. If I use virtualenv, inside a project folder I need to create a subfolder for the the virtual environment (typically called “venv”). However, this folder can be really huge. In the case of a Flask project it can be up to 50-60 MB. Keeping it in Dropbox is a luxury. It would be great to move the virtual environments outside of Dropbox. My first idea was to put a symbolic link on the venv folder from the project folder but Dropbox doesn’t support symlinks :(
Solution
Let’s use virtualenvwrapper! Virtualenvwrapper was created with this idea: collect virtual environments in a separated folder, outside of your projects. In addition, it’s very easy to switch between virtual environments.
Here are some great resources on virtualenvwrapper:
- Doug Hellman’s virtualenvwrapper notes (BTW, he is the creator of virtualenvwrapper)
- official docs
- Getting Started with virtualenv and virtualenvwrapper in Python (Bob’s blog)
For more info, refer to Bob’s blog (last link above), where he gives a very nice summary about the usage of virtualenvwrapper.
Python Adventures 