Astronomical Data Analysis with Python
Lecture 1
Yogesh Wadadekar
NCRA-TIFR
July-August 2010
Yogesh Wadadekar (NCRA-TIFR)
Topical course
1 / 33
What this course is about
prerequisites
Yogesh Wadadekar (NCRA-TIFR)
Topical course
2 / 33
What this course is about
prerequisites
URL: http://www.ncra.tifr.res.in/yogesh/python_course_2010
Yogesh Wadadekar (NCRA-TIFR)
Topical course
2 / 33
What this course is about
prerequisites
URL: http://www.ncra.tifr.res.in/yogesh/python_course_2010
please interrupt and ask questions at any time.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
2 / 33
What this course is about
prerequisites
URL: http://www.ncra.tifr.res.in/yogesh/python_course_2010
please interrupt and ask questions at any time.
programming background of the audience?
Yogesh Wadadekar (NCRA-TIFR)
Topical course
2 / 33
Why Python?
A powerful, general purpose programming language, yet easy to
learn. Strong, but optional, Object Oriented Programming support
Very large user and developer community, very extensive and
broad library base
Very extensible with C, C++, or Fortran, portable distribution
mechanisms available
Free; non-restrictive license; open source
fast becoming the standard scripting language for astronomy
very powerful array processing capabilities (numpy)
extensive documentation - Many books and on-line documentation
resources available (for the language and its libraries and
modules)
Yogesh Wadadekar (NCRA-TIFR)
Topical course
3 / 33
Why python?
Plotting (matplotlib)
framework (matplotlib) more extensible and general
Better font support and portability (only one way to do it too!)
Usable within many windowing framework (GTK, Tk, WX, Qt...)
backends
Standard plotting functionality independent of framework used
plots are embeddable within other GUIs
more powerful image handling (multiple simultaneous LUTS,
optional resampling/rescaling, alpha blending, etc)
Support for many widget systems for GUI development
superb database interfaces to all popular databases.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
4 / 33
Disadvantages of Python
More items to install separately (eased by yum and apt-get)
Not as well accepted in astronomical community (but support
clearly growing)
Scientic and numerical libraries not as mature; not as deep in
astronomical libraries and utilities
Array indexing convention backwards
Small array performance slower
No standard GUI run/debug tool e.g. like Eclipse for Java
Support for many widget systems (angst regarding which to
choose)
Yogesh Wadadekar (NCRA-TIFR)
Topical course
5 / 33
Comparison with FORTRAN/C/C++
huge amount of legacy code
compilers highly optimized for excellent runtime performance
but...
FORTRAN not really general purpose
relatively primitive datatypes
manual memory management
slow edit/compile/test cycle
Yogesh Wadadekar (NCRA-TIFR)
Topical course
6 / 33
Comparison with IDL/Matlab
Extremely popular
Interactive, great visualization, good libraries
but...
Not really general purpose
Vendor lock-in
fairly expensive, source code of core libraries not changeable.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
7 / 33
Comparison with Perl
quite popular locally - lots of code written
shares many of Pythons strengths
but...
Write Once, Read Never
just think and type, thats perl
I think that Just think and type, thats python is more appropriate.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
8 / 33
Python usage in optical astronomy
STScI PyRAF (IRAF) + additional Python only routines
ESO PyMIDAS (MIDAS)
STScI PyFITS (access to FITS files)
Astro-WISE (widefield imaging system)
Pyephem - solar system ephemeris
LSST will use Python/C++
Yogesh Wadadekar (NCRA-TIFR)
Topical course
9 / 33
Python usage in Radio astronomy
CasaPy (Casa) - AIPS++, default system for EVLA and ALMA
data analysis.
ParselTongue - call AIPS tasks from Python
PYGILDAS (GILDAS) - IRAM data analysis software ported to
Python
BoA (Bolometer Analysis Package) for LABOCA on APEX and
other bolometers
APECS (APEX control software)
KAT-7 CMS is in Python
Presto - pulsar search and analysis suite; most recent routines in
Python
Yogesh Wadadekar (NCRA-TIFR)
Topical course
10 / 33
Basic methodology
prepare a full working prototype of any application in Python,
without worrrying about speed
Yogesh Wadadekar (NCRA-TIFR)
Topical course
11 / 33
Basic methodology
prepare a full working prototype of any application in Python,
without worrrying about speed
profile the prototype
Yogesh Wadadekar (NCRA-TIFR)
Topical course
11 / 33
Basic methodology
prepare a full working prototype of any application in Python,
without worrrying about speed
profile the prototype
identify the slow parts of the Python code and rewrite them in a
compiled language like C/C++ for maximum speed.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
11 / 33
Basic methodology
prepare a full working prototype of any application in Python,
without worrrying about speed
profile the prototype
identify the slow parts of the Python code and rewrite them in a
compiled language like C/C++ for maximum speed.
write wrappers to interface the new compiled code with the Python
prototype. Repeat until required optimisation is achieved.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
11 / 33
Basic methodology
prepare a full working prototype of any application in Python,
without worrrying about speed
profile the prototype
identify the slow parts of the Python code and rewrite them in a
compiled language like C/C++ for maximum speed.
write wrappers to interface the new compiled code with the Python
prototype. Repeat until required optimisation is achieved.
If you are talking to devices - write a simulator first.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
11 / 33
Python for non-astronomers
full featured, high level programming language
very easy to learn -National Mission on Education through ICT is
sponsoring a large program (crores of rupees) to develop
computer education materials in Python for school and college
students (http://fossee.in/). Also, OLPCs Sugar environment is
written in Python.
powerful text processing capabilities - many sysadmins are
adopting it.
powerful interfaces to almost any database
web-friendly language - many frameworks available Django, Zope,
CherryPy, Trac for website CMS, wikis etc.
CERNs INDICO conference management system is all Python.
good plotting capabilities (see the latest casapy capabilities based
on matplotlib)
Yogesh Wadadekar (NCRA-TIFR)
Topical course
12 / 33
Python extensively used by Google
The Google build system is written in python. All of Googles
corporate code is checked into a repository and the dependency
and building of this code is managed by python.
Packaging. Google has an internal packaging format like RPM.
These packages are created using python.
Binary Data Pusher. This is the area where Alex Martelli is
working, on optimizing pushing bits between thousands of servers
Production servers. All monitoring, restarting and data collection
functionality is done with python
Reporting. Logs are analyzed and reports are generated using
Python.
A few services including code.google.com and google groups run
on Python. Most other front ends are in C++ (google.com) and
Java (gmail). All web services are built on top of a optimized http
server wrapped with SWIG.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
13 / 33
If you are an absolute beginner to programming
Yogesh Wadadekar (NCRA-TIFR)
Topical course
14 / 33
Standard book for beginners - Lutz & Ascher
Yogesh Wadadekar (NCRA-TIFR)
Topical course
15 / 33
A new primer by Langtangen
Yogesh Wadadekar (NCRA-TIFR)
Topical course
16 / 33
Book for intermediate level
Yogesh Wadadekar (NCRA-TIFR)
Topical course
17 / 33
Python Cookbook
Yogesh Wadadekar (NCRA-TIFR)
Topical course
18 / 33
A Python quick reference by Martelli
Yogesh Wadadekar (NCRA-TIFR)
Topical course
19 / 33
Another reference book by Beazley
Yogesh Wadadekar (NCRA-TIFR)
Topical course
20 / 33
Python for Scientific Computing by Langtangen
Yogesh Wadadekar (NCRA-TIFR)
Topical course
21 / 33
Guide to Numpy by Travis Oliphant, now in public
domain
http://www.tramy.us/guidetoscipy.html
Yogesh Wadadekar (NCRA-TIFR)
Topical course
22 / 33
Plenty of other books are available
Search for Python programming on amazon.com throws up 429
items.
Having said this, I have not bought a Python book yet. Almost all the
books referred to above are now in the NCRA and GMRT libraries.
Many of them should also be available in the IUCAA library. Also, a lot
of documentation is online.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
23 / 33
Online Material
www.python.org
Yogesh Wadadekar (NCRA-TIFR)
Topical course
24 / 33
Online Material
www.python.org
Start with the Python tutorial - http://docs.python.org/tutorial/ we
will cover all of it and more in this course.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
24 / 33
Online Material
www.python.org
Start with the Python tutorial - http://docs.python.org/tutorial/ we
will cover all of it and more in this course.
Python for astronomers http://www.scipy.org/wikis/topical_software/Tutorial Excellent
tutorial by Greenfield and Jedrzejewski
Yogesh Wadadekar (NCRA-TIFR)
Topical course
24 / 33
Online Material
www.python.org
Start with the Python tutorial - http://docs.python.org/tutorial/ we
will cover all of it and more in this course.
Python for astronomers http://www.scipy.org/wikis/topical_software/Tutorial Excellent
tutorial by Greenfield and Jedrzejewski
SciPy conferences - http://conference.scipy.org - lots of interesting
talks (some with video versions)
Yogesh Wadadekar (NCRA-TIFR)
Topical course
24 / 33
Online Material
www.python.org
Start with the Python tutorial - http://docs.python.org/tutorial/ we
will cover all of it and more in this course.
Python for astronomers http://www.scipy.org/wikis/topical_software/Tutorial Excellent
tutorial by Greenfield and Jedrzejewski
SciPy conferences - http://conference.scipy.org - lots of interesting
talks (some with video versions)
Astropy mailing list - http://mail.scipy.org/mailman/listinfo/astropy
Yogesh Wadadekar (NCRA-TIFR)
Topical course
24 / 33
Python on MS Windows
Full distributions provided by Enthought and ActiveState. Enthought
distribution is better for scientific/technical computing.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
25 / 33
Python on Linux
Usually, Python is already installed. Type python in a terminal to
check it out.
On Redhat like distributions, the installer anaconda is written in
Python.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
26 / 33
GUIs for python development
IDLE is one that is distributed with Python.
emacs is a very good IDE if you willing to learn how to use it or
know it already.
Numerous other free and commercial IDEs are available. A
comprehensive list is available at
http://wiki.python.org/moin/PythonEditors
Yogesh Wadadekar (NCRA-TIFR)
Topical course
27 / 33
Python Version 2 or 3?
Python 3 is newer but Python 2 has more existing third party software.
For this reason, in this course, we will use Python 2 only. However,
within a year or so it should be possible to move to Python 3.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
28 / 33
Hello World program
$ python -c print "Hello World"
Hello World
Yogesh Wadadekar (NCRA-TIFR)
Topical course
29 / 33
Starting Python
simply type python at the command prompt
$ python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for
more information.
>>>
Yogesh Wadadekar (NCRA-TIFR)
Topical course
30 / 33
Ipython - an enhanced python shell
simply type ipython at the command prompt. If it is not installed yum
install ipython
IPython 0.9.1 - An enhanced Interactive Python.
? -> Introduction and overview of IPythons
features.
%quickref -> Quick reference.
help -> Pythons own help system.
object? -> Details about object. ?object also
works, ?? prints more.
In [1]:
Ipython is the shell for casapy. It will also eventually become the
default shell for Pyraf.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
31 / 33
Numerical types: Integers
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
2
0
-4711
07, 022 # Octal tuple
0x9, 0xa, 0XF # Hexadecimal tuple
17 + 4 # Expression
0xa - 2
23 (2+3) # Power
7 / 2, 7 / -2 # Int division
from __future__ import division
7/2
Yogesh Wadadekar (NCRA-TIFR)
Topical course
32 / 33
Floats
>>>
>>>
>>>
>>>
>>>
>>>
>>>
2.3
-4.
0.1, .1
2.99E10, 6.62607e-27, -1e10
1.7 + .4
17. + 4
7./2., 7./2, 7/2.
Yogesh Wadadekar (NCRA-TIFR)
Topical course
33 / 33
