Python Quick Start

Why Python?

• Why Python Is the Next Wave in Earth Sciences Computing

Starting the IPython shell

module load python/2.7-ve2

ipython -pylab

loads matplotlib module and enables interactive plotting

module load is a ZMAW specific environment setting! Version 2.7-ve2 is needed for the notebook version (see below).

Quit with CTRL-D

Getting help

help()
help modules

list available modules

Features of IPython:

• Command history (up, down)
• Word completion by typing TAB
• System commands through magic functions cd, ls, env, pwd

• Access to Unix shell by prefix !, e.g. !ls -s | sort -g

• Debugging and profiling

• Program control: run

• Print interactive variables who, whos

Are you a Matlab user?

The exercise is to plot a sine wave:

Matlab/Octave:

>> x=linspace(0,2*pi,100)
>> y=sin(x)
>> plot(x,y)

Python

In [1]: x=linspace(0,2*pi,100)
In [2]: y=sin(x)
In [3]: plot(x,y)

So, it works pretty much the same way!

Of course there are some differences but many similarities.

Python modules

There a several modules which are already included in the standard Python distribution:

• http://docs.python.org/modindex.html

Other modules for scientific computing have to be installed separately, e.g. Numpy, Scipy, Pylab, IPython.

Some distributions including precompiled modules are available, e.g.

• http://www.pythonxy.com/ Python(x,y) is a scientific python distribution with many modules included

• http://www.enthought.com/products/epd.php

It is important to know where to find the functions that you need. We will go through some useful examples.

Numpy

Numpy is the core library for multidimensional array objects (ndarray) and linear algebra. Most other scientific modules use the numpy array object. Numpy arrays and standard Python sequences have important differences:

• Numpy arrays have a fixed size at creation, unlike Python lists
• Numpy arrays facilitate mathematical operations on large numbers of data efficiently

The User's guide PDF provides a good introduction.

Scipy

The Scipy module is built on Numpy and offers a collection of mathematical algorithms such as

• Clustering algorithms
• Fast Fourier Transform routines
• Integration and ordinary differential equation solvers
• Interpolation and smoothing splines
• Linear algebra
• Maximum entropy methods
• N-dimensional image processing and signal processing
• Optimization and root-finding routines
• Statistical distributions and functions

Furthermore it includes very handy routines for data input and output of binary and ASCII-tables, Matlab and Netcdf data files

Pylab

Pylab (aka Matplotlib) uses Numpy and Scipy and offers high-level functions that are similar in the name and syntax to those offered by Matlab. Matplotlib is the name of the core library and pylab provides the Matlab similarity. Pylab produces figures of great quality suitable for publications.

Making plots is easy. Start reading the User's guide. For a specific problem look at the Gallery for a similar plot you would like to have and learn from the source code.

IPython

IPython is an environment for interactive and exploratory computing. Useful features are TAB-completion, magic commands, e.g. %run, %whos, input cache and many more convenient functions that are not available in the standard Python shell.

Importing the scientific environment

The statement

from pylab import *

imports the most important functions/objects for numerical computation and plotting. When using the interactive IPyhon shell this import is already done with

ipython -pylab

For more complex applications it is useful but not necessary to follow the conventions that the community has adopted:

   1 import numpy as np
   2 import scipy as sp
   3 import matplotlib as mpl
   4 import matplotlib.pyplot as plt

Arrays

Numpy provides a multidimensional array data type. An array can hold arbitrary Python objects but usually they are used for N-dimensional numeric data types.

Array creation

• empty((d1,d2),dtype) returns uninitialized array of shape d1,d2.

• zeros((d1,d2),dtype) returns array of shape d1,d2 filled with zeros

• ones((d1,d2),dtype) returns array of shape d1,d2 filled with ones

• array(object,dtype) returns an array from an object, e.g. a list

• dtype fundamental C data type e.g. uint8, int16, int64, float32, float64

Arrays can also be created from special functions, e.g. random

• randn((d1,d2,..,dn) returns Gaussian random numbers in an array of shape (d0, d1, ..., dn).

• arange([start,] stop[, step,]) returns evenly spaced values within a given interval.

Array indexing

• A[y,x] returns (y,x) element of the array

• A[:,x] returns all elements of the y-dimension at x

• A[y1:y2,x]

• A[:,:] returns a copy of two dimensional array A

• A[:,:,0] returns the first sub-image of a 3-dimensional array

Example and exercise

Write two programs, a test and a plotting program. The test program shall create a simulated dataset and the plotting routine shall read and display the data on the screen.

We start with a simple linear relation of two variables x and y. The measurement y shall be influenced by noise with a Gaussian distribution.

Generate test data

We first would like to generate x including 11 numbers in the intervall [-5,5]

x=linspace(-5,5,11)

The variable y depends on x and includes the random noise

y=x+randn(x.size)

We write both variables in a file

save('data.txt',(x,y))

Now we combine this in a script. First we open an editor from the Unix shell

nedit generate_test_data.py &

Create a new file, copy and paste the sourcode, and save (Ctrl-S).

   1 from pylab import *
   2 
   3 x=linspace(-5,5,11)
   4 y=x+randn(x.size)
   5 savetxt('data.txt',(x,y))
   6 print('Finished!')

Now you can run the script from IPython

In [7]: run generate_test_data.py
Finished!

Have a look at the variables (objects):

In [3]: whos
Variable   Type       Data/Info
-------------------------------
x          ndarray    11: 11 elems, type `float64`, 88 bytes
y          ndarray    11: 11 elems, type `float64`, 88 bytes

In [5]: x
Out[5]: array([-5., -4., -3., -2., -1.,  0.,  1.,  2.,  3.,  4.,  5.])

In [6]: y
Out[6]: 
array([-5.0387728 , -4.28591892, -3.30766546, -1.65761351, -0.19178245,
       -1.52643416,  0.52628016,  0.74130426,  2.99303495,  4.91713939,
        5.13835735])

Plot test data

The following code can be used as a template

   1 from pylab import *
   2 
   3 x,y=loadtxt('data.txt')
   4 
   5 figure() # Opens a new window for plotting
   6 plot(x,y,'ko') # Plots the data
   7 show() # Displays the result on the screen
   8 savefig('plot.pdf') # Saves the plot on disc

Exercise

• Add a line of identity (y=x) (solid line style)
• Add a linear regression line with dashed line style
• Add a legend

Hints:

• Look up help(plot) for style options

• slope,offset=polyfit(x,y,1) can be used for the linear regression

• Use plot labels and legend(), e.g. plot(x,y,'ko',label='Simulated data')

Python basic data types

One step back: here we look at the built-in data types which are available without loading any modules.

Built-in data types and operations

There are four distinct numeric types:

• plain integers int

• long integers long

• floating point numbers float

• complex numbers complex

In addition, Booleans are a subtype of plain integers

Boolean

Boolean (logical) types

• True
• False

Operations

x + y           sum of x and y
x - y           difference of x and y   
x * y           product of x and y      
x / y           quotient of x and y
x // y          (floored) quotient of x and y
x % y           remainder of x / y
-x              x negated       
+x              x unchanged     
abs(x)          absolute value or magnitude of x        
int(x)          x converted to integer
long(x)         x converted to long integer 
float(x)        x converted to floating point   
complex(re,im)  a complex number with real part re, 
                imaginary part im. im defaults to zero
c.conjugate()   conjugate of the complex number c
x ** y          x to the power y

Boolean Operations: and, or, not

x or y          if x is false, then y, else x 
x and y         if x is false, then x, else y 
not x           if x is false, then True, else False    

Comparisons

<               strictly less than      
<=              less than or equal      
>               strictly greater than   
>=              greater than or equal   
==              equal   
!=              not equal       

is              object identity         

Sequence types

• String str

• List list

• Tuple tuple

Operations

x in s  True if an item of s is equal to x, else False  
x not in s      False if an item of s is equal to x, 
        else True       
s + t   the concatenation of s and t    
s[i]    i'th item of s
s[i:j]  slice of s from i to j  
s[i:j:k] slice of s from i to j with step k
len(s)  length of s     
min(s)  smallest item of s      
max(s)  largest item of s       

Variables, assignment

The assignment statement = creates new variables and gives them values

a=2
l=9**1000L
S='Hello'
f=1e5

Lists and tuples

• Lists [] are mutable sequences of values

• Tuples () can not be changed (immutable)

In [5]: a=[1,2,3]
In [6]: b=(1,2,3)

In [7]: a[0]
Out[7]: 1

In [8]: b[0]
Out[8]: 1

In [9]: a[0]=4

In [10]: a
Out[10]: [4, 2, 3]

In [11]: b[0]=4
---------------------------------------------------------------------------
exceptions.TypeError 

Indexing and slicing

• Indices [] select elements of variables

• Indices start from zero

• x[a:b] selects a slice of elements from variable x

In [1]: a=range(10)

In [2]: a
Out[2]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [3]: a[0:4]
Out[3]: [0, 1, 2, 3]

In [4]: a[4:]
Out[4]: [4, 5, 6, 7, 8, 9]

In [5]: a[:4]
Out[5]: [0, 1, 2, 3]

In [6]: a[:-1]
Out[6]: [0, 1, 2, 3, 4, 5, 6, 7, 8]

In [7]: s='Hello world'
In [8]: s[:5]
Out[8]: 'Hello'

In [9]: a=tuple(range(10))
In [10]: a[0:4]
Out[10]: (0, 1, 2, 3)

List built-in functions (methods)

• L.append(object) -- append object to end

• L.count(value) -> integer -- return number of occurrences of value

• L.extend(iterable) -- extend list by appending elements from the iterable

• L.index(value, [start, [stop]]) -> integer -- return first index of value

• L.insert(index, object) -- insert object before index

• L.pop([index]) -> item -- remove and return item at index (default last)

• L.reverse() -- reverse *IN PLACE*

• L.sort() -- sort *IN PLACE*

In [72]: L=['C','A','B']
In [73]: L.sort()

In [74]: L
Out[74]: ['A', 'B', 'C']

In [75]: L.pop()
Out[75]: 'C'

In [76]: L
Out[76]: ['A', 'B']

In [77]: L.insert(1,'C')

In [78]: L
Out[78]: ['A', 'C', 'B']

Nested data structures

In [84]: a=range(10)
In [85]: b=range(0,10,2)
In [86]: b
Out[86]: [0, 2, 4, 6, 8]

In [87]: c=[a,b]

In [88]: c
Out[88]: [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 2, 4, 6, 8]]

In [91]: c[0]
Out[91]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [92]: c[1]
Out[92]: [0, 2, 4, 6, 8]

In [93]: c[0][0:4]
Out[93]: [0, 1, 2, 3]

In [94]: c[1][0:4]
Out[94]: [0, 2, 4, 6]

Dictionary

• A dictionary is like a list, but more general.
• In a list, the indices have to be integers; in a dictionary they can be (almost) any type.

• d={}  creates empty dictionary

•  d[key]=value  add a key, value pair to dictionary

•  d.keys()  returns list of keys

In [95]: D={}
In [96]: D['a']=range(10)
In [97]: D['b']=range(0,10,2)

In [98]: D
Out[98]: {'a': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 'b': [0, 2, 4, 6, 8]}

In [99]: D['a']
Out[99]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

In [100]: D.keys()
Out[100]: ['a', 'b']

Loops

• Looping over a list is done for the  for  statement

•  enumerate()  returns an enumerated object

In [106]: a=['Hund','Katze','Maus']
In [108]: for i in a:
   .....:     print i
   .....:
Hund
Katze
Maus

In [111]: for i,j in enumerate(a):
   .....:     print i,j
   .....:
0 Hund
1 Katze
2 Maus

Control statements

• Conditions can be checked with if-else statement

•  if  condition: indented block

•  else: 

In [116]: if a==1:
   .....:     print 'a=1'
   .....: else:
   .....:     print 'a != 1'
   .....:
a=1

Documentation and tutorials

• Python at KlimaCampus

• Python tutorial

• Matplotlib/pylab for interactive plotting

• SymPy for symbolic mathematics