#acl AdminGroup:read,write,delete,revert  EditorGroup:read,write,delete,revert All:read
#format wiki
#language en
#pragma section-numbers off

<<TableOfContents(2)>>

= Python modules =

There a several modules which are already included in the standard Python distribution:

 * http://docs.python.org/modindex.html

Other modules have to be installed separately, e.g. Numpy, Scipy, Pylab, IPython. 

Some distributions including precompiled modules are available, e.g.

 * http://www.pythonxy.com/ Python(x,y) is a scientific python distribution with many modules included
 * http://www.enthought.com/products/epd.php

It is important to know where to find the functions that you need. We will go through some useful examples.

== At ZMAW ==
Remember to apply
{{{
module load Python
}}}
to configure the Unix environment for the most recent versions.

= Numpy, Scipy, Pylab and IPython =

== Numpy ==

Numpy is the core library for multidimensional array objects (ndarray) and linear algebra. Most other scientific modules use the numpy array object. Numpy arrays and standard Python sequences have important differences:

 * Numpy arrays have a fixed size at creation, unlike Python lists
 * Numpy arrays facilitate mathematical operations on large numbers of data efficiently

The [[http://docs.scipy.org/doc/numpy/user/|User's guide]] [[http://docs.scipy.org/doc/numpy/numpy-user.pdf|PDF]] provides a good introduction. 

== Scipy ==

The Scipy module is built on Numpy and offers a collection of mathematical algorithms such as

 * Clustering algorithms
 * Fast Fourier Transform routines
 * Integration and ordinary differential equation solvers
 * Interpolation and smoothing splines
 * Linear algebra
 * Maximum entropy methods
 * N-dimensional image processing and signal processing
 * Optimization and root-finding routines
 * Statistical distributions and functions

Furthermore it includes very handy routines for [[http://docs.scipy.org/doc/scipy/reference/tutorial/io.html|data input and output]] of binary and ASCII-tables, Matlab and Netcdf data files

== Pylab ==

[[http://matplotlib.sourceforge.net/|Pylab (aka Matplotlib)]] uses Numpy and Scipy and offers high-level functions that are similar in the name and syntax to those offered by Matlab. Matplotlib is the name of the core library and pylab provides the Matlab similarity. Pylab produces figures of great quality suitable for publications. 

Making plots is easy. Start reading the [[http://matplotlib.sourceforge.net/contents.html|User's guide]]. For a specific problem look at the [[http://matplotlib.sourceforge.net/gallery.html|Gallery]] for a similar plot you would like to have and learn from the source code.

== IPython ==

[[http://ipython.scipy.org/|IPython]] is an environment for interactive and exploratory computing. Useful features are TAB-completion, magic commands, e.g. {{{%run, %whos}}}, input cache and many more convenient functions that are not available in the standard Python shell. 


== Importing the scientific environment ==


The statement
{{{
from pylab import *
}}}
imports the most important functions/objects for numerical computation and plotting. When using the interactive IPyhon shell this import is already done with
{{{
ipython -pylab
}}}



For more complex applications it is useful but not necessary to follow the conventions that the community has adopted:
{{{#!python
import numpy as np
import scipy as sp
import matplotlib as mpl
import matplotlib.pyplot as plt
}}}

= Arrays =

Numpy provides a multidimensional '''array''' data type. An array can hold arbitrary Python objects but usually they are used for N-dimensional numeric data types. 

== Array creation ==

 * {{{empty((d1,d2),dtype)}}} returns uninitialized array of shape d1,d2. 
 * {{{zeros((d1,d2),dtype)}}} returns array of shape d1,d2 filled with zeros
 * {{{ones((d1,d2),dtype)}}} returns array of shape d1,d2 filled with ones
 * {{{array(object,dtype)}}} returns an array from an object, e.g. a list
 * {{{dtype}}} fundamental C data type e.g. uint8, int16, int64, float32, float64

Arrays can also be created from special functions, e.g. random

 * {{{randn((d1,d2,..,dn)}}} returns Gaussian random numbers in an array of shape (d0, d1, ..., dn).


== Array indexing ==

 * {{{A[y,x]}}} returns (y,x) element of the array
 * {{{A[:,x]}}} returns all elements of the y-dimension at x
 * {{{A[y1:y2,x]}}} 
 * {{{A[:,:]}}} returns a copy of two dimensional array A 
 * {{{A[:,:,0]}}} returns the first sub-image of a 3-dimensional array

  
 
= Example and exercise =

Write two programs, a test and a plotting program. The test program shall create a simulated dataset and the plotting routine shall read and display the data on the screen.

We start with a simple linear relation of two variables x and y. The ''measurement'' y shall be influenced by noise with a Gaussian distribution. 


== Generate test data ==

We first would like to generate x including 11 numbers in the intervall [-5,5]
{{{
x=linspace(-5,5,11)
}}}

The variable y depends on x and includes the random noise
{{{
y=x+randn(x.size)
}}}

We write both variables in a file 
{{{
save('data.txt',(x,y))
}}}

Now we combine this in a script. First we open an editor from the IPython shell 
{{{
!nedit generate_test_data.py &
}}}
Create a new file, copy and paste the sourcode, and save (Ctrl-S).

{{{
#!python
from pylab import *

x=linspace(-5,5,11)
y=x+randn(x.size)
save('data.txt',(x,y))
print('Finished!')
}}}

Now you can run the script from IPython 
{{{
In [7]: run generate_test_data.py
Finished!
}}}

What happens if you leave out the first line ({{{from pylab import *}}})?

== Plot test data ==

The following code can be used as a template
{{{
#!python
from pylab import *

x,y=load('data.txt')

figure() # Opens a new window for plotting
plot(x,y,'ko') # Plots the data
show() # Displays the result on the screen
savefig('plot.pdf') # Saves the plot on disc
}}}

=== Exercise ===

 * Add a line of identity (y=x) (solid line style)
 * Add a linear regression line with dashed line style
 * Add a legend

Hints: 

 * Look up help(plot) for style options
 * {{{slope,offset=polyfit(x,y,1)}}} can be used for the linear regression
 * Use plot labels and legend(), e.g. plot(x,y,'ko',label='Simulated data')




= Links and References =

 * http://heim.ifi.uio.no/~hpl/scripting/all-nosplit/index.html 
 * http://www.springer.com/math/cse/book/978-3-540-73915-9 Python Scripting for Computational Science