Cookies

We use cookies to ensure that we give you the best experience on our website. You can change your cookie settings at any time. Otherwise, we'll assume you're OK to continue.

Durham University

Advanced Research Computing

Python

Hamilton supports using the Python programming language. By default, a fairly old version of Python is available via the python command, which is provided by the underlying operating system.

In addition to this, a number of other versions of Python are made available through the module command. module avail python will show what is availble.

The current version of Python we recommend on Hamilton is python/3.6.8. This module also contains a number of additional Python packages suitable for numerical work, such as numpy and scipy.

To supplement the python module, each user is able to install additional Python packages accessible only to them. As people will need different packages and versions of those packages, this is a useful way of giving users the flexibility to install software without waiting for the Hamilton administrators to install it for you.

To see what packages are installed:

$ module load python/3.6.8
$ pip list

To install additional packages:

$ module load python/3.6.8
$ pip install --user <package_name>

This will download, build and install the package in your home directory under the directory .local Your Python programs will be able to automatically find them using the normal import statement.

Example python program, in file my_python_program.py:

#!/usr/bin/env python3

print("hello, world!")

Example job script my_python_job.sh to run a python program using a single CPU core:

#!/bin/bash

# Request resources:
#SBATCH -n 1          # 1 CPU core
#SBATCH --mem=1G      # 1 GB RAM
#SBATCH --time=6:0:0  # 6 hours (hours:minutes:seconds)

# Run on the queue for serial ("sequential") work
# (job will share node with other jobs)
#SBATCH -p seq7.q

# Make python available:
module load python/3.6.8

# Commands to be run:

./my_python_program.py

Submit it to the queue with the command: sbatch my_python_job.sh

If you need to make your Python code run faster, there are a number of things you can try. Roughly, in order of importance (most important first):

  1. Use a profiler such as the cProfile and profile packages, or benchmark using timeit to understand: where your Python program is spending most of its time; where you need to concentrate your effort; and the impact of any changes made to speed up the code.
  2. Avoid writing your own matrix or vector manipulation functions, use the numpy package.
  3. Avoid native Python datatypes such as lists and dictionaries for numerical work, instead use equivalent functionality provided by the numpy and scipy packages.
  4. Try using the Python package numba to automatically optimise specific functions using a compiler.
  5. Try writing the most numerically intensive part of your program in another language, such as Fortran or C, and call it from Python using the Cython package. Consider parallelising the Fortran or C code using OpenMP.
  6. Try using packages such as multiprocessing or mpi4py to make use of more than one CPU core.