Skip to main content
Advanced Research Computing
View of Durham Castle, Durham Cathedral and Framwellgate Bridge across the river

File Storage

Internal Storage

Users have three locations on Hamilton where they can store their files:

  • Home directory (/home/<username>)

By default, new users are provided with 10GB of storage space in their home directory. Home directories are backed up and backups are kept for 30 days.

  • Data directory (/nobackup/<username>)

New users are also provided with 300GB of storage space in their data directory. Please contact us if your research requires more disk space.
Unlike the home directories, data directories are not backed up. Please ensure you save a copy of anything you cannot easily recreate to another service, for example the PRS and SRS services provided by CIS.

Your current usage of your home and data directories can be seen by running the quota command on hamilton.

  • Temporary directory ($TMPDIR)

In addition to the two areas above, each compute node has 400GB disk space available for temporary files. The disk space is accessible only from the node it is attached to. Space here can be reserved as part of a scheduled job and any files stored in $TMPDIR will be removed automatically at the end of that job.

External Storage

In some cases data can be transferred between Hamilton and external storage by dragging and dropping, e.g. to a Windows PC via MobaXterm or WinSCP .  Some other methods for copying data are described below:

Linux and Apple Mac OS X

CIS Research storage

Microsoft OneDrive

The safe storage and management of your data is your responsibility.  The University provides further advice and support for research data management.

Linux and Apple Mac OS X

In addition to the scp command described on the login page, files can be transferred between Hamilton and other Linux/Mac machines using the rsync command:

rsync -av <directory> <username>@hamilton8.dur.ac.uk:

Enter the command (replacing <username> with your University user name, and <directory> with the directory you wish to transfer to hamilton). Unlike scp, which transfers the file(s) specified, this command only transfers files that are different - so can speed up the process of synchronising a directory if there have been a small number of changes.

    Detailed information on how to use this command can be found by entering the command man rsync

Other graphical interfaces include:

  • Apple Mac - Cyberduck

The Cyberduck application can be used to provide a drag and drop interface between a Mac desktop and Hamilton.

  • Linux - the GNOME Files application

The GNOME desktop environment provides a file browser which is able to access remote storage and provide a drag and drop interface between a Linux desktop and Hamilton.

To use, start the Files application and click on + Other Locations on the bottom left. A small Connect to Server section should appear on the bottom right. In the Enter server address... section, type sftp://<username>@hamilton8.dur.ac.uk (replacing <username> with your University user name). Enter your password when prompted.

    Once connected, you are able to bookmark the connection so you can more easily use it later. Right click on the newly created <username> on @hamilton8.dur.ac.uk section on the left and select Add Bookmark.

Figure 1.  Other locations

Gnome files application showing locations

Figure 2.  Bookmark

Gnome files application showing bookmark

CIS Research Storage

The University provides storage for researchers - please see the CIS pages for more details:

  • Personal Research Storage (PRS)
    Storage for research-active staff
  • Shared Research Storage (SRS)
    Storage for research projects

Access has not been integrated into Hamilton yet, but this storage is accessible via mira.dur.ac.uk:

  1. Login to the Storage Manager website, taking note of the "Linux Desktop Service" path to the storage you wish to access (referred to below as <prs_path>)
  2. Login to Hamilton and take a note of the path (referred to below as <hamilton_path>) to the Hamilton data you wish to transfer to PRS
  3. Login to Mira
  4. On Mira, transfer files from Hamilton to PRS with: scp hamilton8:<hamilton_path> <prs_path>

Microsoft OneDrive

Each member of staff has access to some Microsoft OneDrive storage, normally available through a web interface, or the OneDrive folder on a CIS management PC. Files can be transfered to/from this using the rclone utility on Hamilton. To use:

Execute module load rclone to make the rclone command available. man rclone  for detailed documentation.
Execute rclone config and:

as you are about to store credentials in rclone, secure its configuration by selecting

    1. s) Set configuration password
    2. a) Add Password
    3. Enter a password to secure your credentials with
    4. q) Quit to main menu

authorise rclone to use your Microsoft account with:

    1. n) New remote
    2. name> (something memorable, like "durham")
    3. Storage> onedrive
    4. client_id> (leave blank)
    5. client_secret> (leave blank)
    6. Edit advanced config? y/n> n
    7. Use auto config? y/n> y
    8. A web browser will start, showing the Microsoft login page: login using your University credentials
    9. Your choice> onedrive
    10. Chose drive to use:> 0
    11. Is that okay? y/n> y
    12. y) Yes this is OK
    13. q) Quit config

You can now use rclone to access your University Onedrive:

  • List files: rclone ls durham:
  • Copy files to onedrive: rclone <file> durham:
  • Copy files from onedrive to a local directory: rclone durham:<file> <local_dir>

rclone is a versatile tool that can be used to transfer files between many different services.