How to set up a virtual machine on the Nectar cloud

What is it

• Nectar is cloud computers (= virtual machines, = VMs, = instances) for Australian researchers.
• You can set up a virtual computer, install software, and run analyses.
• Info: https://cloud.nectar.org.au/faq/
• All Nectar training: http://training.nectar.org.au/
• Or just start here: http://training.nectar.org.au/package05/sections/connecting.html
• More info: https://docs.cloud.unimelb.edu.au/training/first_instance/ (some is unimelb specific)

Start

• Go to http://cloud.nectar.org.au/start-now/
• Get a free project trial. This is free, 3-month access to a small amount of cores/memory.
• Go to your dashboard: https://dashboard.rc.nectar.org.au/project/
• How much do I have? Dashboard -> Project -> Compute -> Overview
• What instances do I have running? Dashboard -> Project -> Compute -> Instances

Set-up

Set up some stuff in: Dashboard -> Project -> Compute

Key pairs

• create key pair - called it nectarkeys
• click on it. it should download the private key with a .pem file type
• put it in your .ssh folder
• change permissions for this file: chmod u+rw,og-rwx nectarkeys.pem

Instances

• launch instance
• name it
• then click on the source tab in the left hand panel
• find e.g. NeCTAR Ubuntu 16.04 LTS (Xenial) amd64, and tick
• flavour tab: eg m3.xsmall
• note: Small (1 core), Medium (2 cores), Large (4 cores), Extra-large or XL (8 cores), XXL (16 cores)
• security groups tab: click on both default and ssh
• key pair tab: make sure my “nectarkeys” is showing
• bottom right: launch instance

Your instance (= VM)

• should now be running; might take a few minutes to show up.
• find IP address e.g. 115.333.444.255.
• This identifies this particular instance.
• Look at the Flavour column and click on the flavour (e.g. m3.xsmall).
• This shows how many cpus (e.g. 1) and how much RAM (e.g. 2GB)

Connect

We will login (= connect) to this instance (= VM).

Work out your username

• your username depends on the particular instance you set up.
• It is often one of these: ubuntu, debian, fedora, root, ec2-user.
• see list of images and usernames in here https://support.ehelp.edu.au/support/solutions/articles/6000106269-image-catalog

Connect

• open terminal (or equivalent)
• type in:
• ssh -i .ssh/nectarkeys.pem ubuntu@15.333.444.255 // replace with your info
• -i // flag for key file
• .ssh/nectarkeys.pem // key file location (.ssh/) and name (nectarkeys.pem)
• ubuntu // username for this VM
• 15.333.444.255 //IP address for this VM

Install Bioconda so it’s easy to install other tools

Note: This is about getting tools (software) onto your VM. There are other ways to do this!

Get Miniconda

• While still logged in to your VM:
• wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

Install Miniconda

• bash Miniconda3-latest-Linux-x86_64.sh
• Need to disconnect (type in exit) then re-connect (with ssh and your info)
• Type in these lines one at a time:
• conda config --add channels defaults
• conda config --add channels bioconda
• conda config --add channels conda-forge

Install tools

e.g. the spades assembler

• conda install spades

Get some sample data

e.g. some Illumina fastQ sequencing reads

• wget https://zenodo.org/record/1009308/files/R1.fq
• wget https://zenodo.org/record/1009308/files/R2.fq

Run an analysis

e.g. spades assembly. Note that we will set number of threads (1) and max amount of RAM (2 GB), as per the VM’s limits.

• spades.py -1 R1.fq -2 R2.fq -o spades-out --threads 1 --memory 2

• output will be in the spades-out folder

How to close and keep it running

• If you shut your terminal or lose your internet connection, the run will stop.
• To be able to shut and have it keep running, you can add some extra commands before and after the usual commands:
• nohup [usual commands here] 2>&1 &
• nohup at the start (means no hangup)
• 2>&1 after the commands (means error messages will go to the same place as output info)
• & at the end of the line (puts the job to run in the background)
• You can see the job running if you type in top (q to exit)
• While this runs in the background, you can still work in the terminal window, or exit.
• When you re-login, you’ll see any output files (if it’s finished), plus a file called nohup.out which has the info that would normally be printed to the screen during the run.