Globus online is a simple and reliable way to transfer files to or from your desktop or between HPC systems. It is the preferred transfer option for large files. Even if your connection is altered, the data is not lost and transfer will continue from where it left once the connection is re-established. This is unlike scp or rsync.
Each computer is defined as an 'endpoint', and you connect two endpoints in the browser. Most HPC centers (e.g. CSC, the national supercomputing centers, etc.) have endpoints predefined - and you just login with the username and password that you login to your accounts with (i.e. your Knot username and password).
One endpoint is your local computer (i.e. your Desktop or Laptop) which needs to be setup, the other one, in case of Knot, is ucsb#knot-storage
To get started (the first time)
- First, create a globus account
- Then, install the Globus Connect Personal program on your computer.
- You'll see a screen like this, and you should set a name for your computer (it is only visible to you - but pick something that makes sense to you)
- Click the Generate Setup Key button and copy/paste that into the window that pops up when doing the install.
- Then choose your local operating system and install on your computer. You will need the setup key you have generated.
How to transfer data (after you have it installed on your computer)
Once you've done this, login to the globus website, and pick the endpoints to connect (see picture below). For Knot, just type in UCSB in one of the windows and you'll see UCSB#knot-storage. It will pop up a window that you type in your login information into:
You then have an interface where you click to select files, and can click to transfer entire directories. Globus will take care of making sure the transfer completes, even if it gets interrupted.
Choose the files on the left hand side (your local endpoint), and transfer by clicking on > and vice versa by <.
There is one annoyance in that it will send e-mail notifications of transfers - they can be disabled, but not entirely trivial to do.