Using Globus Connect with the BRC Supercluster

This document describes how to use Globus Connect (formerly Globus Online) to transfer data between your computer (or any other computer on the Internet that is acting as a Globus "endpoint" and to which you have access) and the Berkeley Research Computing (BRC) supercluster, consisting of the Savio, Vector, and Cortex high-performance computing clusters at the University of California, Berkeley. An "endpoint" is simply a location (e.g., laptop, desktop, cluster) that you want to transfer files to or from.

Globus Connect is ideal for transferring large files and/or large numbers of files at high speed. It allows you to start an unattended transfer and have confidence that it will be completed reliably; even sending you an email notification when it's done.

For information on how to use it, see Globus's Step-by-Step Getting Started Guide.

This video walks through the process of using Globus with Savio:

A quick summary for users:

  1. Visit the Globus website.
  2. Click the Log In button at upper right and log in using your institutional login (e.g. UC Berkeley or LBNL), or Globus ID.
  3. You should land on the File Manager page.
    • Turn on two-panel mode by toggling the switch near the top right.
  4. Use the File Manager page to transfer files:
    • Click in the Collection field at left to search for or select an endpoint.
    • Search for ucb#brc in the Collection field. (This is the Globus endpoint for the BRC supercluster, which connects via the supercluster's Data Transfer Node,
    • Select the ucb#brc entry when it is displayed below. Doing so will return you to the original Transfer Files page.
    • If prompted to authenticate, click the "Continue" button and follow the on-screen instructions. (In brief, you'll be taken to an LBNL page where you can enter your BRC cluster username, along with your token PIN followed by a one-time password that you will generate via the Google Authenticator application. See Logging In for details on generating that password.)
    • Enter the path for the directory on the BRC cluster to/from which you'd like to copy files, into the Path field at left. (E.g. /~/ or /global/home/users/myusername for your home directory, where myusername is your BRC cluster username. For more details on storage locations, see the relevant User Guide page for the cluster on which you are working.)
    • Click in the Collection field at right and repeat the process for your second endpoint. This second endpoint would often be your own computer, which you would need to set up as an endpoint as described below. (Globus also provides test endpoints, go#ep1 and go#ep2, each of which provides sample files that you can practice transferring to the cluster.)
    • In the list of files in either of the two endpoints you have chosen (i.e., in either the left or right panel), select one or more files or folders.
    • Click the appropriate "Start" button at the bottom of each endpoint to transfer these files to the other endpoint.

To transfer files between your computer and the BRC supercluster, you'll need to set up Globus Connect Personal. Here's a quick overview of how to install and use it:

  1. Log into the Globus website. (If you're not already logged in, that is.)
  2. Visit the Endpoints page.
  3. Click the "Create New Endpoint" link, then select "Globus Connect Personal".
  4. Follow the onscreen instructions on how to set up and install this software. You can also find detailed, official instructions for installing Globus Connect Personal for Mac OS X, Microsoft Windows, and Linux via links on this Globus Support "How To" page.
  5. Once you've installed Globus Connect Personal, and that software is actively running on your computer, go back to the "File Manager" page and select your new endpoint as above.