Using Globus

This document describes how to use Globus to transfer data between your computer (or any other computer on the Internet that is acting as a Globus "endpoint" or "collection" and to which you have access) and the Berkeley Savio and Vector clusters. An "endpoint" is simply a location (e.g., laptop, desktop, server, cluster, storage system) that you want to transfer files to or from. A Globus "collection" is a discoverable access point that allows data to be transferred through GridFTP or HTTPS, and contains policies that govern sharing data with other users.

Some locations that you can use to transfer data to/from include your own personal computer, bDrive/Google Drive, Wasabi, Cloudian, AWS, the Secure Research Data and Compute (SRDC) platform at UCB, and some campus departmental facilities such as the Econometrics Laboratory and the Statistical Computing Facility.

Globus is ideal for transferring large files and/or large numbers of files at high speed. It allows you to start an unattended transfer and have confidence that it will be completed reliably; even sending you an email notification when it's done.

For information on how to use it, see Globus's Step-by-Step Getting Started Guide.

Using Globus with Savio

This video walks through the process of using Globus with Savio:

A quick summary for users:

1. Visit the Globus website.
2. Click the Log In button at upper right and log in using your institutional login (e.g. UC Berkeley or LBNL), or Globus ID. -If this your first time, you can create a free Globus ID here.
3. You should land on the File Manager page.
   • Turn on two-panel mode by toggling the switch near the top right.
4. Use the File Manager page to transfer files:
   • Click in the Collection field at left to search for or select an endpoint.
   • Search for ucb#brc in the Collection field. (This is the Globus endpoint for the BRC supercluster, which connects via the supercluster's Data Transfer Node, dtn.brc.berkeley.edu.)
   • Select the ucb#brc entry when it is displayed below. Doing so will return you to the original Transfer Files page.
   • If prompted to authenticate, click the "Continue" button and follow the on-screen instructions. (In brief, you'll be taken to an LBNL page where you can enter your BRC cluster username, along with your token PIN followed by a one-time password that you will generate via the Google Authenticator application. See Logging In for details on generating that password.)
   • Enter the path for the directory on the BRC cluster to/from which you'd like to copy files, into the Path field at left. (E.g. /~/ or /global/home/users/myusername for your home directory, where myusername is your BRC cluster username. For more details on storage locations, see the relevant User Guide page for the cluster on which you are working.)
   • Click in the Collection field at right and repeat the process for your second endpoint. This second endpoint would often be your own computer, which you would need to set up as an endpoint as described below. (Globus also provides test endpoints, go#ep1 and go#ep2, each of which provides sample files that you can practice transferring to the cluster.)
   • In the list of files in either of the two endpoints you have chosen (i.e., in either the left or right panel), select one or more files or folders.
   • Click the appropriate "Start" button at the bottom of each endpoint to transfer these files to the other endpoint.

Transferring files between Savio and your personal computer

To transfer files between your computer and the BRC supercluster, you'll need to set up Globus Connect Personal. Here's a quick overview of how to install and use it:

1. Log into the Globus website. (If you're not already logged in, that is.)
2. Visit the Collections page.
3. Click on "Get Globus Connect Personal".
4. Follow the onscreen instructions on how to set up and install this software. You can also find detailed, official instructions for installing Globus Connect Personal for Mac OS X, Microsoft Windows, and Linux via links on this Globus Support "How To" page.
5. Once you've installed Globus Connect Personal, and that software is actively running on your computer, go back to the "File Manager" page and select your new endpoint as above.

Keep in mind that, by default, Globus Connect Personal will only have access to certain folders on your personal or local computer (e.g., your laptop or desktop system), such as your home directory on your Mac, for example. Any other area, such as a network drive, must be enabled within the Globus Connect Personal application. To configure your Globus Connect Personal instance to access other directories on your personal or local computer, see the instructions for

• Mac
• Windows
• Linux

For example, on a Mac laptop, click on the Globus Connect Personal icon, select "Preferences", then select the "Access" menu to adjust what directories Globus can have access to.

Transferring files between Savio and bDrive/Google Drive

To transfer files between your computer and the bDrive/Google Drive, follow these steps:

1. In the File Manager, click in the Collection field and search for "UCB Google Drive Collections".
2. When prompted to authenticate, from the Federation/Identity page, look for "University of California, Berkeley", select it, and use your CalNet credentials to log in.
3. Now just navigate to your bDrive/Google Drive directory/folder of interest and transfer files in similar fashion as described in the instructions above regarding transferring between Savio and a personal computer.

Transferring files between Savio and other locations

Instructions for transferring to other locations, such as Wasabi and AWS, are planned. Please contact us if you'd like help in the meantime. Unfortunately at this time it is not possible to use Globus with your Berkeley Box account.

Note that if you're only using Globus to move files between the Savio cluster and other external computing systems that use Globus (such as another University cluster or national lab cluster), then there is no need to install and use Globus Connect Personal on your personal or local computer.