Data Transfer Overview
This is an overview of how to transfer data to or from the Berkeley Research Computing (BRC) supercluster, consisting of the Savio and Vector high-performance computing clusters, at the University of California, Berkeley.
When transferring data using file transfer software, you should connect only to the supercluster's Data Transfer Node,
dtn.brc.berkeley.edu. (Note: if you're using Globus Connect, you'll instead connect to the Globus endpoint
After connecting to the Data Transfer Node, you can transfer files directly into (and/or copy files directly from) your Home directory, Group directory (if applicable), and Scratch directory.
For information on making your files accessible to other users (in particular members of your group), see these instructions.
Medium- to large-sized data transfers
When transferring a large number of files and/or large files, we recommend you use:
- Globus Connect (formerly Globus Online): This method allows you to make unattended transfers which are fast and reliable. For basic instructions, see Using Globus Connect.
Small-sized data transfers
When transferring a modest number of smaller-sized files, you can also use:
- SFTP: For basic instructions, see Using SFTP via FileZilla.
- SCP: For basic instructions, see Using SCP.
Transfers to/from repositories under version control
When your code and/or data are stored in repositories under version control, client software is available for accessing them via:
- Subversion (SVN)
See Accessing Software for information on finding and loading this software via the BRC supercluster's Environment Modules.
Transfers to/from specific systems
For bDrive (Google Drive) and Box, we recommend using rclone to transfer data to and from Savio.
- See here for instructions for using Box and bDrive with Savio.
- See here for additional information about bDrive (Google Drive) provided by our Research Data Management program.
Additional tutorials for transferring files to/from Amazon Web Services (AWS) S3 and other popular data storage systems are in planning or development. If you have any interest in working on or testing one of these, or have suggestions for other data transfer tutorials, please contact us via our Getting Help email address!