Condo Storage Service¶
Summary
Berkeley Research Computing (BRC) offers a Condo Storage service for researchers who have Savio projects and need additional persistent storage to hold their data sets while using the Savio cluster. Researchers pay a one-time upfront cost for this storage, which is available in 112 TB increments at a cost of $5,750 each (plus sales tax) (price subject to change due to supply chain issues). The storage is maintained by BRC for a 5-year period thereafter at no additional cost to the researcher. Please feel free to contact us with your questions about this offering.
Overview¶
By default, each user on Savio is entitled to a 30 GB home directory that receives regular backups; in addition, each Condo-using research group receives 200 GB of project space to hold software and related materials shared among the group's users. All users also have access to the large Savio high performance scratch filesystem for working with non-persistent data.
This Condo Storage service supplements the above offerings. It is intended to provide a cost-effective, persistent storage solution for users or research groups that need to import and store large data sets over a long period of time to support their use of Savio. Condo storage is NOT backed up.
Program Details¶
Similar in concept to the Condo Computing program, BRC provides the storage infrastructure consisting of a DataDirect Networks 18K storage server and covers the cost of supporting the system. Researchers can purchase storage shelves and have them hosted in the BRC storage infrastructure. This allows researchers to 1) pay a one-time upfront cost for the storage without having to incur ongoing support costs; and 2) pay only for the incremental cost of the storage shelf and disk drives because the cost of the storage servers and disk controllers are covered by BRC.
Storage shelves are purchased and maintained on a 5-year life cycle. At the end of 5 yrs, Condo Storage contributors who wish to continue participating in the program will be offered one of the following options, and possibly both: (1) purchase new storage; or (2) cover the cost of annual vendor hardware maintenance for their existing storage shelves. Which of these options are available will depend on when the contributor’s storage was purchased, and when the BRC Program determines that server hardware for the storage infrastructure must be upgraded.
Access to the storage is based on membership in a UNIX group (generally the same group as set up for an associated condo cluster) and with a single quota for the group. In some cases it may be possible to set up multiple UNIX groups upon request, but it is not possible for the research group to manage these groups directly. It is also not possible to set sub-quotas for individual group members or subsets of members. The expectation is that the research group will collectively determine how to share use of the storage across individual members.
Backups¶
Backups are the responsibility of the owner of the storage. Faculty can work with BRC and IST staff to obtain regular backups or to set up regular filesystem snapshots.
Testing¶
Users who wish to test the performance of Condo Storage can run their tests using their scratch directory; as of summer 2021, new Condo storage allocations are set up on the same storage system as users' scratch space.
Purchasing¶
Interested faculty and PIs should contact us at brc-hpc-help@berkeley.edu for further information. BRC will assist with entering a storage shelf purchase requisition in increments of 112 TB, and initiate the procurement. The entire process from start to production is anticipated to be 6 weeks.
Eligibility¶
The Condo Storage service is available to all Savio PIs, although it has generally been intended for UC Berkeley faculty who are Savio Condo Cluster Service contributors. This is because an FCA provides limited service units that may not provide sufficient computational time for work with very large datasets.