Overview

Savio provides a variety of software installed by the system administrators and BRC consultants, ranging from compilers and interpreters to statistical analysis and visualization software, bioinformatics and computational biology software, and much more. To access much of the software available on the Savio cluster, you'll use Environment Module commands.

These commands allow you to display a list of many of the software packages provided on the cluster, as well as to conveniently load and unload different packages and their associated runtime environments, as you need them.

As a quick overview, Environment Modules are used to manage users’ runtime environments dynamically on the Savio cluster. This is accomplished by loading and unloading modulefiles which contain the application specific information for setting a user’s environment, primarily the shell environment variables, such as PATH, LD_LIBRARY_PATH, etc. Modules are useful in managing different applications, as well as different versions of the same application, in a cluster environment.

Finally, in addition to the software provided on the Savio cluster, you're also welcome to install your own software.

Accessing Software Using Environment Modules

Below are some of the Environment Module commands that you'll be using most frequently. All of these commands begin with module and are followed by a subcommand. (In this list of commands, a vertical bar (“|”) means “or”, e.g., module add and module load are equivalent. And you'll need to substitute actual modulefile names for modulefile, modulefile1, and modulefile2 in the examples below.)

  • module avail - List all available modulefiles in the current MODULEPATH. (This is the command to use when you want to see the list of software that you can use on Savio via Environment Module commands.)
  • module list - List loaded modules. (This shows you what software you currently have available in your environment. Please note that, by default, no modules are loaded.)
  • module add|load modulefile ... - Load modulefile(s) into the shell environment. (This allows you to add more software packages to your environment.)
  • module rm|unload modulefile ... - Remove modulefile(s) from the shell environment.
  • module swap|switch [modulefile1] modulefile2 - Switch loaded modulefile1 with modulefile2.
  • module show|display modulefile ... - Display configuration information about the specified modulefile(s).
  • module whatis [modulefile ...] - Display summary information about the specified modulefile(s).
  • module purge - Unload all loaded modulefiles.

For more detailed usage instructions for the module command, please run man module on the cluster.

Below are representative examples of how to use these commands. Depending on which system you have access to and when you are reading this instruction, what you see here could be different from the actual output from the system that you work on. On systems like Savio, where a hierarchical structure is used, some modulefiles will only be available after their root modulefile is loaded. (For instance, libraries for C compilers, and the like, will only become available after their respective parent modules have been loaded.)

It can be helpful to try out each of the following examples in sequence, to more fully understand how environment modules work. Commands you'll enter are shown in bold, followed by samples of output you might see:

[casey@n0000 ~]$ module avail

---- /global/software/sl-7.x86_64/modfiles/langs ----
clang/3.9.1  cuda/8.0  gcc/4.8.5  gcc/6.3.0  java/1.8.0_121  python/2.7  python/3.6  r/3.4.2

---- /global/software/sl-7.x86_64/modfiles/tools ----
arpack-ng/3.4.0  gflags/2.2.0  gv/3.7.4  lmdb/0.9.19  nano/2.7.4   qt/5.4.2  texlive/2016

---- /global/software/sl-7.x86_64/modfiles/apps ----
bio/blast/2.6.0  math/octave/current  ml/mxnet/0.9.3-py35  ml/theano/current-py36 

...

[casey@n0000 ~]$ module list
No Modulefiles Currently Loaded.

[casey@n0000 ~]$ module load intel

[casey@n0000 ~]$ module list

Currently Loaded Modulefiles:
  1) intel/2016.4.072

[casey@n0000 ~]$ module load openmpi mkl

[casey@n0000 ~]$ module list
Currently Loaded Modulefiles:
  1) intel/2016.4.072   3) mkl/2016.4.072
  2) openmpi/2.0.2-intel

[casey@n0000 ~]$ module unload openmpi
[casey@n0000 ~]$ module list
Currently Loaded Modulefiles:
  1) intel/2016.4.072   2) mkl/2016.4.072

[casey@n0000 ~]$ module switch mkl lapack
[casey@n0000 ~]$ module list
Currently Loaded Modulefiles:
  1) intel/2016.4.072   2) lapack/3.8.0-intel

[casey@n0000 ~]$ module show mkl

-------------------------------------------------------------------
/global/software/sl-7.x86_64/modfiles/intel/2016.4.072/mkl/2016.4.072:

module-whatis    This module sets up MKL 2016.4.072 in your environment.
setenv           MKL_DIR /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl
setenv           MKLROOT /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl
prepend-path     CPATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include
prepend-path     CPATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include/fftw
prepend-path     FPATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include
prepend-path     FPATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include/fftw
prepend-path     INCLUDE /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include
prepend-path     INCLUDE /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/include/fftw
prepend-path     LIBRARY_PATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin
prepend-path     LD_LIBRARY_PATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin
prepend-path     MIC_LIBRARY_PATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin_mic
prepend-path     MIC_LD_LIBRARY_PATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin_mic
prepend-path     SINK_LD_LIBRARY_PATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin_mic
prepend-path     NLSPATH /global/software/sl-7.x86_64/modules/langs/intel/2016.4.072/mkl/lib/intel64_lin/locale/en_US
-------------------------------------------------------------------

[casey@n0000 ~]$ module whatis mkl
mkl                  : This module sets up MKL 2016.4.072 in your environment.

[casey@n0000 ~]$ module purge  
[casey@n0000 ~]$ module list
No Modulefiles Currently Loaded.

[casey@n0000 ~]$ module avail

---- /global/software/sl-7.x86_64/modfiles/langs ----
clang/3.9.1  cuda/8.0  gcc/4.8.5  gcc/6.3.0  java/1.8.0_121  python/2.7  python/3.6  r/3.4.2

---- /global/software/sl-7.x86_64/modfiles/tools ----
arpack-ng/3.4.0  gflags/2.2.0  gv/3.7.4  lmdb/0.9.19  nano/2.7.4   qt/5.4.2  texlive/2016

---- /global/software/sl-7.x86_64/modfiles/apps ----
bio/blast/2.6.0  math/octave/current  ml/mxnet/0.9.3-py35  ml/theano/current-py36 

...

[casey@n0000 ~]$ module load gcc

[casey@n0000 ~]$ module avail

---- /global/software/sl-7.x86_64/modfiles/langs ----
clang/3.9.1  cuda/8.0  gcc/4.8.5  gcc/6.3.0  java/1.8.0_121  python/2.7  python/3.6  r/3.4.2

---- /global/software/sl-7.x86_64/modfiles/tools ----
arpack-ng/3.4.0  gflags/2.2.0  gv/3.7.4  lmdb/0.9.19  nano/2.7.4   qt/5.4.2  texlive/2016

---- /global/software/sl-7.x86_64/modfiles/gcc/6.3.0 ----
antlr/2.7.7-gcc  fftw/2.1.5-gcc  hdf5/1.8.18-gcc-p  ncl/6.3.0-gcc  ncview/2.1.7-gcc  openmpi/2.0.2-gcc

...
NOTE: Some modulefiles will become available only after the “intel” or “gcc” modulefile is loaded, for example openmpi, hdf5, and netcdf. To view an extensive list of all modules including those only visible after loading other prerequisite modules, enter the following command:

find /global/software/sl-7.x86_64/modfiles -type d -exec ls -d {} \;

Software Provided on Savio

Research IT provides and maintains a set of system level software modules. The purpose is to provide an ecosystem that most users can rely on to accomplish their research and studies. The range of applications and libraries that Research IT supports highly depend on the use case and the frequency of how often a support request is received.

For a detailed and up-to-date list of software provided on the cluster, run the module avail command, as described in the usage examples above.

(Note: if you're interested in whether a particular C library is provided on the cluster, make sure that you first load the parent software itself – namely the Intel or GCC compilers – before checking the list of provided software, as that list is dynamically adjusted based on your current environment.)

Currently the following categories of applications and libraries are supported, with some key examples of each shown below:

  • Development Tools
  • Data processing, Machine Learning, and Visualization Tools
  • Typesetting and Publishing Tools
  • Miscellaneous Tools (examples not yet listed below)
     
Category Application/Library Name
Development Tools
Editor/IDE Emacs, Vim, Nano, cmake, cscope, ctags
SCM Git, Mercurial
Debugger/Profiler/Tracer GDB, gprof, Valgrind, TAU, Allinea DDT
Languages/Platforms GCC, Intel, Perl, Python, Java, Boost, CUDA, UPC, Open MPI, TBB, MPI4Py, Python / IPython, R, MATLAB, Octave, Julia
Math Libraries MKL, ATLAS, FFTW, FFTW3, GSL, LAPACK, ScaLAPACK, NumPy, SciPy, Eigen
IO Libraries HDF5, NetCDF, NCO, NCL
Data processing, Visualization, and Machine Learning Tools
Data Processing/Visualization Gnuplot, Grace, Graphviz, ImageMagick, MATLAB, Octave, ParaView, Python / IPython, R, VisIt, VMD, yt, Matplotlib
Machine Learning Tensorflow, Caffe, H2O, MXNet, Theano, Torch, Scikit-learn
Bioinformatics BLAST, Bowtie, Picard, SAMtools, VCFtools
Typesetting and Publishing Tools
Typesetting TeX Live, Ghostscript, Doxygen

 

Chaining Software Modules

Environment Modules also allow a user to optionally integrate their own application environment together with the system-provided application environment, by allowing different categories of modulefiles to be chained together. This provides a common interface for simplicity, while still maintaining diversity and flexibility:

  • The first category of the modulefiles are provided and maintained by Research IT, which include the commonly used applications and libraries, such as compilers, math libraries, I/O libraries, data processing and visualization tools, etc. We use a hierarchical structure to maintain the cleanness without losing the flexibility of it.
     
  • The second category of the modulefiles are automatically chained for the group of users who belong to the same group on the cluster, if the modulefiles exist in the designated directory. This allows the same group of users to share some of the common applications that they use for collaboration and saves spaces. Normally the user group maintains these modulefiles. But Research IT can also provide assistance under support agreement and on a per request basis.
     
  • The third category of the modulefiles can also be chained on demand by a user if the user chooses to use Environment Modules to manage user specific applications as well. To do that, user needs to append the location of the modulefiles to the environment variable MODULEPATH. This can be done in one of the following ways:1). For bash users, please add the following to ~/.bashrc:
    export MODULEPATH=$MODULEPATH:/location/to/my/modulefiles

    2). For csh/tcsh users, please add the following to ~/.cshrc:

    setenv MODULEPATH "$MODULEPATH":/location/to/my/modulefiles
Tags: hpc All Tags