Software Usage: Environment Modules

Loading and initializing software modules

On the Lichtenberg HPC, various compilers, (scientific) libraries, and other (application) software are installed and readily available, often in different versions. Each of these software packages needs different settings in terms of $PATH, $LD_LIBRARY_PATH and other environment variables, which can adversely affect each other or even be mutually exclusive.

The settings for all these software packages and their supported versions are encapsulated in environment modules maintained by the LMod module system. These modules are not related to Perl or Python modules and should not be confused with those.

By means of the module system, all software currently available can be listed, loaded, and unloaded, by using the command module.

With the module command, you can explicitly address particular versions of a module by giving the full name (e.g., gcc/9.2.0 or fftw/3.3.8), or use the short forms (e.g., gcc or fftw). When using this short form, you address the version that is currently set as default (indicated by a (D) in the list of all modules).

Many modules depend on a compiler or an MPI implementation. These modules cannot be loaded directly -- instead, they become available only after loading a compiler and an MPI module (see module spider to find these dependencies).

  • module load Module_Names
    Loads the named software module(s). Only after this command (and when all other required modules have been loaded), the software becomes available to you.
  • module unload Module_Names
    Unloads the named modules. From this time on, the corresponding software is no longer usable.
  • module avail
    Shows all software modules that can currently be loaded, depending on which other modules are (not yet) loaded.
  • module avail Module_Name
    Shows all available versions of the given software.
    For example with module avail gcc, all available versions of GCC will be listed
  • module list
    Lists modules currently loaded
  • module help Module_Names
    Shows the detailed description (if any) of the named module(s).
  • module help
    Lists all subcommands of module .
  • module whatis Module_Names
    Shows the short descriptions of the named modules.
  • module spider
    Shows all modules in the module system (whether currently loadable or not).
  • module spider Module_Names
    Lists all available incarnations of this module. When specifying the version in the module name (“ml spider Module/Version”), it lists all combinations of prerequisites in which this module version is available.
  • module switch Module_Namenew or module switch Module_Nameold Module_Namenew
    Replaces the previously loaded (old) module version by the named version (new). Similar to the pair module unload / module load, but keeping the former order of the loaded modules.
  • module purge
    Unloads all currently loaded software modules.
    Recommended for job scripts before all “module load” commands, so as to start your jobs with a clean and reproducible environment.

Shortcut “ml”:

  • ml (without parameter) = module list
  • ml Module_Name(s) = module load Modul-Name(s)

Caution!

All batch jobs you submit will inherit the currently loaded modules of your login session! It is thus good practice to begin your job scripts always with “module purge”, followed by only the “module load … …” commands necessary for this job. This way your jobs always run with a clean and reproducible environment.

In our "Tips and Tricks " section, you can learn

  • how to load modules automatically at login
  • how to put together “collections” of certain modules, to be loaded time and again in one line.

Visibility and availability of conditional modules

Many software and libraries depend on a compiler or an MPI implementation. Their modules can thus not be loaded directly--instead, they become available only after loading a compiler and a mpi module.

The „module avail“ command will list only those modules you can currently load—dependent on what other modules are (not yet) loaded.

Therefore, some programs and libraries will show up only after loading a compiler and perhaps a MPI implementation. Some loaded modules will be amended (or even be getting new functionality) after loading other modules.

To find those “initially hidden” modules nonetheless, you can use the command „module spider“, showing all modules in the module system.

Another way is adding --show_hidden to “ml avail”:
module --show_hidden avail fftw

which will again list all modules named “fftw” in the whole module tree.

Example: according to „module spider“, there is the fftw module. In the list given by „module avail“ however, it will not show up, unless you load a compiler (via „ml gcc“ or „ml intel“). After having loaded a compiler module, „ml fftw“ will give you exactly the fftw binaries compiled with that compiler.

In case you load openmpi or intelmpi (before or after loading fftw), the loaded fftw will be replaced by the respective MPI-capable fftw (still based on the very same compiler).

in order to find out which modules are required by fftw, you should use the module spider fftw command.

Please don't hesitate to in case of problems with modules--we are happy to assist.