# Computing

This is a system that allows you to easily change between different versions of compilers and other software, no matter what shell you use, without having to set a lot of environment variables by hand each time.

The authors say: The Modules package is a database and set of scripts that simplify shell initialization and lets users easily modify their environment during the session. The Modules package lessens the burden of UNIX environment maintenance while providing a mechanism for the dynamic manipulation of application environment changes as single entities.

Availability:

All managed Linux workstations. Also used on all of the clusters.

Instructions for users:

### Basics

The machines have a variety of Fortran compilers, libraries, and other bits of software installed. Some of them come in multiple conflicting versions. It is becoming increasingly complicated to set up the user environment to use the appropriate ones, especially if people often need to switch between different versions for testing. The purpose of the module package is to simplify this.

On most of the machines some standard modules are automatically loaded on login.

To get started, list the loaded modules:

 cen1001@ucctemp2:~> module list
1) molden/4.6        3) vmd/1.8.6   2) mathematica/7.0   4) standard-apps 

You can also see a list of all available modules:

 cen1001@ucctemp2:~> module avail
---------------------------------- /usr/local/Modules/modulefiles -----------------------------------
FuDA                               matlab/R2010b                       acml/64/gfortran/4.4.0/mp          matlab/R2011b                       ...etc etc ... 

If you don't know what a module is then you can check:

cen1001@ucctemp2:~> module whatis mathematica/7.0
mathematica/7.0 : Loads version 7.0 of Mathematica

That didn't say anything we couldn't guess, but we can get more detailed information with the 'help' command:

 cen1001@ucctemp2:~> module help mathematica/7.0
----------- Module Specific Help for 'mathematica/7.0' ------------
mathematica - loads version 7.0 of Mathematica
to the PATH. Mathematica is a numeric and symbolic maths package.
It relies on a licence server to operate. If you cannot get
a licence then use the monitorlm command to see who is using
them. Please remember to close Mathematica after use.
There is extensive internal documentation in the Help menu
in the GUI, and online at http://documents.wolfram.com/mathematica/ 

 cen1001@ucctemp2:~> module show mathematica/7.0
-------------------------------------------------------------------
/usr/local/shared/suse-11.1/Modules/modulefiles/mathematica/7.0:
module-whatis    Loads version 7.0 of Mathematica
conflict         mathematica
prepend-path     PATH /usr/local/shared/Wolfram/Mathematica/7.0/Executables
------------------------------------------------------------------- 

Some modules conflict with others. For example, you may only have one of the ifort modules loaded at once. If you try to load another you get an error:

 cen1001@ucctemp2:~> module add ifort/64/11.0/074
ifort/64/10.1/021(6):ERROR:150: Module 'ifort/64/10.1/021' conflicts with the currently loaded module(s) 'ifort/64/11.0/074'
ifort/64/10.1/021(6):ERROR:102: Tcl command execution failed: conflict ifort  

To fix this, unload the conflicting module and try the load again:

cen1001@ucctemp2:~> module rm ifort/64/11.0/074
cen1001@ucctemp2:~> module add ifort/64/10.1/021
Licence Details:

Documentation:

The modules system can do much more than what I've covered above. There are manpages for it and the module command has built in help: type module help. Lots of helpful stuff for admins can be found on the modules-interest mailing list archive.

### Where to find things

The module configuration files live in /etc/environment-modules on newer managed workstation machines and /usr/local/Modules on older ones. Either way, the files are installed from a local Debian package. On compute servers they are generally in /usr/local/Modules plus a couple of places which sync from the NFS server: /usr/local/shared/intel/modulefiles and the like. The config files are written in Tcl, but you don't need to know Tcl (much) because most of the commands you use are extensions written for the modules package. The easiest way to learn is to copy an existing modulefile and edit it. A full list of modulefile commands is in man 4 modulefile .

Modules can be versioned. Instead of a single modulefile, there is a directory named what the basic module would have been named, and modulefiles within that directory named after the different versions. The directory may also contain a .modulerc file. This is also a Tcl modulefile and will be run if any module out of the directory is requested. This file can set the default version of the module to use. This doesn't work well for multiple levels of nesting; when making deeply nested modulefiles there has to be a .modulerc at each level pointing to the appropriate next directory level, rather than just one at the top level.

You can add new module directory trees to the PATH by editing the init/.modulepath file in the module install directory, but this file is controlled by FAI so edit it on the FAI server. Users can use their own private modules directory by loading the module use.own, which sets modules up to search ~/privatemodules as well as the system module directories.

### Modules for libraries

Modules for libraries are awkward, because the library location has to be made available to the linker at link time, and the location also has to be available to the runtime linker at runtime if it's a dynamic library.

This can be done by setting LD_LIBRARY_PATH but this has disadvantages. If you set it to contain an NFS filesystem and your NFS server becomes unavailable then your shell will freeze.

Instead, where possible, we set a variable LIBRARY_PATH in the library module to contain the location of the library, and configure our compilers to a) use that as a directory to look for libraries when linking and b) to hardwire that directory into the finished binary's RUNPATH in the ELF header, so the runtime linker looks in that directory too, but only for that particular program. This is a lot less dangerous than using LD_LIBRARY_PATH.

This doesn't work in every case- MKL being a particular example- because if you link a library that requires another library, the RUNPATH is not used to search for the second library. In this case you have to use LD_LIBRARY_PATH or other methods.

Thanks to these problems, the MKL modules have to set LD_LIBRARY_PATH. The shell freezing problem is avoided by never loading the MKL modules by default- the user has to make a conscious decision to load them. Some versions of the Intel MKL modulefiles set MKLPATH too, which I used to use for similar purposes as LIBRARY_PATH.

### Initialization considerations

You have to be very careful when forcing a default set of modules to be loaded in system startup files, because if the user loads a module in their personal shell startup that conflicts with the system ones you get problems. The only really safe thing to do is to give the user a default set of shell startup files in their home directory which load the default modules you want- don't put them in the system startup files. As their own files are completely under their control they can edit out any default modules they don't want.

With the C-shell and friends it's important to only initalise and load modules for interactive shells. This isn't a problem for Bourne type shells because they don't run initialisation for non-interative shells by default, but .cshrc is run for evey C-shell, so you can end up reloading modules when you don't want to (for example when you start a shell script from your interactive shell) and getting conflicts.

### Defaults and deep nesting

There is a problem with the current version of Modules where defaults only work totally correctly for modulefiles with all their versions in the top level; ie you can't have nested directories. You can set a .modulerc at each level as described above, but if any of the module families have common pathname components (like for example ifort and icc do- the '64' and '32' components) you can only do this for one family or you get errors about duplicate defaults. This is a problem for the local collection of compilers and libraries because we have many modules like 'mkl/64/serial/9.1/023' and sometimes want to set a particular module version as default. The best fix I have found for this so far is to make a dummy modulefile in the top level that contains something like:

 module load ifort/32/9.0/032  proc ModulesHelp { } {   puts "This is a dummy module; to see what it does type \"module disp [module-info name ]\"\n"   }    module-whatis "This is a dummy module; type \"module disp [module-info name ]\" to see what it does\n"

and then make a .modulerc file in top level setting that module as the default. When users type 'module load ifort' they get the right default.

### Zero effort modulefiles

I found a nice trick on the modules-interest mailing list which lets you use the exact same modulefile for every single version of a compiler family. You can use the command

 set components [ file split [ module-info name ] ]

to grab the filename of the modulefile and split it into its parts, then you can operate on those parts and use them to set things like the compiler version variables. We now use these extensively for things like the Intel and PGI compilers. The 'magic' modulefiles are kept in a directory 'basefiles' which is at the same level as the 'modulefiles' directory, and we make symlinks to the contents from the 'modulefiles' tree. Some older modules installations around the place use the similar trick of naming the 'magic' modulefile something starting with a dot within the 'modulefiles' tree, and linking to that.

### System status

System monitoring page

## Can't find what you're looking for?

Then you might find our A-Z site index useful. Or, you can search the site using the box at the top of the page, or by clicking here.