Let us think about the way we run programs from the terminal. If we want to list the content of a directory, we simply type ls
. Why does this work? You probably know the answer: Because the program ls
is residing in the /bin/ls
folder and that folder is in the $PATH
variable. In fact, you can check this by typing:
which ls
and
echo $PATH
What now if we have multiple versions of a particular program. For instance, we could have multiple versions installed of the GNU compiler. How would you then distinguish the programs? One could then append the version number to the binaries, like so:
and place all the folders where these executables reside in , in the $PATH
variable.
Let me now propose an alternative, which is often implemented in computational clusters. It is called Modules Environment. Using this tool, you simply type module avail
to check which programs are installed on the system and if you would like to use a particular program, let's say gcc-4.8.4, you would type module load gcc-4.8.4
. The Environment Modules tool will then prepend the folder where the executables of gcc-4.8.4 reside in to the $PATH
variable so that when you type gcc
, you will automatically use that particular version. Pretty nifty tool right? Well, you can use it on your local system as well and in this blog post I will explain you how.
Download the tarball
wget http://sourceforge.net/projects/modules/files/latest/download?source=files -O modules-3.2.10.tar.gz
and extract it
tar -xvzf modules-3.2.10.tar.gz
You need to have the tcl libraries installed, on Debian this can be done via:
sudo apt-get install tcl-dev
Next, run configure
like so
./configure --with-module-path=/opt/modulefiles --prefix=/opt/modules-3.2.10
Feel free to change the paths mentioned above to something suitable for your system. (Though the /opt
folder is meant to place installed programs in. Read more about the Linux directory structure here).
After configuring, run make
and make install
.
make
sudo make install
Lastly, you need to initialize the Environment Modules every time you load bash
. On Debian, this is done by adding two lines to /etc/profile
(a file that is run every time you run bash
).
Open /etc/profile
sudo nano /etc/profile
and add at the bottom of the file
# load the modules
source /opt/modules-3.2.10/Modules/3.2.10/init/bash
And you are all set up. If you now log out and back in, and type modules avail
, you should see something like this
--------- /opt/gcc-4.7.2/modules-3.2.10/Modules/versions ---------
3.2.10
---- /opt/gcc-4.7.2/modules-3.2.10/Modules/3.2.10/modulefiles ----
dot module-info null
module-git modules use.own
In the file /opt/modules-3.2.10/Modules/3.2.10/init/.modulespath
a list of directories are given in which modules
will search for module files. Personally, I don't like the default directories given there, and I tend to change it to something like this:
#
# This file defines the initial setup for the module files search path.
# Comments may be added anywhere, which begin on # and continue until the
# end of the line
# Each line containing a single path will be added to the MODULEPATH
# environment variable. You may add as many as you want - just
# limited by the maximum variable size of your shell.
#
/opt/modulefiles
and I then put in /opt/modulefiles
all my module files.
To exemplify, let us assume that we have two different versions of gcc installed, being 4.7.2 and 4.8.3. We then need to create two modulefiles as such
/opt/modulefiles/gnu-gcc-4.7.2
#%Module1.0#####################################################################
##
## null modulefile
##
## modulefiles/null. Generated from null.in by configure.
##
# for Tcl script use only
set root /opt/gcc-4.7.2
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib
prepend-path LD_LIBRARY_PATH $root/lib64
and
/opt/modulefiles/gnu-gcc-4.8.3
#%Module1.0#####################################################################
##
## null modulefile
##
## modulefiles/null. Generated from null.in by configure.
##
# for Tcl script use only
set root /opt/gcc-4.8.3
prepend-path PATH $root/bin
prepend-path LD_LIBRARY_PATH $root/lib
prepend-path LD_LIBRARY_PATH $root/lib64
If you would now run module avail
, (assuming you have a similar .modulespath
as mine) you would see something like this
-------- /opt/modulefiles --------
gnu-gcc-4.7.2 gnu-gcc-4.8.3
If I would now like to use gcc-4.7.2 to compile, I would type module load gnu-gcc-4.7.2
and have the bin directories of gcc-4.7.2
appended to my $PATH
ensuring that I use the right compiler.
More advanced configuration would actually modify the modulefiles in such a way that when gcc-4.7.2 is loaded and you would run module load 4.8.3
that then a conflict error would be given.