Imagine you would like to perform a large series of calculation. Obviously, you would not run the complete series of calculations at the same time. In principle, you would like to start as many jobs as the number of processors on your computer can handle and start the next job in the series when the previous job has finished. You could write your own program for this, but there are many of such programs available. In this blog post, I will show you how to install and use Torque. Although Torque can be set up to run in a computer cluster, I will show you how you can install and use it on just a single machine. This tutorial is written for Linux Debian, but should in principle also work for Linux Ubuntu and with (hopefully) small modifications for the other distributions.
Download the tarball from the website. Extract, compile and install it on your machine.
wget http://www.adaptivecomputing.com/index.php?wpfb_dl=2868 -O torque-5.1.0.tar.gz
cd torque-5.1.0
./configure --prefix=/opt/gcc-4.7.2/torque-5.1.0
make -j5
sudo make install
Torque uses four daemons that have to be loaded at boot time. Copy their init.d
scripts to the /etc/init.d
folder like so
sudo cp contrib/init.d/debian.trqauthd /etc/init.d/trqauthd
sudo cp contrib/init.d/debian.pbs_mom /etc/init.d/pbs_mom
sudo cp contrib/init.d/debian.pbs_server /etc/init.d/pbs_server
sudo cp contrib/init.d/debian.pbs_sched /etc/init.d/pbs_sched
and add these to the boot procedure
sudo update-rc.d pbs_mom defaults
sudo update-rc.d pbs_server defaults
sudo update-rc.d pbs_sched defaults
sudo update-rc.d trqauthd defaults
Next, we would like to set our machine as both the server as well as the (only) node.
Start the trqauthd
daemon
sudo /etc/init.d/trqauthd start
Log in as root and add the binary folders of Torque to the path.
su
export PATH=/opt/gcc-4.7.2/torque-5.1.0/sbin:/opt/gcc-4.7.2/torque-5.1.0/bin:$PATH
./torque.setup root
If you get an error like the following
qmgr obj= svr=default: Bad ACL entry in host list MSG=First bad host:
check your /etc/hosts
file and change the directives in there. Torque only reads the first two columns to match the hostname with an IP adress.
Add your machine to the list of nodes by editing /var/spool/torque/server_priv/nodes
. You have to specify the number of cores in your machine after the np=
directive.
localhost np=6
Edit /var/spool/torque/mom_priv/config
and set your machine as the $pbsserver
. Also configure the bitmap for the logging events.
$pbsserver ST-A1771
$logevent 225
Please note that in the above file, ST-A1771
should be replaced by the name of your local machine. Moreover, this name should match an IP address which can be configured in /etc/hosts
. (thanks to danielmejia55_at_gmail_dot_com
for mentioning this, see comments below)
Start pbs_mom.
/etc/init.d/pbs_mom start
In order for every user to submit files to the queuing system and check the current status of the queue, you would like that every user has the bin
folder of Torque in their $PATH
variable. As such, add the Torque binaries folder to the PATH in /etc/profile
echo 'export PATH=/opt/gcc-4.7.2/torque-5.1.0/bin:$PATH' >> /etc/profile
Finally, log out as root (CTRL+D)
To check that everything is correctly configured, run
pbsnodes -a
If you do not get something like this, you can try to reset pbs_server
. (see below)
state = free
power_state = Running
np = 6
ntype = cluster
status = rectime=1424867568,cpuclock=OnDemand:1998MHz,varattr=,jobs=,state=free,netload=6890398942,gres=,loadave=0.00,ncpus=4,physmem=8066840kb,availmem=11324404kb,totmem=11970324kb,idletime=240,nusers=1,nsessions=2,sessions=3336 27395,uname=Linux ST-A1771 3.2.0-4-amd64 #1 SMP Debian 3.2.65-1+deb7u1 x86_64,opsys=linux
mom_service_port = 15002
mom_manager_port = 15003
To reset pbs_server
, run
sudo /etc/init.d/pbs_server restart
Also start the scheduler
sudo /etc/init.d/pbs_sched start
And test a job by running
echo "sleep 30" | qsub
When you run
qstat
you should see something like
Job ID Name User Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
0.ST-A1771 STDIN ivo 0 R batch
Note: danielmejia55_at_gmail_dot_com
has noted (see comments below) that if you encounter an error that certain queue directives are missing, that you need to set these first. He has kindly provided the settings he has used.
Below, an example submission file for a multiprocessor job is given. In the submission file, you specify the name of the job after the PBS -N
directive, the number of nodes and the number of processors per node and finally the maximum time the job is allowed to run. Typically, you would like to run the job in the same folder as where the jobfile is residing. To do so, you can use the $PBS_O_WORKDIR
variable. Furthermore, you can use the $PBS_NP
variable to pass the number of processes to the mpirun
program.
#!/bin/bash
#
#This is an example script example.sh
#
#These commands set up the Torque Environment for your job:
#PBS -N TestJob
#PBS -l nodes=1:ppn=4,walltime=00:12:00
pwd
cd $PBS_O_WORKDIR
pwd
#print the time and date
date
mpirun -np $PBS_NP ./testjob
#print the time and date again
date
No post data.
No querystring data.
To view Cookies, add CookieComponent to Controller
Query | Affected | Num. rows | Took (ms) | Actions |
---|---|---|---|---|
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 25 | 1 | 1 | 0 | |
SELECT `Post`.`active`, `Post`.`id` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 25 LIMIT 1 | 1 | 1 | 0 | |
SELECT `Comment`.`email`, `Comment`.`comment`, `Comment`.`id`, `Comment`.`post_id`, `Comment`.`created` FROM `ivofilot_nl`.`comments` AS `Comment` WHERE `Comment`.`post_id` = (25) | 3 | 3 | 0 | |
SELECT `Tag`.`id`, `Tag`.`name`, `Tag`.`icon`, `Tag`.`color`, `PostsTag`.`post_id`, `PostsTag`.`tag_id` FROM `ivofilot_nl`.`tags` AS `Tag` JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`PostsTag`.`post_id` = 25 AND `PostsTag`.`tag_id` = `Tag`.`id`) | 1 | 1 | 1 | |
UPDATE `posts` SET `watched`=`watched`+1 WHERE `id`='25' | 1 | 1 | 0 | |
SELECT `Comment`.`id`, `Comment`.`email`, `Comment`.`comment`, `Comment`.`post_id`, `Comment`.`parent_id`, `Comment`.`lft`, `Comment`.`rght`, `Comment`.`active`, `Comment`.`code`, `Comment`.`deleted`, `Comment`.`created`, `Comment`.`modified`, `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified`, `ParentComment`.`id`, `ParentComment`.`email`, `ParentComment`.`comment`, `ParentComment`.`post_id`, `ParentComment`.`parent_id`, `ParentComment`.`lft`, `ParentComment`.`rght`, `ParentComment`.`active`, `ParentComment`.`code`, `ParentComment`.`deleted`, `ParentComment`.`created`, `ParentComment`.`modified` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 25 AND `Comment`.`active` = '1' | 3 | 3 | 1 | maybe slow |
SELECT `Tag`.`id`, `Tag`.`name`, `Tag`.`icon`, `Tag`.`color` FROM `ivofilot_nl`.`tags` AS `Tag` inner JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`Tag`.`id` = `PostsTag`.`tag_id`) inner JOIN `ivofilot_nl`.`posts` AS `Post` ON (`PostsTag`.`post_id` = `Post`.`id`) WHERE `Post`.`id` = 25 | 1 | 1 | 1 | |
SELECT `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified`, `PostsTag`.`post_id`, `PostsTag`.`tag_id` FROM `ivofilot_nl`.`posts` AS `Post` JOIN `ivofilot_nl`.`posts_tags` AS `PostsTag` ON (`PostsTag`.`tag_id` = 18 AND `PostsTag`.`post_id` = `Post`.`id`) | 4 | 4 | 1 | maybe slow |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 18 | 1 | 1 | 1 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 23 | 1 | 1 | 0 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 25 | 1 | 1 | 0 | |
SELECT `Post`.`id`, `Post`.`title`, `Post`.`content`, `Post`.`watched`, `Post`.`active`, `Post`.`created`, `Post`.`modified` FROM `ivofilot_nl`.`posts` AS `Post` WHERE `Post`.`id` = 25 LIMIT 1 | 1 | 1 | 1 | |
SELECT COUNT(*) AS `count` FROM `ivofilot_nl`.`comments` AS `Comment` LEFT JOIN `ivofilot_nl`.`posts` AS `Post` ON (`Comment`.`post_id` = `Post`.`id`) LEFT JOIN `ivofilot_nl`.`comments` AS `ParentComment` ON (`Comment`.`parent_id` = `ParentComment`.`id`) WHERE `Comment`.`post_id` = 25 | 1 | 1 | 0 |
Peak Memory Use 3.61 MB
Message | Memory use |
---|---|
Component initialization | 792 KB |
Controller action start | 838 KB |
Controller render start | 1.44 MB |
View render complete | 1.91 MB |
Total Request Time: 223 (ms)
Message | Time in ms | Graph |
---|---|---|
Core Processing (Derived from $_SERVER["REQUEST_TIME"]) | 10.41 | |
Event: Controller.initialize | 0.07 | |
Event: Controller.startup | 6.04 | |
Controller action | 108.40 | |
Event: Controller.beforeRender | 26.68 | |
» Processing toolbar data | 26.61 | |
Rendering View | 54.62 | |
» Event: View.beforeRender | 0.03 | |
» Rendering APP/View/Posts/view.ctp | 53.18 | |
» » Rendering APP/View/Elements/code_highlighting.ctp | 0.46 | |
» » Rendering APP/View/Elements/post.commentform.ctp | 31.45 | |
» » » Rendering APP/View/Elements/post.comment.captcha.ctp | 4.91 | |
» » Rendering APP/View/Elements/post.comments.ctp | 5.30 | |
» » » Rendering APP/View/Elements/post.comment.ctp | 3.99 | |
» » » » Rendering APP/View/Elements/post.reply.ctp | 1.64 | |
» » » » » Rendering APP/View/Elements/post.comment.captcha.ctp #2 | 0.23 | |
» » » Rendering APP/View/Elements/post.comment.ctp #2 | 0.79 | |
» » » » » Rendering APP/View/Elements/post.reply.ctp #2 | 0.64 | |
» » » » » » Rendering APP/View/Elements/post.comment.captcha.ctp #3 | 0.20 | |
» » » Rendering APP/View/Elements/post.comment.ctp #3 | 0.73 | |
» » » » Rendering APP/View/Elements/post.reply.ctp #3 | 0.60 | |
» » » » » Rendering APP/View/Elements/post.comment.captcha.ctp #4 | 0.20 | |
» » Rendering APP/View/Elements/post.relatedpost.ctp | 0.60 | |
» Event: View.afterRender | 0.02 | |
» Event: View.beforeLayout | 0.02 | |
» Rendering APP/View/Layouts/default.ctp | 0.73 | |
» » Rendering APP/View/Elements/navbar.ctp | 0.17 | |
» » Rendering APP/View/Elements/footer.ctp | 0.15 | |
» » » Rendering APP/View/Elements/biography.ctp | 0.04 | |
Event: View.afterLayout | 0.00 |
Constant | Value |
---|---|
CONFIG | /customers/e/2/e/ivofilot.nl/httpd.www/app/Config/ |
Constant | Value |
---|---|
APP | /customers/e/2/e/ivofilot.nl/httpd.www/app/ |
APP_DIR | app |
APPLIBS | /customers/e/2/e/ivofilot.nl/httpd.www/app/Lib/ |
CACHE | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/cache/ |
CAKE | /customers/e/2/e/ivofilot.nl/httpd.www/lib/Cake/ |
CAKE_CORE_INCLUDE_PATH | /customers/e/2/e/ivofilot.nl/httpd.www/lib |
CORE_PATH | /customers/e/2/e/ivofilot.nl/httpd.www/lib/ |
CAKE_VERSION | 2.10.13 |
CSS | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/css/ |
CSS_URL | css/ |
DS | / |
FULL_BASE_URL | https://ivofilot.nl |
IMAGES | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/img/ |
IMAGES_URL | img/ |
JS | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/js/ |
JS_URL | js/ |
LOGS | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/logs/ |
ROOT | /customers/e/2/e/ivofilot.nl/httpd.www |
TESTS | /customers/e/2/e/ivofilot.nl/httpd.www/app/Test/ |
TMP | /customers/e/2/e/ivofilot.nl/httpd.www/app/tmp/ |
VENDORS | /customers/e/2/e/ivofilot.nl/httpd.www/vendors/ |
WEBROOT_DIR | webroot |
WWW_ROOT | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/ |
Environment Variable | Value |
---|---|
Php Version | 7.4.14 |
Onecom Domain Name | ivofilot.nl |
Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Onecom Memorylimit | 1073741824 |
Onecom Cpu Shares | 1024 |
Onecom Exec | latest |
Onecom Dir Layout Ver | 0 |
Content Length | 0 |
Http Connection | close |
Script Name | /app/webroot/index.php |
Request Uri | /posts/view/25/Creating+a+local+queuing+system+on+Linux+Debian+using+Torque |
Query String | |
Request Method | GET |
Server Protocol | HTTP/1.1 |
Gateway Interface | CGI/1.1 |
Redirect Url | /app/webroot/posts/view/25/Creating+a+local+queuing+system+on+Linux+Debian+using+Torque |
Remote Port | 53952 |
Script Filename | /customers/e/2/e/ivofilot.nl/httpd.www/app/webroot/index.php |
Server Admin | support@one.com |
Context Document Root | /var/www |
Context Prefix | |
Request Scheme | https |
Remote Addr | 184.72.102.217 |
Server Port | 80 |
Server Addr | 10.27.35.20 |
Server Name | ivofilot.nl |
Server Software | Apache |
Server Signature | |
Path | /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin |
Http X Varnish | 1055492369 |
Http Accept Encoding | gzip |
Http Host | ivofilot.nl |
Http X Onecom Host | ivofilot.nl |
Http X Forwarded Proto | https |
Http X Onecom Forwarded Proto | https |
Http X Forwarded For | 184.72.102.217 |
Http Accept Language | en-US,en;q=0.5 |
Http Accept | text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 |
Http User Agent | CCBot/2.0 (https://commoncrawl.org/faq/) |
Env Vcv Env Addons Id | one.com |
Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Onecom Wp Addons Api | https://wpapi.one.com |
Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Https | on |
Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Domain Name | ivofilot.nl |
Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Status | 200 |
Redirect Env Vcv Env Addons Id | one.com |
Redirect Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Redirect Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Redirect Onecom Wp Addons Api | https://wpapi.one.com |
Redirect Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Redirect Https | on |
Redirect Onecom Cpu Shares | 1024 |
Redirect Onecom Memorylimit | 1073741824 |
Redirect Onecom Exec | latest |
Redirect Onecom Dir Layout Ver | 0 |
Redirect Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Redirect Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Redirect Onecom Domain Name | ivofilot.nl |
Redirect Domain Name | ivofilot.nl |
Redirect Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Redirect Status | 200 |
Redirect Redirect Env Vcv Env Addons Id | one.com |
Redirect Redirect Env Vcv Token Url | https://wpapi.one.com/api/v1.0/plugins/visualcomposer/activate |
Redirect Redirect Onecom One Photo Url | https://onephoto.one.com/domain_discover |
Redirect Redirect Onecom Wp Addons Api | https://wpapi.one.com |
Redirect Redirect Onecom Webshop Host | webshop2.cst.webpod8-cph3.one.com |
Redirect Redirect Https | on |
Redirect Redirect Onecom Cpu Shares | 1024 |
Redirect Redirect Onecom Memorylimit | 1073741824 |
Redirect Redirect Onecom Exec | latest |
Redirect Redirect Onecom Dir Layout Ver | 0 |
Redirect Redirect Onecom Tmpdir | /customers/e/2/e/ivofilot.nl//tmp |
Redirect Redirect Onecom Domain Root | /customers/e/2/e/ivofilot.nl/ |
Redirect Redirect Onecom Domain Name | ivofilot.nl |
Redirect Redirect Domain Name | ivofilot.nl |
Redirect Redirect Onecom Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Redirect Redirect Document Root | /customers/e/2/e/ivofilot.nl/httpd.www |
Fcgi Role | RESPONDER |
Php Self | /app/webroot/index.php |
Request Time Float | 1611324035.8227 |
Request Time | 1611324035 |