Gqsub Quickstart Guide
From ScotGrid
gqsub offers a way to submit jobs to the Grid, using the same sort of interface many are familiar with for using local batch system (Torque, PBS, SGE etc).
It is already installed on svr020, so you can just start using it.
When you use gqsub to launch a job, the output will end up in the same directory that you started the job from. If you run it within $CLUSTER_SHARED, then the shared filesystem will be used; otehrwise you will need to be explicit with stage in and stage out files.
gqsub aims to conforms to IEEE standard (POSIX) for batch system interfaces. There are a few features that are not implemented yet, and a few others that it would be impossible or impractical to do so. The most noticeable of these is email notification on job completion, which is not available.
A few common extensions to POSIX are also supported - in particular the use of array jobs (via -t flag), to launch a collection of jobs at once.
| Table of contents |
gqsub examples on ScotGrid
#!/bin/sh #PBS -l cput=0:30:00 #PBS -l walltime=0:30:00 #GQSUB -q UKI-SCOTGRID-GLASGOW #GQSUB -W stagein=test.c # A sample submission script that just does some diagnostics. # The key points are in the specification of the cpu time and wall time limits. # gqsub passes these to the WMS for use in job matching - they are therefore not optional! # The GQSUB directive -q indicates a 'batch server' that should be used. # Within the context of the gLite grid, this can be a fully specified queue, or a site name. echo in user script hostname pwd echo PBS_ARRAY_ID = $PBS_ARRAY_ID echo Compiling test file gcc --std c99 -o test test.c ./test
This example is the stock test script. In addition to the #PBS directives, it also supports a #GQSUB directive. This allows for Grid useage to co-exist with local useage (e.g. if it was installed on a cluster head node). Note that the #PBS can be changed, either with the -C command line argument, or in $HOME/.gqsubrc
If an array job was specified, (via -t) then the job number for this particular sub job would be printed.
This script supplies, compiles, and runs a C file - if you want to use this as a test script, expect it report errors as is - I suggest commenting out the ./test line.
Using alternate shells
Although the default assumption is to assume a shell script, this need not be the case. It is perfectly possible to submit script from any language. This is easier if the language supports that lines begining with the '#' character are comments - which includes a very large number of languages. For example, in Python:
#!/usr/bin/python #PBS -l cput=00:30:00 #PBS -S /usr/bin/python arr = range(1, 100) print sum(arr)
Note that we actually have the name of the shell interpreter (/usr/bin/python) twice in the script. The first one, at the top and beginning with '#!' is an instruction to the _local_ computer, if you were to run the script locally. This is typically used in development and debugging.
The second time it appears, in the -S directive to qsub, is an instruction to the _remote_ computer, where the job will be executed. In principle it is possible to have these different. That is not recommended, and a good way to complicate things!.
It is not possible to pass arguments to the shell interpreter via -S at the moment. The spec is ambiguous as to what should happen in that case. Therefore if that is needed, as in the case of the R statistics package, then the following is recommended:
#!/bin/bash #PBS -l cput=00:30:00 #GQSUB -W stagein=simple.csv R --vanilla << EndOfScript heisenberg <- read.csv(file="simple.csv",head=TRUE,sep=",") summary(heisenberg) EndOfScript
This will run equally well on local and remote systems. The R script is embedded using a BASH Here Document (http://tldp.org/LDP/abs/html/here-docs.html)' - i.e. it is fully contained between the two 'EndOfScript' tags (In principle, these can be any word, but must both be the same). In theory, you can skip the last one, if the file then ends. However, it is recommended to always include it - that gives a natural place to have some post processing, or chaining several invocations together.
MyProxy support
If you need myproxy support, for example you have jobs that last longer than the VOMS server maximum time limit, then the short term solution is handle that manually.
This means creating a file called 'default.jdl' in your $GQSUBDIR (normally $HOME/.gqsub). The contents of this file should be a a set of valid JDL directives, which will be included with each job submitted. For MyProxy support, put:
MyProxyServer = "lcgrbp01.gridpp.rl.ac.uk";
in this file.
Then, you need to set up the myproxy server, by running:
myproxy-init -d -n -c 200
which will lodge credentials for eight days. If you run that command once a week, you shouldn't need to worry about proxy expiry again.
Proper support (i.e. making gqsub aware of myproxy, and auto set it up, not prompt for password if MyProxy will cover it) is in the pipeline.
