DQS is actually a simple system which provides a multitude of options to accommodate the requirements of a wide variety of sites, and users. As the number of options increase, as they do with each succeeding generation of DQS, a user might mistakenly come to view the system as quite complex. This user guide is intended to provide an introduction to DQS for the new user as well as explaining those features most often used by the experienced user. In particular the concept of "resources" is explored with attention focused on the new DQS 3.1.3 feature, "consumable resources". A DQS job

Any job which a user needs to execute on one or more computers can be a "DQS job". For those whose sole contact with computers has been through the means of personal UNIX workstations the concept of running their jobs in a "batch" mode may be somewhat disconcerting. Users accustomed to submitting their jobs to mainframe computers will be more familiar with the attributes of DQS. But, unlike the mainframe system the DQS batch environment customarily includes multiples of autonomous UNIX based computational platforms heterogeneous in hardware architecture and operating system variant.

In its most fundamental form, a DQS job is an extension of a UNIX script used to run an application, as one might even on their own personal workstation. Let us use the "traditional" example of a FORTRAN compilation and execution of a simple application:

FTN test.f -o test

Now let's make a change in the middle of the page. Is this going to work?

test

where test simply produces the classical "Hello World" output which is sent to standard error.

If we then wish to run this same application within a UNIX script we would create a file called "test.run" with the following lines: # #!/bin/sh # FTN test.f -o test >& test.errors # test > test.out

Note that we redirect the stdout and stderr files to "test.out" and "test.errors" respectively.

This script would then be executed by the user on a machine of their choice, most likely their own workstation.

What then is needed to turn this script into a DQS job ? Nothing… as long as one doesn't care what machine it will be executed on. All that is needed is to "submit" this job to the DQS batch queuing system.

submitting a job

The simple example becomes a "DQS job" by submitting it to the DQS system with the "qsub" utility :

qsub test.run

The qsub utility will contact the qmaster and request that the job be "validated" for execution within the system. This "validation" process of determining whether or not the not the job requires something which does not exist in the current system. Since our test script makes no obvious requests for resources (the FTN command is not recognized as a request for a compiler resource known by DQS) all that is needed is for any host in this hypothetical cell to be idle, and available to execute the job.

Let us now take advantage of some basic DQS facilities. First we would like to have an email message sent to us upon job termination. We must instruct DQS to perform this task by inserting a "DQS directive" into the test.run script. By default DQS interprets any line of script as "DQS directive" if the first two characters of the line are the string "#$". This can be changed by the user (see qsub -C option in the Reference Manual).

Thus we add one line to our script: # #!/bin/sh # #$ -me # FTN test.f -o test >& test.errors # test > test.out

The DQS directive #$ -me tells the system that a mail message should be sent to the person submitting the job at the end of the job. We could also have directed that we wish to have a mail message sent at the beginning of the job and also if the job aborts with the directive #$ -meab. The order of the symbol 'e' 'a' and 'b' in this list is not significant.

Note that the directive could also be communicated with DQS on the qsub command line. Instead of inserting the directive in the script, we could perform the submission with:

qsub -me test.run

In cases where only a few directives are needed this approach might be used, but as the user will see many job submissions will benefit from more complex sets of DQS directives which are better "captured" in the job script. querying job status

Once a user has relinquished their job to the "welcoming arms" of a queuing system they need a means for monitoring and controlling its destiny. A first step is to query the system to establish the status and "DQS identity" of the job. The "qstat" utility is used to display the state of queues and jobs. There are three forms of this display:

default (no options) Displays the state of user jobs in summary form

full listing (-f option) Displays summary queue and job status

extended listing (-ext option ) Displays the full queue and job descriptions

The simplest command then to get in touch with our job is to execute the command:

qstat

and scan through the output looking for jobs we have submitted. Instead of being deluged with information about every other job in the system one can execute:

qstat -u

where is the login name of the user who submitted the job.

-- JohnRowe - 04 Dec 2006