How to Run PBS Jobs in PTP

Introduction

The PBS Eclipse plug-in, part of the Parallel Tools Platform (PTP), allows you to launch and monitor applications on local or remote PBS (Portable Batch System) Distributed Resource Managers.

This plugin consists of two parts. The first is the Eclipse PTP plugin code which is responsible for the PTP views presented to the user and the user's interaction with those views. The second part is a resource manager (proxy), which is responsible for communicating with the PTP PBS plugin code, submitting the application as a job to the PBS batch system, and reporting application status to the PTP PBS plugin.

Requirements

In addition to having Eclipse and PTP installed, you need to have the following software packages installed.

CDT for Helios, available from the Eclipse download site;
If you wish to run jobs locally (LINUX), you will need to have installed a PBS system (PBS Pro, PBS Torque) there.

Installation

Nothing need be done beyond the normal intallation of the PTP feature. The PBS plugin features a pre-built executable .jar (pbs_proxy.jar) containing the proxy, which is then automatically deployed to the chosen location (local or remote) when the proxy is launched; there is nothing else to do by way of copying or compiling/building on the target resource.

Creating a Resource Manager

The PBS Resource Manager has no general preferences to be set. However, when an instance of the manager is created, the user has the option to pre-define which batch-script template it will use, or to create a custom template, as will be described below.

To create a resource manager instance, first open the Parallel Runtime perspective:

From the Window menu, select "Open Perspective" >> "Other"; in the dialog box, choose "Parallel Runtime", then "OK":

Choose Parallel Runtime

Within the Parallel Runtime Perspective, in the "Resource Managers" view, right- (control-) click and select the "Add Resource Manager..." action:

You should now see the first of five wizard pages that appear in the dialog ("Choose Resource Manager Type"). This one allows you to select which type of resource manager to instantiate:

In the "Choose Resource Manager Type" dialog, choose "PBS", and click "Next". The following page ("PBS Proxy Configuration") allows you to set up the PBS provider. The location can be either local or remote; in the former case, nothing else need be done on this page. For remote PBS, you will need to configure the connection to the host/resource. For this case, next to "Remote service provider", select the "Remote Tools" option:

Next, still in the "PBS Proxy Configuration" dialog, click the "New" button next to "Connection name" to bring up the connection dialog entitled "Target Environment Configuration" as shown here:

Configure Remote Connection

Here under "Remote Host" you can name the resource, provide its address, and set up the SSH connection with the user name and password, or choose public key authentication.

When you have finished entering the appropriate values, click "Finish" to return to the wizard page.

Also, back in the "PBS Proxy Configuration" dialog, be sure to choose the "SSH port forwarding" option under "Tunneling Options".

Click "Next" to bring you to the template configuration page. Here you can set up which batch template will be used to configure launches for this resource manager (this can, however, be changed in the Run Configuration; see Launch Tab). You can also use this page to create, modify, or delete templates.

Two pre-defined templates are provided with the PTP PBS plugin: the default_template has only a minimal set of configurable job attributes; the full_template provides all of the standard PBS job attributes, and also allows you to insert additional commands into the batch script. In this view, the configurable values of the template are displayed, along with the the tool tip, if any, that can be seen in the run configuration view by hovering over the attribute name.

To create a new template, click the "Edit Template" button. This pops up a dialog asking you which template to display:

Create New Template

If you are modifying a pre-existing template, just choose the template from the drop-down list; to create a new template, choose the template on which you wish to base the customized one. This now opens a dialog which allows you to edit and save the changes.

Edit Template

A fuller description of the template contents can be found below (PBS Batch Script Template).

To remove a template, use the "Delete Template" button.

Note: the two provided templates can neither be edited nor deleted; attempting to do so will display an error message.

Clicking "OK" should persist the changes and add the new template to the drop-down list (in the current example, alr_template was created):

The new template is thus available for use by any PBS resource manager instance.

The last two pages of this wizard allow you to name the resource manager and create its configuration (or add it to an empty pre-existing one). After you have completed the wizard configuration and clicked "Finish", the new resource manager should appear in the Resource Managers view (see next section).

We are now ready to deploy this resource manager and submit a job to it. The following section demonstrates how to use the PBS launch configuration to complete this process.

Submitting a Job

While not strictly necessary, we will first start the PBS resource manager before proceeding to the creation of a run configuration.

Select the PBS resource manager and choose "Start Resource Manager" from the context menu (right mouse):

The PBS resource manager should turn green indicated it has been started successfully.

In order to launch a job, you will also have to have defined an appropriate project, either a C/C++ project using CDT, or a Fortran project using Photran, and to have compiled an executable for it.

Next, we will create a configuration for launching jobs to this manager. The procedure is identical to that for creating any Run Configuration in Eclipse.

Either select Run >> "Run Configurations ..." using the menu, or click on the small arrow to the right of the Run icon on the toolbar:

From the list of configuration types displayed, right- (control-) click on "Parallel Application", and choose "New". A new configuration will be added and the tab viewer associated with it displayed. If only one resource manager is available from the list in the Resource Managers view, the "Resource" tab will be automatically configured with it; otherwise, you will need to select a resource manager from the drop down list at the top of the view. Here, we have selected the PBS@Local resource manager, whose template was defined to be the customized alr_template; we have also renamed this configuration "PBS_alr":

Note at this point several things: first, the "Apply" button, which allows you to save changes, is not yet enabled. This is because there are still "errors" to be fixed before the configuration is considered valid. These errors are displayed at the top of the panel proceeded by an error icon. In our case we see displayed there "[Application]: Application program is not specified". The Application tab is warning us that we need to tell the manager which application it will run. Clicking on the tab switches to that wizard page, where you can choose the project (at least one project in the workspace needs to be defined, as mentioned above) and point the "Application program" to the executable path (if your proxy is remote, the remote file system will be accessed by the "Browse" button). In the current release, you also need to define the debugger (this will be fixed in a subsequent release). However, even if there is no debugger available, all you need to do is open the Debugger tab, select the debugger (usually "SDM") and give it a dummy path (any file will do). You should now be able to save the configuration by clicking "Apply".

The PBS Launch Attributes

The "Launch Attributes" area of the "Resources" tab, which allows you to configure the job -- in more specific terms, generate the script to be given to the PBS batch system via the proxy -- is built from the chosen template, and consists of three sections. The top panel allows you to switch out the current template for any of the other available ones, and also allows you to see what the batch script to be given to the proxy will look like, on the basis of the actual values you have provided. The second panel allows you to choose among no MPI command and the two standard ones (mpiexec, mpirun). If the chosen template provides for arbitrary commands to be added to the script, the two "Edit" buttons will appear; clicking on these brings up a dialog in which you may enter any arbitrary commands that will be understood by the bash shell. The bottom panel exposes the "#PBS" directives included in this template, and allows you to provide their values: "spinners" are provided for integer values, selection boxes for values with a fixed set of choices, checkboxes for boolean (true/false, yes/no) values, and text boxes for all the other types. Hovering over the attribute name will often give you additional information on formatting for that particular attribute.

Launch Tab Details

Persistence

When you click "Apply", the values you entered are saved to persistent store; however, there are actually two different mappings to be aware of in this context. The template values, such as "MPI Command", bash commands, job attribute values, etc., belong to this particular Run Configuration (in our example, PBS_alr); the choice of template, however, belongs to the Resource Manager (in our example, PBS@Local). To illustrate this, let us suppose we had two different configurations, PBS_A and PBS_B; let us further suppose they both use the PBS@Local manager:

PBS_A changes the template for this resource manager. Inside PBS_A, you will now see the tab rebuilt for that template, but with the values (e.g., Account_Name, etc.) that were originally there. If you switched (after saving these changes) to PBS_B, you would now see its orignal values reappear inside the new template as well.

Now let us say that PBS_B actually swaps out its resource manager for a different (PBS) one. Again, it will see the original attribute values reappear inside the template associated with the new resource manager. PBS_A, however, will remain unaffected.

Launching

As with other resource managers, submission to the proxy takes place when you click on the "Run" button at the bottom of the Run Configuration. If the launch is successful, you should see your job appear in the Jobs List view; this viewer is populated by the update events generated by the proxy in connection with the polling of the batch system, so when the job completes, it will also be removed from this list. Any errors in the submission should appear in an error dialog.

The PBS Batch Script Template

For those wishing to create customized templates, we provide here a description of the template contents.

By way of illustration, the following reproduces the full_template provided with the PBS plugin:

#!/bin/bash

#####################################################################
## Template for PBS Batch Script Generated by PBS Resource Manager
## Launch Tab
## 
## This template contains all the currently recognized Job Attributes.
## 
## Placeholders (@NAME@) are included for the PBS Job Attribute 
## names as specified by qsub, plus the following internal variables:
##
## - env                     : place for defining extra environment
##                             variables (NB: should not be removed)
## - prependedBash           : dynamically change arbitrary bash 
##                             commands which should precede the 
##                             execution of the main application code
## - mpiCommand mpiOptions   : run under MPI
## - executablePath progArgs : the actual application
## - postpendedBash          : dynamically change arbitrary bash 
##                             commands which should follow the 
##                             execution of the main application code
##
## A template can also directly contain arbitrary shell scripting (not 
## to be replaced via the 'prepended' and 'postpended' placeholders);
## these lines will remain fixed and will not be exposed through
## the Launch Tab for modification (they can however be altered by 
## using the Resource Manager Properties "Edit" Tab).
##
## NOTE: We advise removing either the ncpus or the nodes resource,
##       depending on the PBS configuration (nodes is more common);
##       otherwise, the correct value must be set on both redundantly
##       in order for the MPI computation to be correct (and 
##       some systems might reject a script with both set).
#####################################################################

#PBS -A @Account_Name@
#PBS -c @Checkpoint@
#PBS -C @directive@
#PBS -e @Error_Path@
#PBS -h @Hold_Types@
#PBS -j @Join_Path@
#PBS -k @Keep_Files@
#PBS -l arch=@Resource_List.arch@
#PBS -l cput=@Resource_List.cput@
#PBS -l file=@Resource_List.file@
#PBS -l host=@Resource_List.host@
#PBS -l mem=@Resource_List.mem@
#PBS -l ncpus=@Resource_List.ncpus@
#PBS -l nice=@Resource_List.nice@
#PBS -l nodes=@Resource_List.nodes@
#PBS -l ompthreads=@Resource_List.ompthreads@
#PBS -l pcput=@Resource_List.pcput@
#PBS -l pmem=@Resource_List.pmem@
#PBS -l pvmem=@Resource_List.pvmem@
#PBS -l vmem=@Resource_List.vmem@
#PBS -l walltime=@Resource_List.walltime@
#PBS -m @Mail_Points@
#PBS -M @Mail_Users@
#PBS -N @Job_Name@
#PBS -o @Output_Path@
#PBS -p @Priority@
#PBS -q @destination@
#PBS -r @Rerunnable@
#PBS -S @Shell_Path_List@
#PBS -u @User_List@
#PBS -v @Variable_List@
#PBS -V @export_all@
#PBS -W depend=@depend@
#PBS -W group_list=@group_list@
#PBS -W stagein=@stagein@
#PBS -W stageout=@stageout@
@env@
@prependedBash@
cd @directory@
@mpiCommand@ @mpiOptions@ @executablePath@ @progArgs@
@postpendedBash@

The PBS Launch tab (Run Configuration) uses placeholder syntax to generate the actual batch script; in substituting the actual values, a placeholder marker (or, in the case of the PBS directives, the entire line) is eliminated if it is unused or would be an empty string. The following placeholders are valid:

@qsubAttribute@

These are the attributes associated with valid qsub flags. The currently recognized flags are all contained in the full_template. These are attributes generally accepted by all systems; future releases may add PBS Pro- or Torque- specific attributes to this list. If the attribute name is currently not recognized, an error will be reported.

@env@

If the user defines environment variables via the Run Configuration Environment tab, these will be captured and inserted as bash "export" commands at this location in the batch script.

@prependedBash@, @postpendedBash@

Any arbitrary bash commands added via the "Edit ... Commands" button/dialog are inserted here. Note: the template itself can be modified to carry arbitrary commands; these, however, will not appear in the launch tab, and will not be configurable (that is, they will be the same for all Run Configurations using this template); these placeholders allow you to use the same template but vary the additional commands from Run Configuration to Run Configuration, just as you can the attribute values.

@directory@

Not all PBS systems accept the -d qsub option, so any explicit directory change associated with the execution of the main application is taken care of here.

@mpiCommand@

Placeholder for the MPI command; if there is no MPI command, the mpiOptions placeholder is also eliminated.

@mpiOptions@

Currently, this consists of the "-n [mpicores]" argument. This is automatically computed from the appropriate qsub attribute: if ncpus is defined, this is used; else the nodes value is parsed for nodes X ppn.

@executablePath@

Replaced on the basis of the Run Configuration Application tab's executable path.

@progArgs@

Replaced on the basis of the Run Configuration Arguments tab.

Customization will normally involve either the inclusion or exclusion of individual qsubAttributes, or the bash command placeholders, but probably not the elimination or repositioning of the others; it may also involve the inclusion of other fixed bash commands or directives.

Back to Table of Contents