Submission Interface - Usage

Overview

The general procedure for using the Submission Interface is as follows:

1. Modify our script/tool to call xgSubmit.exe and (optionally) xgWait.exe so that processes who's execution we wish to distribute are queued in the Grid Engine queue.
2. Run the script/tool with xgConsole.exe.

An Example

Let's say we are trying to distributed execution of a script called "Utest.bat", which runs a simple automated QA procedure consisting of the following steps:

1. Run the application "Utest.exe" 5 times, each time with different input data. Each run creates an output file with the execution results.
2. When done, run "UReport.exe" which checks the script results and displays a summary report.

Our original script looks like this:

Rem UTest.bat
Rem =========
UTest.exe /in=Input1.dat /out="Results\"
UTest.exe /in=Input2.dat /out="Results\"
UTest.exe /in=Input3.dat /out="Results\"
UTest.exe /in=Input4.dat /out="Results\"
UTest.exe /in=Input5.dat /out="Results\"
UReport.exe /in="Results\" /ShowReport
:end

If we could get the script to execute the different "Utest.exe" runs in parallel (utilizing local and remote CPUs) we would achieve a substantial speed-up. Here is the modified version of the .bat file which achieves this using the Submission Interface:

Rem UTest.bat - Distributed Version
Rem ===============================
xgSubmit.exe /group=UTest # UTest.exe /in=Input1.dat /out="Results\"
xgSubmit.exe /group=UTest # UTest.exe /in=Input2.dat /out="Results\"
xgSubmit.exe /group=UTest # UTest.exe /in=Input3.dat /out="Results\"
xgSubmit.exe /group=UTest # UTest.exe /in=Input4.dat /out="Results\"
xgSubmit.exe /group=UTest # UTest.exe /in=Input5.dat /out="Results\"
xgWait.exe /group=UTest
UReport.exe /in="Results\" /ShowReport
:end

Explanation

xgSubmit.exe replaces calls to UTest.exe, replacing the standard execution with execution through XGE. Note that the original command line is given as input following the "#" character. In this example, all "UTest.exe" calls are assigned the same group name - "UTest".
xgWait.exe is called before "UReport.exe" is run in order to pause execution of the script until all tasks in the group "UTest" have completed (this is required since "UReport.exe" reads output data created by the "UTest.exe" tool). This makes it possible to define dependencies between different execution parts in scripts.

Running the script using xgConsole.exe

Once our script has been properly modified, the following command line can be used to run it using IncrediBuild:

xgConsole.exe UTest.bat

For a comprehensive overview of the xgConsole.exe command line options see this section.

xgSubmit.exe

The xgSubmit.exe tool is used to submit tasks to the IncrediBuild Grid Engine . Submitted tasks are queued for execution and executed on either a remote or local CPU as soon as one becomes available. Each submitted task can be assigned a group identifier using the /group argument. xgWait.exe can then be used throughout the script to pause execution until all tasks with a given group identifier have completed.

xgSubmit.exe can be used to submit a single task or a set of tasks specified in a command file (for better performance when submitting a large group of tasks):

Single Task Submission Syntax

Use this syntax to submit a single task to the Grid Engine queue. The usage syntax is:

xgSubmit [/group=<Group Identifier>] # <Tool Command Line>

Arguments:

/group – The Group Identifier. If no group identified is given the submitted task will belong to a default group name.
# - The command line for execution should be specified following this character.

Command File Submission Syntax

Use this syntax for better performance when submitting a large set of tasks belonging to the same task group. A command file containing the command lines to be submitted separated by a newline character is given as input. xgSubmit.exe then reads and submits the commands specified in the command file. The usage syntax is:

xgSubmit [/group=<Group Identifier>] @ <Path to Command File>

Arguments:

/group – The Group Identifier. If no group identified is given the submitted task will belong to a default group name.
@ - The path to the command file storing the commands should be specified following this character.

xgWait.exe

The xgWait.exe tool can be used to pause execution until all tasks with a given group identifier have completed. This allows creating dependencies between different execution parts in the script.

The usage syntax is:

xgWait /group=<group[,group2,group3,...]> [/exitcode=<#|highest|lowest>] [/silent] [/all]

Switches:

/group – The Group Identifier(s) to wait for.
/exitcode - Exit code for xgWait.exe to return once all tasks have completed. Specify "highest" or "lowest" in order to return respectively the highest or lowest exit code of all codes returned by the task processes, or specify a fixed numeric value.
/silent - prevents xgWait.exe from writing to the standard output.
/all - instead of using /group, /all can be used to wait until all tasks have completed.

Notes

"xgWait.exe /all" is implicitly called at the end of the script/application's execution, in order to allow all tasks submitted with "xgSubmit.exe" to complete before returning control.

Converted from CHM to HTML with chm2web Standard 2.7 (unicode)