qresub

SUBMIT(1)                  Grid Engine User Commands                 SUBMIT(1)



NAME
       qsub   -  submit a batch job to Grid Engine.

       qsh    -  submit an interactive X-windows session to Grid Engine.

       qlogin -  submit an interactive login session to Grid Engine.

       qrsh   -  submit an interactive rsh session to Grid Engine.

       qalter -  modify a pending batch job of Grid Engine.

       qresub -  submit a copy of an existing Grid Engine job.

SYNTAX
       qsub [ options ] [ scriptfile | -- [ script_args ]]

       qsh [ options ] [ -- xterm_args ]

       qlogin [ options ]

       qrsh [ options ] [ command [ command_args ]]

       qalter [ options ] job/task_id_list [ -- [ script_args ]]

       qalter [ options ] -u user_list | -uall [ -- [ script_args ]]

       qresub [ options ] job_id_list

DESCRIPTION
       Qsub submits batch jobs to the Grid Engine queuing system.  Grid Engine
       supports single and multiple node jobs.  scriptfile contains the
       commands to be run by the job using a shell (for example, sh(1) or
       csh(1)).  Arguments to the job script are given by script_args.  Grid
       Engine flags may be entered as arguments to qsub or as embedded flags
       in the scriptfile if the first two characters of a script line either
       match '#$' or are equal to the prefix string defined with the -C option
       described below.

       Qsh submits an interactive X-windows session to Grid Engine. An
       xterm(1) is brought up from the executing machine with the display
       directed either to the X-server indicated by the DISPLAY environment
       variable or as specified with the -display qsh option. Interactive jobs
       are not spooled if no resource is available to execute them. They are
       either dispatched to a suitable machine for execution immediately or
       the user submitting the job is notified by qsh that appropriate
       resources to execute the job are not available.  xterm_args are passed
       to the xterm(1) executable.

       Qlogin is similar to qsh in that it submits an interactive job to the
       queueing system. It does not open an xterm(1) window on the X display,
       but uses the current terminal for user I/O. Usually, qlogin establishes
       a telnet(1) connection with the remote host, using standard client- and
       server-side commands. These commands can be configured with the
       qlogin_daemon (server-side, Grid Engine telnetd if not set, otherwise
       something like /usr/sbin/in.telnetd) and qlogin_command (client-side,
       Grid Engine telnet if not set, otherwise something like
       /usr/bin/telnet) parameters in the global and local configuration
       settings of sge_conf(5).  The client side command is automatically
       parameterized with the remote host name and port number to connect to
       (i.e. resulting in an invocation like /usr/bin/telnet my_exec_host
       2442).  Qlogin is invoked exactly like qsh and its jobs can only run on
       INTERACTIVE queues.  Qlogin jobs can only be used if the sge_execd(8)
       is running under the root account.

       Qrsh is similar to qlogin in that it submits an interactive job to the
       queuing system.  It uses the current terminal for user I/O.  Usually,
       qrsh establishes a rsh(1) connection with the remote host. If no
       command is given to qrsh, a rlogin(1) session is established.  The
       server-side commands used can be configured with the rsh_daemon and
       rlogin_daemon parameters in the global and local configuration settings
       of sge_conf(5).  A Grid Engine rshd or rlogind is used, if the
       parameters are not set or otherwise something like /usr/sbin/in.rshd or
       /usr/sbin/in.rlogind.  On the client-side, the rsh_command and
       rlogin_command parameters can be set in the global and local
       configuration settings of sge_conf(5).  If they are not set, special
       Grid Engine rsh(1) and rlogin(1) binaries delivered with Grid Engine
       are used.  Use the cluster configuration parameters to integrate
       mechanisms like ssh or the rsh(1) and rlogin(1) facilities supplied
       with the operating system.

       Qrsh jobs can only run in INTERACTIVE queues unless the option -now no
       is used (see below).  They can only be used, if the sge_execd(8) is
       running under the root account.

       Qrsh provides an additional feature useful for the integration with
       interactive tools providing a specific command shell. If the
       environment variable QRSH_WRAPPER is set when qrsh is invoked, the
       command interpreter pointed to by QRSH_WRAPPER will be executed to run
       qrsh commands instead of the users login shell or any shell specified
       in the qrsh command-line.

       Qalter can be used to change the attributes of pending jobs. Once a job
       is executing, changes are no longer possible. For array jobs, for which
       a part of the tasks can be pending and another part can be running (see
       the -t option below), modifications with qalter only affect the pending
       tasks.  Qalter can change most of the characteristics of a job (see the
       corresponding statements in the OPTIONS section below), including those
       which were defined as embedded flags in the script file (see above).

       Qresub allows to create jobs as copies from existing pending or running
       jobs. The copied jobs will have exactly the same attributes as the ones
       from which they are copied, but a new job ID. The only modification to
       the copied jobs supported by qresub is to assign a hold state with the
       -h option. This can be used to first copy a job and then change its
       attributes via qalter.

       For qsub, qsh, qrsh, and qlogin the administrator and the user may
       define default request files (see sge_request(5)) which can contain any
       of the options described below.  If an option in a default request file
       is understood by qsub and qlogin but not by qsh the option is silently
       ignored if qsh is invoked. Thus you can maintain shared default request
       files for both qsub and qsh.

       A cluster wide default request file may be placed under
       $SGE_ROOT/$SGE_CELL/common/sge_request.  User private default request
       files are processed under the locations $HOME/.sge_request and
       $cwd/.sge_request. The working directory local default request file has
       the highest precedence, then the home directory located file and then
       the cluster global file.  The option arguments, the embedded script
       flags and the options in the default request files are processed in the
       following order:

              left to right in the script line,
              left to right in the default request files,
              from top to bottom of the script file (qsub only),
              from top to bottom of default request files,
              from left to right of the command line.

       In other words, the command line can be used to override the embedded
       flags and the default request settings.  The embedded flags, however,
       will override the default settings.

       Note, that the -clear option can be used to discard any previous
       settings at any time in a default request file, in the embedded script
       flags, or in a command-line option. It is, however, not available with
       qalter.

       The options described below can be requested either hard or soft.  By
       default, all requests are considered hard until the -soft option (see
       below) is encountered. The hard/soft status remains in effect until its
       counterpart is encountered again.  If all the hard requests for a job
       cannot be met, the job will not be scheduled.  Jobs which cannot be run
       at the present time remain spooled.

OPTIONS
       -@ optionfile
              Forces qsub, qrsh, qsh, or qlogin to use the options contained
              in optionfile. The indicated file may contain all valid options.
              Comment lines are starting with a "#" sign.

       -a date_time
              Available for qsub and qalter only.

              Defines or redefines the time and date at which a job is
              eligible for execution. Date_time conforms to
              [[CC]]YY]MMDDhhmm.[ss], where:

              CC  denotes the century in 2 digits.
              YY  denotes the year in 2 digits.
              MM  denotes the month in 2 digits.
              DD  denotes the day in 2 digits.
              hh  denotes the hour in 2 digits.
              mm  denotes the minute in 2 digits.
              ss  denotes the seconds in 2 digits (default 00).

              If any of the optional date fields is omitted, the corresponding
              value of the current date is assumed. If CC is not specified, a
              YY of < 70 means 20YY.
              Usage of this option may cause unexpected results if the clocks
              of the hosts in the Grid Engine pool are out of sync. Also, the
              proper behavior of this option very much depends on the correct
              setting of the appropriate timezone, e.g. in the TZ environment
              variable (see date(1) for details), when the Grid Engine daemons
              sge_qmaster(8) and sge_execd(8) are invoked.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -ac variable[=value],...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Adds the given name/value pair(s) to the job's context. Value
              may be omitted. Grid Engine appends the given argument to the
              list of context variables for the job.  Multiple -ac, -dc, and
              -sc options may be given.  The order is important here.

              Qalter allows changing this option even while the job executes.

       -A account_string
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Identifies the account to which the resource consumption of the
              job should be charged. The account_string may be any arbitrary
              ASCII alphanumeric string but may contain no blank, colon or
              separator characters. The underbar '_' is considered a non-
              separator. In the absence of this parameter Grid Engine will
              place the default account string "sge" in the accounting record
              of the job.

              Qalter allows changing this option even while the job executes.

       -c occasion_specifier
              Available for qsub and qalter only.

              Defines or redefines whether the job should be checkpointed, and
              if so, under what circumstances. The specification of the
              checkpointing occasions with this option overwrites the
              definitions of the when parameter in the checkpointing
              environment (see checkpoint(5)) referenced by the qsub -ckpt
              switch.  Possible values for occasion_specifier are

              n           no checkpoint is performed.
              s           checkpoint when batch server is shut down.
              m           checkpoint at minimum CPU interval.
              x           checkpoint when job gets suspended.
              <interval>  checkpoint in the specified time interval.

              The minimum CPU interval is defined in the queue configuration
              (see queue_conf(5) for details).  <interval> has to be specified
              in the format hh:mm:ss.  The maximum of <interval> and the
              queue's minimum CPU interval is used if <interval> is specified.
              This is done to ensure that a machine is not overloaded by
              checkpoints being generated too frequently.

       -ckpt ckpt_name
              Available for qsub and qalter only.

              Selects the checkpointing environment (see checkpoint(5)) to be
              used for a checkpointing the job. Also declares the job to be a
              checkpointing job.

       -clear Available for qsub, qsh, qrsh, and qlogin only.

              Causes all elements of the job to be reset to the initial
              default status prior to applying any modifications (if any)
              appearing in this specific command.

       -cwd   Available for qsub, qsh, qrsh and qalter only.

              Execute the job from the current working directory.  This switch
              will activate Grid Engine's path aliasing facility, if the
              corresponding configuration files are present (see
              sge_aliases(5)).

              In case of qalter, the previous definition of the current
              working directory will be overwritten, if qalter is executed
              from a different directory than the preceding qsub or qalter.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -C prefix_string
              Available for qsub only.

              Prefix_string defines the prefix that declares a directive to
              qsub in the job's scriptfile. The prefix is not a job attribute,
              but affects the behavior of qsub. If the -C option is presented
              with the value of the directive prefix as a null string, qsub
              will not scan the scriptfile
              The directive prefix consists of two ASCII characters which when
              appearing in the first two bytes of a script line indicate that
              what follows is a Grid Engine command (default is "#$").
              The user should be aware that changing the first delimiter
              character can produce unforeseen side effects. If the script
              file contains anything other than a "#" character in the first
              byte position of the line, the shell processor for the job will
              reject the line and may exit the job prematurely.
              If the -C option is present in the script file, it is ignored.

       -dc variable,...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Removes the given variable(s) from the job's context.  Multiple
              -ac, -dc, and -sc options may be given.  The order is important
              here.

              Qalter allows changing this option even while the job executes.

       -display display_specifier
              Available for qsh only.

              Directs xterm(1) to use display_specifier in order to contact
              the X server.  The display_specifier has to contain the hostname
              part of the display name (e.g. myhost:1).  Local display names
              (e.g. :0) cannot be used in grid environments.  Values set with
              the -display option overwrite settings from the submission
              environment and from -v commandline options.

       -dl date_time
              Available for qsub, qsh, qrsh, qlogin and qalter only.  This
              option is only supported in case of a Grid Engine Enterprise
              Edition system. It is not available for Grid Engine systems.
              Specifies the deadline initiation time in [[CC]YY]DDhhmm[.SS]
              format (see -a option above). The deadline initiation time is
              the time at which a deadline job has to reach top priority to be
              able to complete within a given deadline. Before the deadline
              initiation time the priority of a deadline job will be raised
              steadily until it reaches the maximum as configured by the Grid
              Engine administrator.
              This option is applicable for users allowed to submit deadline
              jobs only.

       -e [[hostname]:]path,...
              Available for qsub and qalter only.

              Defines or redefines the path used for the standard error stream
              of the job. If the path constitutes an absolute path name, the
              error-path attribute of the job is set to its value including
              the hostname. If the path name is relative, Grid Engine expands
              path either with the current working directory path in case the
              -cwd (see above) switch is also specified or with the home
              directory path otherwise. If hostname is present, the standard
              error stream will be placed under the corresponding location if
              the job runs on the specified host. If a pathname with a ":"
              should be specifed, a leading ":" has to be added to the
              pathname.

              By default the file name for standard error has the form
              job_name.ejob_id and job_name.ejob_id.task_id for array job
              tasks (see -t option below).

              If path is a directory, the standard error stream of the job
              will be put in this directory under the default file name.  If
              the pathname contains certain pseudo environment variables,
              their value will be expanded at runtime of the job and will be
              used to constitute the standard error stream path name. The
              following pseudo environment variables are supported currently:

              $HOME       home directory on execution machine
              $USER       user ID of job owner
              $JOB_ID     current job ID
              $JOB_NAME   current job name (see -N option)
              $HOSTNAME   name of the execution host
              $TASK_ID    array job task index number

              Alternatively to $HOME the tilde sign "~" can be used as common
              in csh(1) or ksh(1).  Note, that the "~" sign also works in
              combination with user names, so that "~<user>" expands to the
              home directory of <user>. Using another user ID than that of the
              job owner requires corresponding permissions, of course.

              See sge_pe(1) how the use of the -o/-e submit flags affect the
              name and location of the PE output files.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -hard  Available for qsub, qsh, qrsh, qlogin and qalter only.

              Signifies that all resource requirements following in the
              command line will be hard requirements and must be satisfied in
              full before a job can be scheduled.
              As Grid Engine scans the command line and script file for Grid
              Engine options and parameters it builds a list of resources
              required by a job. All such resource requests are considered as
              absolutely essential for the job to commence. If the -soft
              option (see below) is encountered during the scan then all
              following resources are designated as "soft requirements" for
              execution, or "nice-to-have, but not essential". If the -hard
              flag is encountered at a later stage of the scan, all resource
              requests following it once again become "essential". The -hard
              and -soft options in effect act as "toggles" during the scan.

       -h | -h {u|s|o|n|U|O|S}...
              Available for qsub, qsh, qrsh, qlogin, qalter and qresub.

              List of holds to place on a job, a task, or some tasks of a job.

              `u'  denotes a user hold.
              `s'  denotes a system hold.
              `o'  denotes a operator hold.
              `n'  denotes no hold.

              As long as any hold other than `n' is assigned to the job the
              job is not eligible for execution. Holds can be released via
              qalter and qrls(1).  In case of qalter this is supported by the
              following additional option specifiers for the -h switch:

              `U'  removes a user hold.
              `S'  removes a system hold.
              `O'  removes a operator hold.

              Grid Engine managers can assign and remove all hold types, Grid
              Engine operators can assign and remove user and operator holds
              and users can only assign or remove user holds.

              In the case of qsub only user holds can be placed on a job and
              thus only the first form of the option with the -h switch alone
              is allowed.  As opposed to this, qalter requires the second form
              described above.

              An alternate means to assign hold is provided by the qhold(1)
              facility.

              If the job is a array job (see the -t option below), all tasks
              specified via -t are affected by the -h operation
              simultaneously.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -help  Prints  a listing of all options.

       -hold_jid [job_id|job_name],...
              Available for qsub, qrsh, and qalter only.

              Defines or redefines the job dependency list of the submitted
              job. A reference by job name is only accepted if the referenced
              job is owend by the same user as the refering job. The submitted
              job is not eligible for execution unless all jobs referenced in
              the coma separated job id and/or job name list have completed
              successfully.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -inherit
              Available only for qrsh and qmake(1).

              qrsh allows to start a task in an already scheduled parallel
              job.  The option -inherit tells qrsh to read a job id from the
              environment variable JOB_ID and start the specified command as a
              task in this job. Please note that in this case, the hostname of
              the host where the command shall be executed, must precede the
              command to execute; the syntax changes to

              qrsh -inherit [ other options ] hostname command [ command_args
              ]

              Note also, that in combination with -inherit, most other command
              line options will be ignored.  Only the options -verbose, -v and
              -V will be interpreted.  As a replacement to option -cwd please
              use -v PWD.

              Usually a task should have the same environment (including the
              current working directory) as the corresponding job, so
              specifying the option -V should be suitable for most
              applications.

              Note: If in your system the commd port is not configured as
              service, but via environment variable COMMD_PORT, make sure that
              this variable is set in the enviroment when calling qrsh or
              qmake with option -inherit. If you call qrsh or qmake with
              option -inherit from within a job script, export COMMD_PORT with
              the submit option or special comment "-v COMMD_PORT".

       -j y|n Available for qsub and qalter only.

              Specifies whether or not the standard error stream of the job is
              merged into the standard output stream.
              If both the -j y and the -e options are present, Grid Engine
              sets, but ignores the error-path attribute.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -l resource=value,...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Launch the job in a Grid Engine queue meeting the given resource
              request list.  In case of qalter the previous definition is
              replaced by the specified one.
              complex(5) describes how a list of available resources and their
              associated valid value specifiers can be obtained.
              There may be multiple -l switches in a single command. You may
              request multiple -l options to be soft or hard both in the same
              command line. In case of a serial job multiple -l switches
              refine the definition for the sought queue.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -m b|e|a|s|n,...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Defines or redefines under which circumstances mail is to be
              sent to the job owner or to the users defined with the -M option
              described below. The option arguments have the following
              meaning:

              `b'     Mail is sent at the beginning of the job.
              `e'     Mail is sent at the end of the job.
              `a'     Mail is sent when the job is aborted or
                      rescheduled.
              `s'     Mail is sent when the job is suspended.
              `n'     No mail is sent.

              Currently no mail is sent when a job is suspended.

              Qalter allows changing the b, e, and a option arguments even
              while the job executes. The modification of the b option
              argument will only be in effect after a restart or migration of
              the job, however.

       -M user[@host],...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Defines or redefines the list of users to which the server that
              executes the job has to send mail, if the server sends mail
              about the job.  Default is the job owner at the originating
              host.

              Qalter allows changing this option even while the job executes.

       -masterq queue,...
              Available for qsub, qrsh, qsh, qlogin and qalter.  Only
              meaningful for parallel jobs, i.e. together with the -pe option.

              Defines or redefines a list of queues which may be used to
              become the so called master queue of this parallel job. The
              master queue is defined as the queue where the parallel job is
              started. The other queues to which the parallel job spawns tasks
              are called slave queues.  A parallel job only has one master
              queue.

              This parameter has all the properties of a resource request and
              will be merged with requirements derived from the -l option
              described above.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -notify
              Available for qsub, qrsh (with command) and qalter only.

              This flag, when set causes Grid Engine to send "warning" signals
              to a running job prior to sending the signals themselves. If a
              SIGSTOP is pending the job will receive a SIGUSR1 several
              seconds before the SIGSTOP. If a SIGKILL is pending the job will
              receive a SIGUSR2 several seconds before the SIGKILL.  The
              amount of time delay is controlled by the notify parameter in
              each queue configuration (see queue_conf(5)).

              Note, that the Linux operating system "misuses" the user signals
              SIGUSR1 and SIGUSR2 in its current Posix thread implementation.
              You might not want to use the -notify option if you are running
              threaded applications in your jobs under Linux.

              Qalter allows changing this option even while the job executes.

       -now y[es]|n[o]
              Available for qsub, qsh, qlogin and qrsh.

              -now y tries to start the job immediately or not at all. The
              command returns 0 on success, or 1 on failure (also if the job
              could not be scheduled immediately).  -now y is default for qsh,
              qlogin and qrsh
              With option -now n the job will be put into the pending queue,
              if it cannot be executed immediately. -now n is default for
              qsub.

       -N name
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              The name of the job. The name can be any printable set of
              characters, starting with an alphabetic character.
              If the -N option is not present Grid Engine assigns the name of
              the job script to the job after any directory pathname has been
              removed from the script-name. If the script is read from
              standard input the job name defaults to STDIN.
              In case of qsh or qlogin and if the -N option is absent the
              string `INTERACT' is assigned to the job.

              Qalter allows changing this option even while the job executes.

       -noshell
              Available only for qrsh with a commandline.

              Do not start the commandline given to qrsh in a users login
              shell but execute it without the wrapping shell.

              The option can be used to speed up execution as some overhead
              like the shell startup and sourcing the shell resource files is
              avoided.

              The option can only be used, if no shell specific commandline
              parsing is required. If the commandline contains shell syntax
              like environment variable substitution or (back) quoting, a
              shell must be started.  In this case either do not use the
              -noshell option or include the shell call in the commandline.

              Example:
              qrsh echo '$HOSTNAME'
              Alternative call with the -noshell option
              qrsh -noshell /bin/tcsh -f -c 'echo $HOSTNAME'

       -nostdin
              Available only for qrsh.

              Suppress the input stream STDIN - qrsh will pass the option -n
              to the rsh(1) command. This is especially usefull, if multiple
              tasks are executed in parallel using qrsh, e.g. in a make(1)
              process - it would be undefined, which process would get the
              input.

       -o [[hostname]:]path,...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              The path used for the standard output stream of the job. The
              path is handled as described in the -e option for the standard
              error stream.

              By default the file name for standard output has the form
              job_name.ojob_id and job_name.ojob_id.task_id for array job
              tasks (see -t option below).

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -ot override_tickets
              Available for qalter only.  This option is only supported in
              case of a Grid Engine Enterprise Edition system. It is not
              available for Grid Engine systems.
              Changes the number of override tickets for the specified job.
              Requires manager/operator privileges.

       -P project_name
              Available for qsub, qsh, qrsh, qlogin and qalter only.  This
              option is only supported in case of a Grid Engine Enterprise
              Edition system. It is not available for Grid Engine systems.
              Specifies the project to which this job is assigned. The
              administrator needs to give permission to individual users to
              submit jobs to a specific project. (see -aprj option to
              qconf(1)).

       -p priority
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Defines or redefines the priority of the job relative to other
              jobs.  Priority is an integer in the range -1023 to 1024.  The
              default priority value for the jobs is 0.

              In a Grid Engine system, users may only decrease the priority of
              their jobs.  Grid Engine managers and administrators may also
              increase the priority associated with jobs. If a pending job has
              higher priority, it is earlier eligible for being dispatched by
              the Grid Engine scheduler. The job priority has no effect on
              running jobs in Grid Engine.

              In Grid Engine Enterprise Edition, the job priority influences
              the Share Tree Policy and the Functional Policy. It has no
              effect on the Deadline and Override Policies (see share_tree(5),
              sched_conf(5) and the Grid Engine Enterprise Edition
              Installation and Administration Guide for further information on
              the resource management policies supported by Grid Engine
              Enterprise Edition).

              In case of the Share Tree Policy, users can distribute the
              tickets, to which they are currently entitled, among their jobs
              using different priorities assigned via -p. If all jobs have the
              same priority value, the tickets are distributed evenly. Jobs
              receive tickets relative to the different priorities otherwise.
              Priorities are treated like an additional level in the share
              tree in the latter case.

              In connection with the Functional Policy, the priority can be
              used to weight jobs within the functional job category.  Again
              tickets are distributed relative to any uneven priority
              distribution treated as a virtual share distribution level
              underneath the functional job category.

              If both, the Share Tree and the Functional Policy are active,
              the job priorities will have an effect in both policies and the
              tickets independently derived in each of them are added up to
              the total number of tickets for each job.

       -pe parallel_environment n[-[m]]|[-]m,...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Parallel programming environment (PE) to instantiate. The range
              descriptor behind the PE name specifies the number of parallel
              processes to be run. Grid Engine will allocate the appropriate
              resources as available. The sge_pe(5) manual page contains
              information about the definition of PEs and about how to obtain
              a list of currently valid PEs.
              You can specify the PE name  by using the wildcard character
              "*", thus the request "pvm*" will match any parallel environment
              with a name starting with the string "pvm".
              The range specification is a list of range expressions of the
              form n-m (n as well as m being positive non-zero integer
              numbers), where m is an abbreviation for m-m, -m is a short form
              for 1-m and n- is an abbreviation for n-infinity. The range
              specification is processed as follows: The largest number of
              queues requested is checked first. If enough queues meeting the
              specified attribute list are available, all are allocated. The
              next smaller number of queues is checked next and so forth.
              If additional -l options are present, they restrict the set of
              eligible queues for the parallel job.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -q queue,...
              Available for qsub, qrsh, qsh, qlogin and qalter.

              Defines or redefines a list of queues which may be used to
              execute this job. This parameter has all the properties of a
              resource request and will be merged with requirements derived
              from the -l option described above.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -r y|n Available for qsub and qalter only.

              Identifies the ability of a job to be rerun or not.  If the
              value of -r is 'y', rerun the job if the job was aborted without
              leaving a consistent exit state (this is typically the case if
              the node on which the job is running crashes).  If -r is 'n', do
              not rerun the job under any circumstances.
              Interactive jobs submitted with qsh or qlogin are not re-
              runable.

              Qalter allows changing this option even while the job executes.

       -sc variable[=value],...
              Available for qsub, qsh, qrsh, qlogin and qalter only.

              Sets the given name/value pairs as the job's context. Value may
              be omitted. Grid Engine replaces the job's previously defined
              context with the one given as the argument.  Multiple -ac, -dc,
              and -sc options may be given.  The order is important here.
              Contexts are a way to dynamically attach and remove meta-
              information to and from a job. The context variables are not
              passed to the job's execution context in its environment.

              Qalter allows changing this option even while the job executes.

       -soft  Available for qsub, qsh, qrsh, qlogin and qalter only.

              Signifies that all resource requirements following in the
              command line will be soft requirements and are to be filled on
              an "as available" basis.
              As Grid Engine scans the command line and script file for Grid
              Engine options and parameters it builds a list of resources
              required by a job. All such resource requests are considered as
              absolutely essential for the job to commence. If the -soft
              option is encountered during the scan then all following
              resources are designated as "soft requirements" for execution,
              or "nice-to-have, but not essential". If the -hard flag (see
              above) is encountered at a later stage of the scan, all resource
              requests following it once again become "essential". The -hard
              and -soft options in effect act as "toggles" during the scan.

       -S [[hostname]:]pathname,...
              Available for qsub, qsh and qalter.

              Specifies the interpreting shell for the job. Only one pathname
              component without a host specifier is valid and only one path
              name for a given host is allowed. Shell paths with host
              assignments define the interpreting shell for the job if the
              host is the execution host. The shell path without host
              specification is used if the execution host matches none of the
              hosts in the list.

              Furthermore, the pathname can be constructed with pseudo
              environment variables as described for the -e option above.

              In the case of qsh the specified shell path is used to execute
              the corresponding command interpreter in the xterm(1) (via its
              -e option) started on behalf of the interactive job.

              Qalter allows changing this option even while the job executes.
              The modified parameter will only be in effect after a restart or
              migration of the job, however.

       -t n[-m[:s]]
              Available for qsub and qalter only.

              Submits a so called Array Job, i.e. an array of identical tasks
              being only differentiated by an index number and being treated
              by Grid Engine almost like a series of jobs. The option argument
              to -t specifies the number of array job tasks and the index
              number which will be associated with the tasks. The index
              numbers will be exported to the job tasks via the environment
              variable SGE_TASK_ID.

              Following restrictions apply to the values n and m:

                     1 <= n <= MIN(2^31-1, max_aj_tasks)
                     1 <= m <= MIN(2^31-1, max_aj_tasks)
                     n <= m

              The task id range specified in the option argument may be a
              single number, a simple range of the form n-m or a range with a
              step size. Hence, the task id range specified by 2-10:2 would
              result in the task id indexes 2, 4, 6, 8, and 10, i.e. in a
              total of 5 tasks identical tasks with the environment variable
              SGE_TASK_ID containing one of the 5 index numbers each.

              All array job tasks inherit the same resource requests and
              attribute definitions as specified in the qsub or qalter command
              line, except for the -t option. The tasks are scheduled
              independently and, provided enough resources, concurrently very
              much like separate jobs.  However, a array job or a sub-array
              thereof can be accessed as a total by commands like qmod(1) or
              qdel(1).  See the corresponding manual pages for further detail.

              Array jobs are commonly used to execute the same type of
              operation on varying input data sets correlated with the task
              index number. The number of tasks in a array job is unlimited.

              STDOUT and STDERR of array job tasks will be written into
              different files with the default location

              <jobname>.['e'|'o']<job_id>'.'<task_id>

              In order to change this default, the -e and -o options (see
              above) can be used together with the pseudo environment
              variables $HOME, $USER, $JOB_ID, $JOB_NAME, $HOSTNAME, and
              $SGE_TASK_ID.

              Note, that you can use the output redirection to divert the
              output of all tasks into the same file, but the result of this
              is undefined.

              -u username,... | -uall
                     Available for qalter only. Changes are only made on those
                     jobs which were submitted by users specified in the list
                     of usernames.  For managers it is possible to use the
                     qalter -uall command to modify all jobs of all users.

                     If you use the -u or -uall switch it is not permitted to
                     specify an additional job/task_id_list.

              -v variable[=value],...
                     Available for qsub, qsh, qrsh, qalter and qresub.

                     Defines or redefines the environment variables to be
                     exported to the execution context of the job.  If the -v
                     option is present Grid Engine will add the environment
                     variables defined as arguments to the switch and,
                     optionally, values of specified variables, to the
                     execution context of the job.

                     Qalter allows changing this option even while the job
                     executes. The modified parameter will only be in effect
                     after a restart or migration of the job, however.

              -verbose
                     Available only for qrsh and qmake(1).

                     Unlike qsh and qlogin, qrsh does not output any
                     informational messages while establishing the session
                     compliant with the standard rsh(1) and rlogin(1) system
                     calls.  If the option -verbose is set, qrsh behaves as
                     verbose as the qsh and qlogin commands and outputs
                     informations about the process of establishing the rsh(1)
                     or rlogin(1) session.

              -verify
                     Available for qsub, qsh, qrsh, qlogin, qalter and qresub.

                     Does not submit a job but prints information on the job
                     as being represented by the current command-line and all
                     pertinent external influences.

              -V     Available for qsub, qsh, qrsh, qalter and qresub.

                     Specifies that all environment variables active within
                     the qsub utility be exported to the context of the job.

              -w e|w|n|v
                     Available for qsub, qsh, qrsh, qlogin, qalter and qresub.

                     Specifies a validation level applied to the job to be
                     submitted (qsub, qlogin, and qsh) or the specified queued
                     job (qalter).  The information displayed indicates
                     whether the job possibly can be scheduled assuming an
                     empty system with no other jobs. Resource requests
                     exceeding the configured maximal thresholds or requesting
                     unavailable resource attributes are possible causes for
                     jobs to fail this validation.

                     The specifiers e, w, n and v define the following
                     validation modes:

                     `e'  error - jobs with invalid requests will be
                          rejected; the default for qrsh, qsh
                          and qlogin.
                     `w'  warning - only a warning will be displayed
                          for invalid requests.
                     `n'  none - switches off validation; the default for
                          qsub and qalter.
                     `v'  verify - does not submit the job but prints
                          extensive validation report.

                     Note, that the necessary checks are performance consuming
                     and hence the checking is switched off by default.

                     Note also, that the reasons for job requirements being
                     invalid with respect to resource availability of queues
                     are displayed in the "-w v" case using the format as
                     described for the qstat(1) -F option (see description of
                     Full Format in section OUTPUT FORMATS of the qstat(1)
                     manual page.

              job/task_id_list
                     Specified by the following form:

                         job_id[.task_range][,job_id[.task_range],...]

                     If present, the task_range restricts the effect of the
                     operation to the array job task range specified as suffix
                     to the job id (see the -t option to qsub(1) for further
                     details on array jobs).

                     The task range specifier has the form n[-m[:s]]. The
                     range may be a single number, a  simple range  of the
                     form n-m or a range with a step size.

                     Instead of job/task_id_list it is possible to use the
                     keyword 'all' to modify all jobs of the current user.

              scriptfile
                     Available for qsub only.

                     The job's scriptfile.  If not present or if the operand
                     is the single-character string '-', qsub reads the script
                     from standard input.

              script_args
                     Available for qsub and qalter only.

                     Arguments to the job. Not valid if the script is entered
                     from standard input.

                     Qalter allows changing this option even while the job
                     executes. The modified parameter will only be in effect
                     after a restart or migration of the job, however.

              xterm_args
                     Available for qsh only.

                     Arguments to the xterm(1) executable, as defined in the
                     configuration.  For details, refer to sge_conf(5)).

ENVIRONMENTAL VARIABLES
       SGE_ROOT       Specifies the location of the Grid Engine standard
                      configuration files.

       SGE_CELL       If set, specifies the default Grid Engine cell. To
                      address a Grid Engine cell qsub, qsh, qlogin or qalter
                      use (in the order of precedence):

                             The name of the cell specified in the environment
                             variable SGE_CELL, if it is set.

                             The name of the default cell, i.e. default.


       SGE_DEBUG_LEVEL
                      If set, specifies that debug information should be
                      written to stderr. In addition the level of detail in
                      which debug information is generated is defined.

       COMMD_PORT     If set, specifies the tcp port on which sge_commd(8) is
                      expected to listen for communication requests.  Most
                      installations will use a services map entry instead to
                      define that port.

       COMMD_HOST     If set, specifies the host on which the particular
                      sge_commd(8) to be used for Grid Engine communication of
                      the qsub, qsh, qlogin or qalter client resides.  Per
                      default the local host is used.

       DISPLAY        For qsh jobs the DISPLAY has to be specified at job
                      submission.  If the DISPLAY is not set by using the
                      -display or the -v switch, the contents of the DISPLAY
                      environment variable are used as default.

       In addition to those environment variables specified to be exported to
       the job via the -v or the -V option (see above) qsub, qsh, qrsh, and
       qlogin add the following variables with the indicated values to the
       variable list:


       SGE_O_HOME     the home directory of the submitting client.

       SGE_O_HOST     the name of the host on which the submitting client is
                      running.

       SGE_O_LOGNAME  the LOGNAME of the submitting client.

       SGE_O_MAIL     the MAIL of the submitting client. This is the mail
                      directory of the submitting client.

       SGE_O_PATH     the executable search path of the submitting client.

       SGE_O_SHELL    the SHELL of the submitting client.

       SGE_O_TZ       the time zone of the submitting client.

       SGE_O_WORKDIR  the absolute path of the current working directory of
                      the submitting client.

       For tasks of a tightly integrated parallel job, submitting client is
       the command submitting the task (qrsh -inherit).

       Furthermore, Grid Engine sets additional variables into the job's
       environment, as listed below.

       ARC

       SGE_ARCH       The Grid Engine architecture name of the node on which
                      the job is running. The name is compiled-in into the
                      sge_execd(8) binary.

       SGE_CKPT_ENV   Specifies the checkpointing environment (as selected
                      with the -ckpt option) under which a checkpointing job
                      executes. Only set for checkpointing jobs.

       SGE_CKPT_DIR   Only set for checkpointing jobs. Contains path ckpt_dir
                      (see checkpoint(5) ) of the checkpoint interface.

       SGE_STDERR_PATH
                      the pathname of the file to which the standard error
                      stream of the job is diverted. Commonly used for
                      enhancing the output with error messages from prolog,
                      epilog, parallel environment start/stop or checkpointing
                      scripts.

       SGE_STDOUT_PATH
                      the pathname of the file to which the standard output
                      stream of the job is diverted. Commonly used for
                      enhancing the output with messages from prolog, epilog,
                      parallel environment start/stop or checkpointing
                      scripts.

       SGE_JOB_SPOOL_DIR
                      The directory used by sge_shepherd(8) to store job
                      related data during job execution. This directory is
                      owned by root or by a Grid Engine administrative account
                      and commonly is not open for read or write access to
                      regular users.

       SGE_TASK_ID    The index number of the current array job task (see -t
                      option above). This is an unique number in each array
                      job and can be used to reference different input data
                      records, for example. This environment variable is set
                      to "undefined" for non-array jobs.

       ENVIRONMENT    The ENVIRONMENT variable is set to BATCH to identify
                      that the job is being executed under Grid Engine
                      control.

       HOME           The user's home directory path from the passwd(5) file.

       HOSTNAME       The hostname of the node on which the job is running.

       JOB_ID         A unique identifier assigned by the sge_qmaster(8) when
                      the job was submitted. The job ID is a decimal integer
                      in the range 1 to 99999.

       JOB_NAME       The job name, either `INTERACT' for interactive jobs or
                      built from the qsub script filename, a period, and the
                      digits of the job ID.  This default may be overwritten
                      by the -N.  option.

       LAST_HOST      The name of the preceding host in case of migration of a
                      checkpointing job.

       LOGNAME        The user's login name from the passwd(5) file.

       NHOSTS         The number of hosts in use by a parallel job.

       NQUEUES        The number of queues allocated for the job (always 1 for
                      serial jobs).

       NSLOTS         The number of queue slots in use by a parallel job.

       PATH           A default shell search path of:
                      /usr/local/bin:/usr/ucb:/bin:/usr/bin

       SGE_BINARY_PATH
                      The path where the Grid Engine binaries are installed.
                      The value is the concatenation of the cluster
                      configuration value binary_path and the architecture
                      name $SGE_ARCH

       PE             The parallel environment under which the job executes
                      (for parallel jobs only).

       PE_HOSTFILE    The path of a file containing the definition of the
                      virtual parallel machine assigned to a parallel job by
                      Grid Engine. See the description of the $pe_hostfile
                      parameter in sge_pe(5) for details on the format of this
                      file. The environment variable is only available for
                      parallel jobs.

       QUEUE          The name of the queue in which the job is running.

       REQUEST        Available for batch jobs only.

                      The request name of a job as specified with the -N
                      switch (see above) or taken as the name of the job
                      script file.

       RESTARTED      This variable is set to 1 if a job was restarted either
                      after a system crash or after a migration in case of a
                      checkpointing job. The variable has the value 0
                      otherwise.

       SHELL          The user's login shell from the passwd(5) file. Note:
                      This is not necessarily the shell in use for the job.

       TMPDIR         The absolute path to the job's temporary working
                      directory.

       TMP            The same as TMPDIR; provided for compatibility with NQS.

       TZ             The time zone variable imported from sge_execd(8) if
                      set.

       USER           The user's login name from the passwd(5) file.

RESTRICTIONS
       There is no controlling terminal for batch jobs under Grid Engine and
       any tests or actions on a controlling terminal will fail. If these
       operations are in your .login or .cshrc file, they will possibly cause
       your job to abort.

       Insert the following test before any commands that are not pertinent to
       batch jobs in your .login:

              if ( $?JOB_NAME) then
                     echo "Grid Engine spooled job"
                     exit 0
              endif

       Don't forget to set your shell's search path in your shell start-up
       before this code.

EXIT STATUS
       The following exit values are returned:

       0    Operation was executed successfully.

       25   It was not possible to register a new job according to the
            configured max_u_jobs or max_jobs limit. Additional information
            may be found in sge_conf(5)

       >0   Error occured.

EXAMPLES
       The following is the simplest form of a Grid Engine script file.

       =====================================================


       #!/bin/csh
          a.out


       =====================================================

       The next example is a more complex Grid Engine script.

       =====================================================

       #!/bin/csh

       # Which account to be charged cpu time
       #$ -A santa_claus

       # date-time to run, format [[CC]yy]MMDDhhmm[.SS]
       #$ -a 12241200

       # to run I want 6 or more parallel processes
       # under the PE pvm. the processes require
       # 128M of memory
       #$ -pe pvm 6- -l mem=128

       # If I run on dec_x put stderr in /tmp/foo, if I
       # run on sun_y, put stderr in /usr/me/foo
       #$ -e dec_x:/tmp/foo,sun_y:/usr/me/foo

       # Send mail to these users
       #$ -M santa@heaven,claus@heaven

       # Mail at beginning/end/on suspension
       #$ -m bes

       # Export these environmental variables
       #$ -v PVM_ROOT,FOOBAR=BAR

       # The job is located in the current
       # working directory.
       #$ -cwd

       a.out

       ==========================================================


FILES
       $REQUEST.oJID[.TASKID]      STDOUT of job #JID
       $REQUEST.eJID[.TASKID]      STDERR of job
       $REQUEST.poJID[.TASKID]     STDOUT of par. env. of job
       $REQUEST.peJID[.TASKID]     STDERR of par. env. of job
       $REQUEST.hostsJID[.TASKID]  hosts file of par. env. of job

       $cwd/.sge_aliases         cwd path aliases
       $cwd/.sge_request         cwd default request
       $HOME/.sge_aliases        user path aliases
       $HOME/.sge_request        user default request
       <sge_root>/<cell>/common/.sge_aliases
                                 cluster path aliases
       <sge_root>/<cell>/common/.sge_request
                                 cluster default request
       <sge_root>/<cell>/common/act_qmaster
                                 Grid Engine master host file

SEE ALSO
       sge_intro(1), qconf(1), qdel(1), qhold(1), qmod(1), qrls(1), qstat(1),
       accounting(5), sge_aliases(5), sge_conf(5), sge_request(5), sge_pe(5),
       complex(5).

COPYRIGHT
       If configured correspondingly, qrsh and qlogin contain portions of the
       rsh, rshd, elnet and telnetd code copyrighted by The Regents of the
       University of California.  Therefore, the following note applies with
       respect to qrsh and qlogin: This product includes software developed by
       the University of California, Berkeley and its contributors.

       See sge_intro(1) as well as the information provided in
       <sge_root>/3rd_party/qrsh and <sge_root>/3rd_party/qlogin for a
       statement of further rights and permissions.





GE 5.3                   $Date: 2004/03/25 10:53:48 $                SUBMIT(1)