complex

COMPLEX(5)         Grid Engine Enterprise Edition File Formats        COMPLEX(5)



NAME
       complex - Grid Engine Enterprise Edition complexes configuration file
       format

DESCRIPTION
       Complex reflects the format of the Grid Engine Enterprise Edition
       complexes configuration.  The definition of complexes provides all
       pertinent information concerning the resource attributes a user may
       request for a Grid Engine Enterprise Edition job via the qsub(1) -l
       option and for the interpretation of these parameters within the Grid
       Engine Enterprise Edition system.

       The complexes configuration files should not be accessed directly.  In
       order to add or modify complexes, the qconf(1) options -Ac, -ac, -Mc and
       -mc should be used instead.  While the -Ac and -Mc options take a complex
       configuration file as an argument, the -ac and -mc options bring up an
       editor filled in with a template complex configuration or the
       configuration of an existing complex.

       The Grid Engine Enterprise Edition complexes object integrates 4
       different types of complexes:

   The Queue Complex
       It is referenced by the special name "queue".

       In its default form it contains a selection of parameters in the queue
       configuration as defined in queue_conf(5).  The queue configuration
       parameters being requestable for a job by the user in principal are:

              qname
              hostname
              notify
              calendar
              min_cpu_interval
              tmpdir
              seq_no
              s_rt
              h_rt
              s_cpu
              h_cpu
              s_data
              h_data
              s_stack
              h_stack
              s_core
              h_core
              s_rss
              h_rss
       The queue complex can be extended if further attributes are intended to
       be available for each queue. The queue complex defines the
       characteristics (such as the data type) of the attributes it contains. A
       value setting for the queue complex attributes is defined by the queue
       configuration for each queue in case of the standard parameters enlisted
       above, or by the complex_values entry in the queue configuration (see
       queue_conf(5) for details) if a parameter has been added to the default
       queue complex. If no definition for the value in the complex_values entry
       of the queue configuration is given in the latter case, the value is set
       as defined by the value field described below.

   The Host Complex
       It is referenced by the special name "host" and contains the
       characteristics definition of all attributes which are intended to be
       managed on a host basis. The standard set of host related attributes
       consists of two categories, but it may be enhanced like the queue complex
       as described above. The first category is built by several queue
       configuration attributes which are particularly suitable to be managed on
       a host basis. These attributes are:

              slots
              s_vmem
              h_vmem
              s_fsize
              h_fsize
       (please refer to queue_conf(5) for details).

       Note: Defining these attributes in the host complex is no contradiction
       to having them also in the queue configuration. It allows maintaining the
       corresponding resources on a host level and at the same time on a queue
       level. Total virtual free memory (h_vmem) can be managed for a host, for
       example, and a subset of the total amount can be associated with a queue
       on that host.

       The second attribute category in the standard host complex are the
       default load values Every sge_execd(8) periodically reports load to
       sge_qmaster(8).  The reported load values are either the standard Grid
       Engine Enterprise Edition load values such as the CPU load average (see
       uptime(1)) or load values defined by the Grid Engine Enterprise Edition
       administration (see the load_sensor parameter in the cluster
       configuration sge_conf(5) and the Grid Engine Enterprise Edition
       Installation and Administration Guide for details).  The characteristics
       definition for the standard load values is part of the default host
       complex, while administrator defined load values require extension of the
       host complex. Please refer to the file <sge_root>/doc/load_parameters.asc
       for detailed information on the standard set of load values.

       The host complex commonly is not only extended to include non-standard
       load parameters, but also to manage host related resources such as the
       number of software licenses being assigned to a host or the available
       disk space on a host local filesystem.

       A concrete value for a particular host complex attribute is determined by
       either an associated queue configuration in the case of the queue
       configuration derived attributes, a reported load value or the explicit
       definition of a value in the complex_values entry of the corresponding
       host configuration (see host_conf(5)).  If none of the above is available
       (e.g. the value is supposed to be a load parameter, but sge_execd(8) does
       not report a load value for it), the value field described below is used.

   The Global Complex
       It is referenced by the special name "global".

       The entries configured in the global complex refer to cluster wide
       resource attributes, such as the number of available "floating" licenses
       of a particular software or the free disk space on a network wide
       available filesystem. Global resource attributes can also be associated
       with load reports, if the corresponding load report contains the "GLOBAL"
       identifier (see the corresponding section in the Grid Engine Enterprise
       Edition Installation and Administration Guide for details). Global load
       values can be reported from any host in the cluster. There are no global
       load values reported by Grid Engine Enterprise Edition by default and
       hence there is no default global complex configuration.

       Concrete values for global complex attributes are either determined by
       global load reports or by explicit definition in the complex_values
       parameter of the "global" host configuration (see host_conf(5)).  If none
       of both is present (e.g. a load value has not yet been reported) the
       value field described below is used.

   User Defined Complexes
       By setting up user defined complexes the Grid Engine Enterprise Edition
       administration has the ability to extend the set of attributes managed by
       Grid Engine Enterprise Edition while restricting the influence of those
       attributes to particular queues and/or hosts. A user complex is just a
       named collection of attributes and the corresponding definition as to how
       these attributes are to be handled by Grid Engine Enterprise Edition.
       One or more of these user defined complexes can be attached to a queue
       and/ or host via the complex_list queue and host configuration parameter
       (see queue_conf(5) and host_conf(5)).  The attributes defined in all
       assigned complexes become available to the queue and the host
       respectively in addition to the default complex attributes.

       Concrete values for user defined complexes have to be set by the
       complex_values parameter in the queue and host configuration or otherwise
       the value field described below is used.

FORMAT
       The principal format of a complex configuration is that of a tabulated
       list. Each line starting with a '#' character is a comment line. Each
       line despite comment lines define one element of the complex. A element
       definition line consists of the following 6 column entries per line (in
       the order of appearance):

   name
       The name of the complex element to be used to request this attribute for
       a job in the qsub(1) -l option. An attribute name may appear only once
       across all complexes, i.e. the complex attribute definition is unique.

   shortcut
       A shortcut for name which may also be used to request this attribute for
       a job in the qsub(1) -l option. An attribute shortcut may appear only
       once across all complexes, so as to avoid the possibility of ambiguous
       complex attribute references.

   type
       This setting determines how the corresponding values are to be treated
       Grid Engine Enterprise Edition internally in case of comparisons or in
       case of load scaling for the load complex entries:

       ·  With INT only raw integers are allowed.

       ·  With DOUBLE floating point numbers in double precision (decimal and
          scientific notation) can be specified.

       ·  With TIME time specifiers are allowed. Refer to queue_conf(5) for a
          format description.

       ·  With MEMORY memory size specifiers are allowed. Refer to queue_conf(5)
          for a format description.

       ·  With BOOL the strings TRUE and FALSE are allowed. When used in a load
          formula (refer to sched_conf(5) ) TRUE and FALSE get mapped into '1'
          and '0'.

       ·  With STRING all strings are allowed and strcmp(3) is used for
          comparisons.

       ·  CSTRING is like STRING except comparisons are case insensitive.

       ·  HOST is like CSTRING but the string must be a valid hostname.

   value
       The value field is a pre-defined value setting for an attribute, which
       only has an effect if it is not overwritten while attempting to determine
       a concrete value for the attribute with respect to a queue, a host or the
       Grid Engine Enterprise Edition cluster. The value field can be
       overwritten by

       ·  the queue configuration values of a referenced queue.

       ·  host specific and cluster related load values.

       ·  explicit specification of a value via the complex_values parameter in
          the queue or host configuration (see queue_conf(5) and host_conf(5)
          for details.

       If none of above is applicable, value is set for the attribute.

   relop
       The relation operator.  The relation operator is used when the value
       requested by the user for this parameter is compared against the
       corresponding value configured for the considered queues. If the result
       of the comparison is false, the job cannot run in this queue. Possible
       relation operators are "==", "<", ">", "<=" and ">=". The only valid
       operator for string type attributes is "==".

   requestable
       The entry can be used in a qsub(1) resource request if this field is set
       to 'y' or 'yes'.  If set to 'n' or 'no' this entry cannot be used by a
       user in order to request a queue or a class of queues.  If the entry is
       set to 'forced' or 'f' the attribute has to be requested by a job or it
       is rejected.

   consumable
       The consumable parameter can be set to either 'yes' ('y' abbreviated) or
       'no' ('n'). It can be set to 'yes' only for numeric attributes (INT,
       MEMORY, TIME - see type above). If set to 'yes' the consumption of the
       corresponding resource can be managed by Grid Engine Enterprise Edition
       internal bookkeeping. In this case Grid Engine Enterprise Edition
       accounts for the consumption of this resource for all running jobs and
       ensures that jobs are only dispatched if the Grid Engine Enterprise
       Edition internal bookkeeping indicates enough available consumable
       resources. Consumables are an efficient means to manage limited resources
       such a available memory, free space on a file system, network bandwidth
       or floating software licenses.

       Consumables can be combined with default or user defined load parameters
       (see sge_conf(5) and host_conf(5)), i.e. load values can be reported for
       consumable attributes or the consumable flag can be set for load
       attributes. The Grid Engine Enterprise Edition consumable resource
       management takes both the load (measuring availability of the resource)
       and the internal bookkeeping into account in this case, and makes sure
       that neither of both exceeds a given limit.

       To enable consumable resource management the basic availability of a
       resource has to be defined. This can be done on a cluster global, per
       host and per queue basis while these categories may supersede each other
       in the given order (i.e. a host can restrict availability of a cluster
       resource and a queue can restrict host and cluster resources). The
       definition of resource availability is performed with the complex_values
       entry in host_conf(5) and queue_conf(5).  The complex_values definition
       of the "global" host specifies cluster global consumable settings. To
       each consumable complex attribute in a complex_values list a value is
       assigned which denotes the maximum available amount for that resource.
       The internal bookkeeping will subtract from this total the assumed
       resource consumption by all running jobs as expressed through the jobs'
       resource requests.

       Note: Jobs can be forced to request a resource and thus to specify their
       assumed consumption via the 'force' value of the requestable parameter
       (see above).

       Note also: A default resource consumption value can be pre-defined by the
       administrator for consumable attributes not explicitly requested by the
       job (see the default parameter below). This is meaningful only if
       requesting the attribute is not enforced as explained above.

       See the Grid Engine Enterprise Edition Installation and Administration
       Guide for examples on the usage of the consumable resources facility.

   default
       Meaningful only for consumable complex attributes (see consumable
       parameter above). Grid Engine Enterprise Edition assumes the resource
       amount denoted in the default parameter implicitly to be consumed by jobs
       being dispatched to a host or queue managing the consumable attribute.
       Jobs explicitly requesting the attribute via the -l option to qsub(1)
       override this default value.

SEE ALSO
       sge_intro(1), qconf(1), qsub(1), uptime(1), host_conf(5), queue_conf(5),
       sge_execd(8), sge_qmaster(8), sge_schedd(8),
       Grid Engine Enterprise Edition Installation and Administration Guide.

COPYRIGHT
       See sge_intro(1) for a full statement of rights and permissions.



GEEE 5.3                  $Date: 2001/11/23 15:20:38 $                COMPLEX(5)