Worker Resources Module

The worker resources module can be interrogated by the user function on a worker to obtain the following attributes. The convenience functions below can also be used.

class resources.worker_resources.WorkerResources(num_workers, resources, workerID)

Bases: libensemble.resources.rset_resources.RSetResources

Provide system resources per worker to libEnsemble and executor.

Object Attributes:

Some of these attributes may be updated as the ensemble progresses.

rsets below is used to abbreviate resource sets.

Variables
  • workerID (int) – workerID for this worker.

  • local_nodelist (list) – A list of all nodes assigned to this worker.

  • rset_team (list) – List of rset IDs currently assigned to this worker.

  • num_rsets (int) – The number of resource sets assigned to this worker.

  • slots (dict) – A dictionary with a list of slot IDs for each node.

  • even_slots (bool) – Determines if the slots evenly divide amongst nodes.

  • slot_count (int) – The number of slots per node if even_slots is True, else None.

  • slots_on_node (list) – A list of slots IDs if even_slots is True, else None.

  • local_node_count (int) – The number of nodes available to this worker (rounded up to whole number).

  • rsets_per_node (int) – The number of rsets per node (if a rset > 1 node, will be 1).

The worker_resources attribtues can be queried, and convenience functions called, via the resources class attribute. For example:

With resources imported:

from libensemble.resources.resources import Resources

A user function (sim/gen) may do:

resources = Resources.resources.worker_resources
num_nodes = resources.local_node_count
cores_per_node = resources.slot_count  # One CPU per GPU
resources.set_env_to_slots("CUDA_VISIBLE_DEVICES")  # Use convenience function.

Note that slots are resource sets enumerated on a node (starting with zero). If a resource set has more than one node, then each node is considered to have slot zero.

If even_slots is True, then the attributes slot_count and slots_on_node can be used for simplicity, Otherwise, the slots dictionary can be used to get information for each node.

get_slots_as_string(multiplier=1, delimiter=',')

Returns list of slots as a string

Parameters
  • multiplier – Optional int. Assume this many items per slot.

  • delimiter – Optional int. Delimiter for output string.

set_env_to_slots(env_var, multiplier=1, delimiter=',')

Sets the given environment variable to slots

Parameters
  • env_var – String. Name of environment variable to set.

  • multiplier – Optional int. Assume this many items per slot.

  • delimiter – Optional int. Delimiter for output string.

Example usage in a sim function:

With resources imported:

from libensemble.resources.resources import Resources

Obtain worker resources:

resources = Resources.resources.worker_resources
resources.set_env_to_slots("CUDA_VISIBLE_DEVICES")