RSET Resources Module

class rset_resources.RSetResources(num_workers, resources)

A class that creates a fixed mapping of resource sets to the available resources.

Object Attributes:

These are set on initialization and include inherited. rsets below is used to abbreviate resource sets.

Variables:

  • num_workers (int) – Total number of workers

  • num_workers_2assign2 (int) – The number of workers that will be assigned resource sets.

  • total_num_rsets (int) – The total number of resource sets.

  • split_list (list) – A list of lists, where each element is the list of nodes for a given rset.

  • local_rsets_list (list) – A list over rsets, where each element is the number of rsets that share the node.

  • rsets_per_node (int) – The number of rsets per node (if an rset > 1 node, this will be 1)

__init__(num_workers, resources)

Initializes a new RSetResources instance

Determines the compute resources available for each resource set.

Unless resource sets is set explicitly, the number of resource sets is the number of workers, excluding any workers defined as zero resource workers.

Parameters:

  • num_workers (int) – The total number of workers

  • resources (Resources) – A Resources object containing global nodelist and intranode information

static get_group_list(split_list, gpus_per_node=0)

Return lists of group ids and slot IDs by resource set

static best_split(a, n)

Creates the most even split of list a into n parts and return list of lists

static get_rsets_on_a_node(num_rsets, resources)

Returns the number of resource sets that can be placed on each node

If there are more nodes than resource sets, returns 1.

static get_workers2assign2(num_workers, resources)

Returns workers to assign resources to

static even_assignment(nnodes, nworkers)

Returns True if workers are evenly distributed to nodes, else False

static expand_list(nnodes, nworkers, nodelist)

Duplicates each element of nodelist to best map workers to nodes.

Returns node list with duplicates, and a list of local (on-node) worker counts, both indexed by worker.

static get_split_list(num_rsets, resources)

Returns a list of lists for each worker

Assumes that self.global_nodelist has been calculated (in __init__).

static get_partitioned_nodelist(num_rsets, resources)

Returns lists of nodes available to all resource sets

Assumes that self.global_nodelist has been calculated (in __init__). Also self.global_nodelist will have already removed non-application nodes