Scheduler Module

The scheduler is called within the scope of the allocation function, usually via the alloc_support module function assign_resources() (either called directly or via sim_work() or gen_work()), which is a wrapper for the main scheduler function assign_resources().

The alloc_support module allows users to supply an alternative scheduler that fits this interface. This could be achieved, for example, by inheriting the built-in scheduler and making modifications.

Options can also be provided to the scheduler though the libE_specs["scheduler_opts"] dictionary.

class resources.scheduler.ResourceScheduler(user_resources=None, sched_opts={})

Calculates and returns resource set ids from a dictionary of resource sets by group. The available resource sets are read initially from the resources module or from a resources object passed in.

Resource sets are locally provisioned to work items by a call to the assign_resources function, and a cache of available resource sets is maintained for the life of the object (usually corresponding to one call of the allocation function). Note that work item resources are formally assigned to workers only when a work item is sent to the worker.

__init__(user_resources=None, sched_opts={})

Initiate a ResourceScheduler object

Parameters:

  • user_resources (Resources, optional) – A resources object. If present overrides the class variable.

  • sched_opts (dict, optional) – A dictionary of scheduler options. Passed via libE_specs["scheduler_opts"]

The supported fields for sched_opts are:

"split2fit" [Boolean]:
    Try to split resource sets across more nodes if space is not currently
    available on the minimum node count required. Allows more efficient
    scheduling.
    Default: True

"match_slots" [Boolean]:
    When splitting resource sets across multiple nodes, slot IDs must match.
    Useful if setting an environment variable such as ``CUDA_VISIBLE_DEVICES``
    to specific slots, which should match over multiple nodes.
    Default: True
assign_resources(rsets_req, use_gpus=None, user_params=[])

Schedule resource sets to a work item if possible.

If the resources required are less than one node, they will be allocated to the smallest available sufficient slot.

If the resources required are more than one node, then the scheduler will attempt to find an even split. If no even split is possible, then enough additional resource sets will be assigned to enable an even split.

Returns a list of resource set IDs or raises an exception (either InsufficientResourcesError or InsufficientFreeResources).