Users with persistent
gen_f functions may notice that the persistent workers
are still automatically assigned resources. This can be wasteful if those workers
gen_f functions in-place (i.e., they do not use the Executor
to submit applications to allocated nodes). Suppose the user is using the
parse_args() function and runs:
python run_ensemble_persistent_gen.py --comms local --nworkers 3
If three nodes are available in the node allocation, the result may look like the following.
To avoid the the wasted node above, add an extra worker:
python run_ensemble_persistent_gen.py --comms local --nworkers 4
and in the calling script (run_ensemble_persistent_gen.py), explicitly set the number of resource sets to the number of workers that will be running simulations.
nworkers, is_manager, libE_specs, _ = parse_args() libE_specs["num_resource_sets"] = nworkers - 1
num_resource_sets option is used, libEnsemble will use the dynamic
resource scheduler, and any worker may assign work to any node. This works well
for most users.
Optional: An alternative way to express the above would be to use the command line:
python run_ensemble_persistent_gen.py --comms local --nsim_workers 3
This would automatically set the
num_resource_sets option and add a single
worker for the persistent generator - a common use-case.
In general, the number of resource sets should be set to enable the maximum concurrency desired by the ensemble, taking into account generators and simulators.
Users can set generator resources using the libE_specs options
gen_num_gpus, which take integer values.
gen_num_gpus is set, then the number of processors is set to match.
To vary generator resources,
persis_info settings can be used in allocation
functions before calling the
gen_work support function. This takes the
same options (
Alternatively, the setting
persis_info["gen_resources"] can also be set to
a number of resource sets.
The available nodes are always divided by the number of resource sets, and there may be multiple nodes or a partition of a node in each resource set. If the split is uneven, resource sets are not split between nodes. For example, if there are two nodes and five resource sets, one node will have three resource sets, and the other will have two.
Placing zero-resource functions on a fixed worker
If the generator must always be on worker one, then instead of using
num_resource_sets, use the
zero_resource_workers libE_specs option:
libE_specs["zero_resource_workers"] = 
in the calling script and worker one will not be allocated resources. In general,
set the parameter
zero_resource_workers to a list of worker IDs that should not
have resources assigned.
This approach can be useful if running in distributed mode.
The use of the
zero_resource_workers libE_specs option must be supported by
the allocation function, see start_only_persistent)