Base Executor - Local apps
See the Executor APIs for optional arguments.
Only for running local serial-launched applications. To run MPI applications and use detected resources, use the MPIExecutor
- class libensemble.executors.executor.Executor
The executor can create, poll and kill runnable tasks
Class Attributes:
- Variables
Executor – executor: The executor object is stored here and can be retrieved in user functions.
- __init__()
Instantiate a new Executor instance.
- Returns
A new Executor object is created. This is typically created in the user calling script.
- Return type
- register_app(full_path, app_name=None, calc_type=None, desc=None, precedent='')
Registers a user application to libEnsemble.
The
full_path
of the application must be supplied. Eitherapp_name
orcalc_type
can be used to identify the application in user scripts (in the submit function).app_name
is recommended.- Parameters
full_path (str) – The full path of the user application to be registered
app_name (str, Optional) – Name to identify this application.
calc_type (str, Optional) – Calculation type: Set this application as the default ‘sim’ or ‘gen’ function.
desc (str, Optional) – Description of this application
precedent (str, Optional) – Any str that should directly precede the application full path.
- Return type
None
- manager_poll()
Polls for a manager signal
The executor manager_signal attribute will be updated.
- Return type
int
- manager_kill_received()
Return True if received kill signal from the manager
- Return type
bool
- polling_loop(task, timeout=None, delay=0.1, poll_manager=False)
Optional, blocking, generic task status polling loop. Operates until the task finishes, times out, or is Optionally killed via a manager signal. On completion, returns a presumptive calc_status integer. Potentially useful for running an application via the Executor until it stops without monitoring its intermediate output.
- Parameters
task (object) – a Task object returned by the executor on submission
timeout (int, Optional) – Maximum number of seconds for the polling loop to run. Tasks that run longer than this limit are killed. Default: No timeout
delay (int, Optional) – Sleep duration between polling loop iterations. Default: 0.1 seconds
poll_manager (bool, Optional) – Whether to also poll the manager for ‘finish’ or ‘kill’ signals. If detected, the task is killed. Default: False.
- Returns
calc_status – presumptive integer attribute describing the final status of a launched task
- Return type
int
- submit(calc_type=None, app_name=None, app_args=None, stdout=None, stderr=None, dry_run=False, wait_on_start=False, env_script=None)
Create a new task and run as a local serial subprocess.
The created
task
object is returned.- Parameters
calc_type (str, Optional) – The calculation type: ‘sim’ or ‘gen’ Only used if app_name is not supplied. Uses default sim or gen application.
app_name (str, Optional) – The application name. Must be supplied if calc_type is not.
app_args (str, Optional) – A str of the application arguments to be added to task submit command line
stdout (str, Optional) – A standard output filename
stderr (str, Optional) – A standard error filename
dry_run (bool, Optional) – Whether this is a dry_run - no task will be launched; instead runline is printed to logger (at INFO level)
wait_on_start (bool, Optional) – Whether to wait for task to be polled as RUNNING (or other active/end state) before continuing
env_script (str, Optional) – The full path of a shell script to set up the environment for the launched task. This will be run in the subprocess, and not affect the worker environment. The script should start with a shebang.
- Returns
task – The launched task object
- Return type
Tasks are created and returned by the Executor’s submit()
. Tasks
can be polled, killed, and waited on with the respective poll
, kill
, and wait
functions.
Task information can be queried through instance attributes and query functions.
- class libensemble.executors.executor.Task(app=None, app_args=None, workdir=None, stdout=None, stderr=None, workerid=None, dry_run=False)
Manages the creation, configuration and status of a launchable task
- workdir_exists()
Returns true if the task’s workdir exists
- Return type
Optional[bool]
- file_exists_in_workdir(filename)
Returns true if the named file exists in the task’s workdir
- Parameters
filename (str) –
- Return type
bool
- read_file_in_workdir(filename)
Opens and reads the named file in the task’s workdir
- Parameters
filename (str) –
- Return type
str
- stdout_exists()
Returns true if the task’s stdout file exists in the workdir
- Return type
bool
- read_stdout()
Opens and reads the task’s stdout file in the task’s workdir
- Return type
str
- stderr_exists()
Returns true if the task’s stderr file exists in the workdir
- Return type
bool
- read_stderr()
Opens and reads the task’s stderr file in the task’s workdir
- Return type
str
- poll()
Polls and updates the status attributes of the task
- Return type
None
- wait(timeout=None)
Waits on completion of the task or raises TimeoutExpired exception
Status attributes of task are updated on completion.
- Parameters
timeout (int or float, Optional) – Time in seconds after which a TimeoutExpired exception is raised. If not set, then simply waits until completion. Note that the task is not automatically killed on timeout.
- Return type
None
- result(timeout=None)
Wrapper for task.wait() that also returns the task’s status on completion.
- Parameters
timeout (int or float, Optional) – Time in seconds after which a TimeoutExpired exception is raised. If not set, then simply waits until completion. Note that the task is not automatically killed on timeout.
- Return type
str
- exception(timeout=None)
Wrapper for task.wait() that instead returns the task’s error code on completion.
- Parameters
timeout (int or float, Optional) – Time in seconds after which a TimeoutExpired exception is raised. If not set, then simply waits until completion. Note that the task is not automatically killed on timeout.
- running()
Return
True
if task is currently running.- Return type
bool
- done()
Return
True
if task is finished.- Return type
bool
- kill(wait_time=60)
Kills or cancels the supplied task
- Parameters
wait_time (int, Optional) – Time in seconds to wait for termination between sending SIGTERM and a SIGKILL signals.
- Return type
None
Sends SIGTERM, waits for a period of <wait_time> for graceful termination, then sends a hard kill with SIGKILL. If <wait_time> is 0, we go immediately to SIGKILL; if <wait_time> is none, we never do a SIGKILL.
- cancel()
Wrapper for task.kill() without waiting
- Return type
None
- cancelled()
Return
`True
if task successfully cancelled.- Return type
bool
Note
These should not be set directly. Tasks are launched by the Executor, and task information can be queried through the task attributes below and the query functions.
- task.state
(string) The task status. One of (“UNKNOWN”|”CREATED”|”WAITING”|”RUNNING”|”FINISHED”|”USER_KILLED”|”FAILED”)
- task.process
(process obj) The process object used by the underlying process manager (e.g., return value of subprocess.Popen).
- task.errcode
(int) The error code (or return code) used by the underlying process manager.
- task.finished
(boolean) True means task has finished running - not whether it was successful.
- task.success
(boolean) Did task complete successfully (e.g., the return code is zero)?
- task.runtime
(int) Time in seconds that task has been running.
- task.submit_time
(int) Time since epoch that task was submitted.
- task.total_time
(int) Total time from task submission to completion (only available when task is finished).
Run configuration attributes - some will be autogenerated:
- task.workdir
(string) Work directory for the task
- task.name
(string) Name of task - autogenerated
- task.app
(app obj) Use application/executable, registered using exctr.register_app
- task.app_args
(string) Application arguments as a string
- task.stdout
(string) Name of file where the standard output of the task is written (in task.workdir)
- task.stderr
(string) Name of file where the standard error of the task is written (in task.workdir)
- task.dry_run
(boolean) True if task corresponds to dry run (no actual submission)
- task.runline
(string) Complete, parameterized command to be subprocessed to launch app