calc_status
The calc_status
is similar to an exit code, and is either a built-in integer
attribute with a named (enumerated) value and corresponding description, or a
user-specified string. They are determined within user functions to capture the
status of a calculation, and returned to the manager and printed to the libE_stats.txt
file.
Only the status values FINISHED_PERSISTENT_SIM_TAG
and FINISHED_PERSISTENT_GEN_TAG
are currently used by the manager, but others can still provide a useful summary in libE_stats.txt
.
The user is responsible for determining the status of a user function instance, since
a given instance could include multiple application runs of mixed rates of success.
The calc_status
is the third optional return value from a user function.
Built-in codes are available in the libensemble.message_numbers
module, but
users are also free to return any custom string.
Example of calc_status
used along with Executor in sim_f:
1 from libensemble.message_numbers import WORKER_DONE, WORKER_KILL, TASK_FAILED
2
3 task = exctr.submit(calc_type="sim", num_procs=cores, wait_on_start=True)
4 calc_status = UNSET_TAG
5 poll_interval = 1 # secs
6 while not task.finished:
7 if task.runtime > time_limit:
8 task.kill() # Timeout
9 else:
10 time.sleep(poll_interval)
11 task.poll()
12
13 if task.finished:
14 if task.state == "FINISHED":
15 print("Task {} completed".format(task.name))
16 calc_status = WORKER_DONE
17 elif task.state == "FAILED":
18 print("Warning: Task {} failed: Error code {}".format(task.name, task.errcode))
19 calc_status = TASK_FAILED
20 elif task.state == "USER_KILLED":
21 print("Warning: Task {} has been killed".format(task.name))
22 calc_status = WORKER_KILL
23 else:
24 print("Warning: Task {} in unknown state {}. Error code {}".format(task.name, task.state, task.errcode))
25
26 outspecs = sim_specs["out"]
27 output = np.zeros(1, dtype=outspecs)
28 output["energy"][0] = final_energy
29
30 return output, persis_info, calc_status
Example of defining and returning a custom calc_status
if the built-in values
are insufficient:
1 from libensemble.message_numbers import WORKER_DONE, TASK_FAILED
2
3 task = exctr.submit(calc_type="sim", num_procs=cores, wait_on_start=True)
4
5 task.wait(timeout=60)
6
7 file_output = read_task_output(task)
8 if task.errcode == 0:
9 if "fail" in file_output:
10 calc_status = "Task failed successfully?"
11 else:
12 calc_status = WORKER_DONE
13 else:
14 calc_status = TASK_FAILED
15
16 outspecs = sim_specs["out"]
17 output = np.zeros(1, dtype=outspecs)
18 output["energy"][0] = final_energy
19
20 return output, persis_info, calc_status
See forces_simf.py for a complete example.
See uniform_or_localopt.py for an example of using FINISHED_PERSISTENT_GEN_TAG
.
Available values:
FINISHED_PERSISTENT_SIM_TAG = 11 # tells manager sim_f done persistent mode
FINISHED_PERSISTENT_GEN_TAG = 12 # tells manager gen_f done persistent mode
MAN_SIGNAL_FINISH = 20 # Kill tasks and shutdown worker
MAN_SIGNAL_KILL = 21 # Kill running task - but don't stop worker
WORKER_KILL = 30 # Worker kills not covered by a more specific case
WORKER_KILL_ON_ERR = 31 # Worker killed due to an error in results
WORKER_KILL_ON_TIMEOUT = 32 # Worker killed on timeout
TASK_FAILED = 33 # Calc had tasks that failed
WORKER_DONE = 34 # Calculation was successful
The corresponding messages printed to the libE_stats.txt
file:
calc_status_strings = {
UNSET_TAG: "Not set",
FINISHED_PERSISTENT_SIM_TAG: "Persis sim finished",
FINISHED_PERSISTENT_GEN_TAG: "Persis gen finished",
MAN_SIGNAL_FINISH: "Manager killed on finish",
MAN_SIGNAL_KILL: "Manager killed task",
WORKER_KILL_ON_ERR: "Worker killed task on Error",
WORKER_KILL_ON_TIMEOUT: "Worker killed task on Timeout",
WORKER_KILL: "Worker killed",
TASK_FAILED: "Task Failed",
WORKER_DONE: "Completed",
CALC_EXCEPTION: "Exception occurred",
None: "Unknown Status",
}
Example segment of libE_stats.txt:
Worker 1: Calc 0: gen Time: 0.00 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:43 Status: Not set
Worker 1: Calc 1: sim Time: 4.41 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:48 Status: Worker killed
Worker 2: Calc 0: sim Time: 5.42 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:49 Status: Completed
Worker 1: Calc 2: sim Time: 2.41 Start: 2019-11-19 18:53:48 End: 2019-11-19 18:53:50 Status: Worker killed
Worker 2: Calc 1: sim Time: 2.41 Start: 2019-11-19 18:53:49 End: 2019-11-19 18:53:51 Status: Worker killed
Worker 1: Calc 3: sim Time: 4.41 Start: 2019-11-19 18:53:50 End: 2019-11-19 18:53:55 Status: Completed
Worker 2: Calc 2: sim Time: 4.41 Start: 2019-11-19 18:53:51 End: 2019-11-19 18:53:56 Status: Completed
Worker 1: Calc 4: sim Time: 4.41 Start: 2019-11-19 18:53:55 End: 2019-11-19 18:53:59 Status: Completed
Worker 2: Calc 3: sim Time: 4.41 Start: 2019-11-19 18:53:56 End: 2019-11-19 18:54:00 Status: Completed