calc_status

The calc_status is an integer attribute with named (enumerated) values and a corresponding description that can be used in sim_f or gen_f functions to capture the status of a calculation. This is returned to the manager and printed to the libE_stats.txt file. Only the status values FINISHED_PERSISTENT_SIM_TAG and FINISHED_PERSISTENT_GEN_TAG are currently used by the manager, but others can still provide a useful summary in libE_stats.txt. The user determines the status of the calculation, since it could include multiple application runs. It can be added as a third return variable in sim_f or gen_f functions. The calc_status codes are in the libensemble.message_numbers module.

Example of calc_status used along with Executor in sim_f:

 1  from libensemble.message_numbers import WORKER_DONE, WORKER_KILL, TASK_FAILED
 2
 3  task = exctr.submit(calc_type='sim', num_procs=cores, wait_on_start=True)
 4  calc_status = UNSET_TAG
 5  poll_interval = 1  # secs
 6  while(not task.finished):
 7      if task.runtime > time_limit:
 8          task.kill()  # Timeout
 9      else:
10          time.sleep(poll_interval)
11          task.poll()
12
13  if task.finished:
14      if task.state == 'FINISHED':
15          print("Task {} completed".format(task.name))
16          calc_status = WORKER_DONE
17      elif task.state == 'FAILED':
18          print("Warning: Task {} failed: Error code {}".format(task.name, task.errcode))
19          calc_status = TASK_FAILED
20      elif task.state == 'USER_KILLED':
21          print("Warning: Task {} has been killed".format(task.name))
22          calc_status = WORKER_KILL
23      else:
24          print("Warning: Task {} in unknown state {}. Error code {}".format(task.name, task.state, task.errcode))
25
26  outspecs = sim_specs['out']
27  output = np.zeros(1, dtype=outspecs)
28  output['energy'][0] = final_energy
29
30  return output, persis_info, calc_status

See forces_simf.py for a complete example. See uniform_or_localopt.py for an example of using FINISHED_PERSISTENT_GEN_TAG.

Available values:

# CALC STATUS/SIGNAL FLAGS
FINISHED_PERSISTENT_SIM_TAG = 11  # tells manager sim_f done persistent mode
FINISHED_PERSISTENT_GEN_TAG = 12  # tells manager gen_f done persistent mode
MAN_SIGNAL_FINISH = 20            # Kill tasks and shutdown worker
MAN_SIGNAL_KILL = 21              # Kill running task - but don't stop worker
WORKER_KILL = 30                  # Worker kills not covered by a more specific case
WORKER_KILL_ON_ERR = 31           # Worker killed due to an error in results
WORKER_KILL_ON_TIMEOUT = 32       # Worker killed on timeout
TASK_FAILED = 33                  # Calc had tasks that failed
WORKER_DONE = 34                  # Calculation was successful

The corresponding messages printed to the libE_stats.txt file:

calc_status_strings = {
    FINISHED_PERSISTENT_SIM_TAG: "Persis sim finished",
    FINISHED_PERSISTENT_GEN_TAG: "Persis gen finished",
    MAN_SIGNAL_FINISH: "Manager killed on finish",
    MAN_SIGNAL_KILL: "Manager killed task",
    WORKER_KILL_ON_ERR: "Worker killed task on Error",
    WORKER_KILL_ON_TIMEOUT: "Worker killed task on Timeout",
    WORKER_KILL: "Worker killed",
    TASK_FAILED: "Task Failed",
    WORKER_DONE: "Completed",
    CALC_EXCEPTION: "Exception occurred",
    None: "Unknown Status"
}

Example segment of libE_stats.txt:

Worker     1: Calc     0: gen Time: 0.00 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:43 Status: Not set
Worker     1: Calc     1: sim Time: 4.41 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:48 Status: Worker killed
Worker     2: Calc     0: sim Time: 5.42 Start: 2019-11-19 18:53:43 End: 2019-11-19 18:53:49 Status: Completed
Worker     1: Calc     2: sim Time: 2.41 Start: 2019-11-19 18:53:48 End: 2019-11-19 18:53:50 Status: Worker killed
Worker     2: Calc     1: sim Time: 2.41 Start: 2019-11-19 18:53:49 End: 2019-11-19 18:53:51 Status: Worker killed
Worker     1: Calc     3: sim Time: 4.41 Start: 2019-11-19 18:53:50 End: 2019-11-19 18:53:55 Status: Completed
Worker     2: Calc     2: sim Time: 4.41 Start: 2019-11-19 18:53:51 End: 2019-11-19 18:53:56 Status: Completed
Worker     1: Calc     4: sim Time: 4.41 Start: 2019-11-19 18:53:55 End: 2019-11-19 18:53:59 Status: Completed
Worker     2: Calc     3: sim Time: 4.41 Start: 2019-11-19 18:53:56 End: 2019-11-19 18:54:00 Status: Completed