libEnsemble: A complete toolkit for dynamic ensembles of calculations

libEnsemble empowers model-driven ensembles to solve design, decision, and inference problems on the world’s leading supercomputers such as Frontier, Aurora, and Perlmutter.

  • Dynamic ensembles: Generate parallel tasks on-the-fly based on previous computations.

  • Extreme portability and scaling: Run on or across laptops, clusters, and leadership-class machines.

  • Heterogeneous computing: Dynamically and portably assign CPUs, GPUs, or multiple nodes.

  • Application monitoring: Ensemble members can run, monitor, and cancel apps.

  • Data-flow between tasks: Running ensemble members can send and receive data.

  • Low start-up cost: No additional background services or processes required.

Quickstart

New: libEnsemble nows supports the gest-api generator standard, and can run with Optimas and Xopt generators.

Installation

Install libEnsemble and its dependencies from PyPI using pip:

pip install libensemble

Other install methods are described in the docs.

Basic Usage

Create an Ensemble, then customize it with general settings, simulation and generator parameters, and an exit condition.

import numpy as np
from gest_api.vocs import VOCS

from libensemble import Ensemble
from libensemble.gen_classes.sampling import UniformSample
from libensemble.specs import LibeSpecs, SimSpecs, GenSpecs, ExitCriteria


def six_hump_camel_func(calc_in: dict):
    """
    Definition of the six-hump camel test function.
    """
    x0 = calc_in["x0"]
    x1 = calc_in["x1"]
    term1 = (4 - 2.1 * x0**2 + (x0**4) / 3) * x0**2
    term2 = x0 * x1
    term3 = (-4 + 4 * x1**2) * x1**2

    return {"f": term1 + term2 + term3}


if __name__ == "__main__":

    # Define problem using VOCS
    vocs = VOCS(
        variables={"x0": [-3, 3], "x1": [-2, 2]},
        objectives={"f": "EXPLORE"},
    )

    # General settings
    libE_specs = LibeSpecs(nworkers=4)

    # Specify the simulator function
    sim_specs = SimSpecs(
        simulator=six_hump_camel_func,
        vocs=vocs,
    )

    # Initialize generator
    generator = UniformSample(vocs)

    # Specify the generator and other parameters
    gen_specs = GenSpecs(
        generator=generator,
        vocs=vocs,
        batch_size=50,
    )

    exit_criteria = ExitCriteria(sim_max=100)

    # Create ensemble
    ensemble = Ensemble(
        libE_specs=libE_specs,
        sim_specs=sim_specs,
        gen_specs=gen_specs,
        exit_criteria=exit_criteria,
    )

    # Run ensemble
    ensemble.run()

    ensemble.save_output(__file__)
    print("Some output data:\n", ensemble.H[["x0", "x1", "f"]][:10])

There are many more examples in the regression tests and Community Examples repository.

Resources

Support:

Further Information:

Cite libEnsemble:

@article{Hudson2022,
  title   = {{libEnsemble}: A Library to Coordinate the Concurrent
             Evaluation of Dynamic Ensembles of Calculations},
  author  = {Stephen Hudson and Jeffrey Larson and John-Luke Navarro and Stefan M. Wild},
  journal = {{IEEE} Transactions on Parallel and Distributed Systems},
  volume  = {33},
  number  = {4},
  pages   = {977--988},
  year    = {2022},
  doi     = {10.1109/tpds.2021.3082815}
}

See the tutorial for a step-by-step beginners guide.

See the user guide for more information.

Example Compatible Packages

libEnsemble and the Community Examples repository include example generator functions for the following libraries:

  • APOSMM Asynchronously parallel optimization solver for finding multiple minima. Supported local optimization routines include:

    • DFO-LS Derivative-free solver for (bound constrained) nonlinear least-squares minimization

    • NLopt Library for nonlinear optimization, providing a common interface for various methods

    • scipy.optimize Open-source solvers for nonlinear problems, linear programming, constrained and nonlinear least-squares, root finding, and curve fitting.

    • PETSc/TAO Routines for the scalable (parallel) solution of scientific applications

  • DEAP Distributed evolutionary algorithms

  • Distributed optimization methods for minimizing sums of convex functions. Methods include:

  • ECNoise Estimating Computational Noise in Numerical Simulations

  • Surmise Modular Bayesian calibration/inference framework

  • Tasmanian Toolkit for Adaptive Stochastic Modeling and Non-Intrusive ApproximatioN

  • VTMOP Fortran package for large-scale multiobjective multidisciplinary design optimization

libEnsemble has also been used to coordinate many computationally expensive simulations. Select examples include:

  • OPAL Object Oriented Parallel Accelerator Library. (See this IPAC manuscript.)

  • WarpX Advanced electromagnetic particle-in-cell code.