Installing the full MID Benchmark Suite has three main parts:

  1. Installing the MID Benchmark Suite core library -- this handles common functions across the entire suite, such as the main classes, computing common ID metrics, etc.
  2. Downloading any datasets or pre-trained models that you desire. These are existing curated datasets that we have used in past publications and that are compatible with the MID Benchmark Suite. We do not download these by default, though there are opens in both the install and when using the API to download these when needed.
  3. Installing the individual simulation environments -- these are only needed if you wish to run more advanced metrics that require actually evaluating a given engineering solver.

You can run certain models and metrics using only the core MID library, however for most meaningful benchmarks you will need to access to at least one simulation environment.

Installing the core MID Benchmark Suite Library

For the baseline library, you can install this with pip via:

pip install midbench

Download ID Datasets and Pre-trained Baseline Models

Installing Simulation and Optimization Containers

Singularity Container

Singularity is a container platform. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible. You can build a container using Singularity on your laptop, and then run it on many of the largest HPC clusters in the world, local university or company clusters, a single server, in the cloud, or on a workstation down the hall. Your container is a single file, and you don’t have to worry about how to install all the software you need on each different operating system.

Singularity was created to run complex applications on HPC clusters in a simple, portable, and reproducible way. First developed at Lawrence Berkeley National Laboratory, it quickly became popular at other HPC sites, academic sites, and beyond. Singularity is an open-source project, with a friendly community of developers and users. The user base continues to expand, with Singularity now used across industry and academia in many areas of work.

Many container platforms are available, but Singularity is focused on:

  • Verifiable reproducibility and security, using cryptographic signatures, an immutable container image format, and in-memory decryption.

  • Integration over isolation by default. Easily make use of GPUs, high speed networks, parallel filesystems on a cluster or server by default.

  • Mobility of compute. The single file SIF container format is easy to transport and share.

  • A simple, effective security model. You are the same user inside a container as outside, and cannot gain additional privilege on the host system by default. Read more about Security in Singularity.

For more information about Singularity Container, please check the official website.

Installing Singularity

You will need a Linux system to run Singularity natively. Options for using Singularity on Mac and Windows machines, along with alternate Linux installation options are discussed here.

Install System Dependencies

You must first install development libraries to your host.

$ sudo apt-get update && sudo apt-get install -y \
    build-essential \
    libssl-dev \
    uuid-dev \
    libgpgme11-dev \
    squashfs-tools \
    libseccomp-dev \
    wget \
    pkg-config \
    git \
    cryptsetup

Three steps to install Singularity:

  1. Installing GO (Linux)

  2. Downloading Singularity

  3. Compiling Singularity Source Code

Download Singularity from a release

You can download Singularity from one of the releases. To see a full list, visit the GitHub release page. After deciding on a release to install, you can run the following commands to proceed with the installation.

$ export VERSION=3.5.3 && # adjust this as necessary \
    wget https://github.com/singularityware/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz && \
    tar -xzf singularity-${VERSION}.tar.gz && \
    cd singularity

For our MID Benchmark Suite, the Singularity version 3.5.3 is used to build our Singularity containers for different environments. Users should be able to run the Singularity containers with the same version or higher.

Compile the Singularity source code

Now you are ready to build Singularity. You can build Singularity using the following commands:

$ ./mconfig && \
    make -C builddir && \
    sudo make -C builddir install

Singularity must be installed as root to function properly. You can verify you've installed Singularity by typing the following command:

$ singularity version

Confirm that the command prints the installed version of Singularity.

We use Singularity containers for many of our different environments. An example of using SU2 Singularity container for 2D airfoil simulation and optimization can be found in one of our tutorials about the 2D airfoil inverse design.

Docker Container

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production.

Docker provides the ability to package and run an application in a loosely isolated environment called a container. The isolation and security allows you to run many containers simultaneously on a given host. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host. You can easily share containers while you work, and be sure that everyone you share with gets the same container that works in the same way.

Docker provides tooling and a platform to manage the lifecycle of your containers:

  • Develop your application and its supporting components using containers.
  • The container becomes the unit for distributing and testing your application.
  • When you’re ready, deploy your application into your production environment, as a container or an orchestrated service. This works the same whether your production environment is a local data center, a cloud provider, or a hybrid of the two.

For more information about Docker, please check the official website.

Installing Docker

  • Linux Users: Please use the following commands to install Docker:
$ sudo apt-get remove docker docker-engine docker.io && \
    apt-get update && \
    apt install docker.io
  • Windows and Mac Users: Mac and Windows users should install the Docker Toolbox (this is a simple one-click install). If running on Mac or Windows, make sure you run the following commands inside the Docker Quickstart Terminal.

We use Docker container for our 2D heat exchanger optimization environment. Please find the usage of the dolfin-adjoint Docker in this tutorial.