Automatic design optimization using parallel workflows

Workflows support the automation of scientific processes, leading to a more robust experimental process. They facilitate access to remote instruments, databases and parallel and distributed computers. Importantly, software pipelines can be established that perform multiple complex simulations (leveraging distributed platforms), with one simulation driving another. Such an environment is ideal for performing engineering design, where the goal is to evaluate a range of different scenarios “in-silico”, and find ones that optimize a particular outcome. However, in general, existing workflow tools do not incorporate optimization algorithms, and thus whilst users can specify complex computational and data manipulation pipelines, they need to invoke the workflow as a stand-alone computation within an external optimization tool. Moreover, many existing workflow engines do not leverage parallel and distributed computers, making them unsuitable for executing complex engineering simulations. To solve this problem, we have developed a methodology for integrating optimization algorithms directly into workflows. We implement a range of generic actors for an existing workflow system called Kepler, and discuss how they can be combined in flexible ways to support various different design strategies. We illustrate the system by applying it to an existing bio-engineering design problem running on a Grid of distributed clusters