Seamless management of ensemble climate prediction experiments on HPC platforms

One of the main challenges for the weather, air quality and climate science is how to efficiently perform large numbers of simulations of the Earth system on a variety of supercomputers. In particular, the climate community has developed complex computational systems to obtain climate projections and predictions. A huge amount of computational resources are needed to produce these simulations, as well as to deal with the data coming in and out from the models. Regardless of the platform, climate simulations typically consist of hundreds of programs or scripts whose workflow can be complex. In this paper, Autosubmit, a Python-based tool that allows creating, launching and monitoring climate experiments, is introduced. The experiment is defined as a sequence of jobs that Autosubmit remotely submits and manages in a transparent way to the user. The same experiment can run in more than one supercomputing platform and for different workflow configurations. Autosubmit could be expanded to perform any weather, air quality and climate experiment on any computing platform to ensure the efficient handling of highly-dependent jobs, an optimum use of available computing resources, and a user-friendly management of the experiments, including creation, documentation, start, stop, restart, live monitoring and reproduction.