An adaptive meta-scheduler for data-intensive applications

In data-intensive applications, such as high-energy physics, bio-informatics, we encounter applications involving numerous jobs that access and generate large datasets. Effective scheduling of such applications is a challenge, due to the need to consider for both computational resources and data storage resources. In this paper, we describe an adaptive scheduling model that considers availability of computational, storage and network resources. Based on this model we implement a scheduler used in our campus grid. The results achieved by our scheduler have been analysed by comparing with greedy algorithm that is widely used in computational grids and some data grids.