Assessing the effects of a collaborative research funding scheme: An approach combining meta-evaluation and evaluation synthesis

The Swiss Innovation Agency (CTI) has administrated its collaborative research funding scheme since the early 1980s. Between 1989 and 2002 the scheme was evaluated 14 times. In a study combining meta-evaluation and evaluation synthesis, we assessed these existing evaluations against a variety of criteria, including selected evaluation standards of the Swiss Evaluation Society. The meta-evaluation showed that the existing evaluations were mostly qualitative, internal and ex post and that evaluation practice at the CTI was episodic. The evaluations differed in quality, with most evaluation standards being fulfilled adequately to fairly well. The results of the meta-evaluation were central to the ensuing evaluation synthesis by giving information on the adequacy of the existing evaluations. The synthesis compiled the—mostly qualitative—results of the evaluations. Our study shows that there are patterns of outputs and outcomes in evidence across the evaluation studies that suggest that CTI funding is having an effect in terms of outputs and outcomes. But it also shows that there are gaps in the evidence base of the evaluation studies that make it difficult to come to more precise conclusions.