A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme

OBJECTIVES: To demonstrate the benefits of using appropriate decision-analytic methods and value of information analysis (DA-VOI). Also to establish the feasibility and implications of applying these methods to inform the prioritisation process of the NHS Health Technology Assessment (HTA) programme, and possibly extending their use therein. DATA SOURCES: Three research topics that were considered by the HTA panels in the September 2002 and February 2003 prioritisation rounds. REVIEW METHODS: A brief and non-technical overview of DA-VOI methods was circulated to the panels and Prioritisation Strategy Group (PSG). For each case study the results were presented to the panels and the PSG in the form of brief case-study reports. Feedback on the DA-VOI analysis and its presentation was obtained in the form of completed questionnaires from panel members, and reports from panel senior lecturers and PSG members. RESULTS: Although none of the research topics identified met all of the original selection criteria for inclusion as case studies in the pilot, it was possible to construct appropriate decision-analytic models and conduct probabilistic analysis for each topic. In each case, the tasks were completed within the time-frame required by the existing HTA research prioritisation process. The brief case-study reports provided a description of the decision problem, a summary of the current evidence base and a characterisation of decision uncertainty in the form of cost-effectiveness acceptability curves. Estimates of value of information for the decision problem were presented for relevant patient groups and clinical settings, as well as the value of information associated with particular model inputs. The implications for the value of research in each of the areas were presented in general terms. Details were also provided on what the analysis suggested regarding the design of any future research in terms of features such as the relevant patient groups and comparators, and whether experimental design was likely to be required. CONCLUSIONS: The pilot study showed that, even with very short timelines, it is possible to undertake DA-VOI that can feed into the priority-setting process that has been developed for the HTA programme. There are however a number of areas that need to be established at the beginning of the process, such as clarification of the nature of the decision problem for which additional research is being considered, explicitness about which existing data should be used and how data that exhibit particular weaknesses should be down-weighted in the analysis. Other areas, including optimum application of researcher time, integrating the vignette (a summary of the clinical problem and existing evidence) and the use of DA-VOI, training, use of sensitivity analyses, and deployment of clinical expertise, are also considered in terms of the potential implementation of DA-VOI within the HTA programme. Recommendations for further research include how literature searching should focus on those variables to which the model's results are most sensitive and with the highest expected value of perfect information; methods of evidence synthesis (multiple parameter synthesis) to consider the evidence surrounding multiple comparators and networks of evidence; and ways in which the value of sample information can be used by the NHS HTA programme and other research funders to decide on the most efficient design of new evaluative research. There is also a need for an analytical framework to be developed that can jointly address the question of whether additional resources would better be devoted to additional research or interventions to change clinical practice.