Health services research

Health services research is defined as ‘the multidisciplinary field of scientific investigation that underpins improvements in the way that health services are financed, organised, planned and delivered’. It encompasses a wide range of methodological techniques used to promote evaluation in healthcare. Evaluation, in turn, has been described as ‘ a comparative assessment of the value of an intervention using systematically collected and analysed data, in order to decide how to act’.1 Anaesthesiologists will be most familiar with the comparative evaluation of drugs and techniques using the randomised controlled trial2 and this is the best study design to answer questions about the effectiveness of interventions. However, there are many other types of study design. Because the range of questions relevant to the understanding and improvement of healthcare is so wide, their guiding principle is that a methodology is chosen which best allows the specific research question to be answered. Even studies that do not use specific research methodologies can shed light on patient care by using a systematic approach, as seen in a recent publication that analysed the critical incidents reported to the UK's National Reporting and Learning Scheme between 2006 and 2008.3 In this case, the focus was on developing a methodological approach to retrieving incidents from the database and sharing the lessons for practice more widely than the hospital where they occurred. Qualitative approaches The studies above involved the analysis of numerical data, namely, quantitative research, which is often distinguished from a qualitative approach, which consists of less measurement and more exploration, explanation and eliciting meaning.4 Principal methods in qualitative research include observation, questionnaires, interviews, focus groups and consensus methods. Analysis proceeds by induction, that is, the themes and findings arise not from some prespecified hypothesis, but are suggested by the data themselves.5 Prejudices and prior assumptions can easily influence how data collection is conducted. To keep bias to a minimum, reflexivity – the process whereby the researcher attempts to bring to conscious attention possible biases he or she may have – is part of the qualitative researcher's training. In contrast to some ‘observational studies’,6 in which data items for collection and outcomes of interest were prespecified, qualitative observation is likely to involve a freer, open-ended approach. The advantages of this method are that a picture of actual practice can be drawn, rather than relying on interview data, for instance, wherein respondents may tell the interviewer that they do something different from what they actually do in practice. This is illustrated by a study examining handovers between anaesthesiologists and recovery nurses in a postanaesthesia care unit.7 The researcher observed 45 such handovers, taking notes by hand for later transcription and analysis. Disadvantages of observational studies include observer bias, in which the observations are inadvertently influenced by the personal views of the observer. Studies may also be subject to the Hawthorne effect (or the ‘effect on the behaviour of the subjects of the watcher watching’), which may again influence the validity of the results. It is, therefore, essential that in any observational study, attempts are made in the methodology to minimise these potential limitations. Triangulation – the process whereby the initial results are further validated using a different stream of data – is demonstrated by the use of interviews in the same study.7 For example, the researchers did not ask participants how they handed over, but explored instead their perceptions of the process, often drawing on events from observations already made. Interviews also have disadvantages – they are expensive, time-consuming and subject to interviewer bias. Careful phrasing of prompting questions can also help, as can a good rapport between interviewer and respondent. In particular, when patients are being interviewed, it is preferable to have someone who is clearly independent of the hospital staff conducting the interview, as patients may be guarded in their answers if they feel that these may affect the quality of care they receive. Questionnaires and surveys Questionnaires are structured surveys containing open and/or closed questions that collect data for coding and analysis. Quantitative questionnaires have advantages as they produce numerical data that are easy to count and analyse, they can be administered to a large number of people and they are relatively economical. However, the precoded responses may not be sufficiently comprehensive and subtlety may be lost if answers are ascribed to a code which may not fully reflect the initial response. Questionnaires must be worded and ordered in such a way that patients will understand and give accurate responses. Questionnaires relying on respondents’ recollections of past events are subject both to recall bias and to ‘framing’, wherein answers are influenced by the design of the precoded response. There may also be a ‘social desirability’ bias, wherein respondents answer to portray the most positive image of themselves and their behaviour. A German study examined the use of tracheostomy on intensive care using a postal survey.8 The authors achieved a good response rate (89%) by enclosing a stamped addressed envelope for the return of the completed questionnaire initially, sending a second questionnaire to units that did not respond to the first, and then following up persistent non-responders by telephone. However, surveys usually achieve a poorer response than this figure, depending too on the type of enquiry. Postal surveys are thought to elicit the best response, followed by electronic mail, and Internet-based surveys producing the poorest yield.9 However, electronic mail and Internet-based surveys are easy, quick and inexpensive and may be useful, especially for pilot studies. Representativeness may also be a problem, as enquiries about practice may receive more or less optimistic answers depending on who answers them10 and there may be something different about responders and non-responders. Further, a short questionnaire is easy to answer but will not allow the detailed understanding that free text provides.10 Consensus methods Consensus methods aim to determine the extent of agreement among experts and will be familiar to anaesthesiologists in various consensus statements.11 Consensus can be obtained through face-to-face meetings of a group of experts, but this can be expensive as long-distance travel may be needed, and may be limited by lack of common language and shortage of time. The Delphi technique, on the other hand, allows experts’ views to be gathered singly and remotely. Responses are summarised and sent back to the respondents, asking for further comment and ranking. The results of this second round can be fed back once more for further refinement and the degree of agreement can be assessed formally using statistical techniques. Furthermore, avoiding face-to-face confrontation is thought to make people less inhibited and more willing to express their views. It also prevents discussions being dominated by one person. A recent study attempting to define excellence in anaesthetic practice made use of a modified Delphi technique.12 Anaesthesiologists with an interest in education were invited by electronic mail to list qualities associated with excellence in anaesthesia. However, rather than feeding the responses back to individuals, respondents were invited to take part in focused group interviews – unstructured interviews with small groups of people and a group leader – to rank the items from the first round. This technique is used frequently in business and advertising, and allows participants to discuss topics in depth, air personal views and exchange ideas with others. Systematic reviews A systematic review is a detailed structural analysis, using reproducible methods, of previously conducted primary research. It aims to summarise and critically appraise large bodies of evidence to help bring together and, if necessary, explain differences among studies on the same clinical question. Meta-analysis is the mathematical technique which combines the data from two or more methodologically similar studies.13 A systematic review involves working to a predetermined protocol, in the same way as one would when conducting a clinical study. This methodological aspect may not be so prominent a feature of traditional or ‘narrative’ reviews found in the journals.14 Setting out how studies will be selected, graded for methodological quality, combined and interpreted should reduce bias in the results of the final product. For instance, in a recent review of perphenazine for the prevention of postoperative nausea and vomiting,15 strict inclusion criteria were applied, relevant outcomes defined and the search strategy, data extraction and statistical methods all decided in advance of starting work. Eleven trials including over 2000 patients were included in the meta-analysis, which suggested that perphenazine is effective in both adults and children. Systematic reviews can only be as robust as the quality of their primary trials, and the care with which they are combined, will allow. Incomplete searching and publication bias can affect their validity, but a well conducted review can provide useful, statistically significant results. Conclusion There are many human as well as scientific aspects to practice in anaesthesia and critical care, and each methodological approach is needed for its particular contribution to the scientific advancement of our profession and improvement in the care of our patients. Acknowledgements A.S. is supported by a personal patient safety research grant from the UK National Institute for Health Research. This article was checked and accepted by the editors, but was not sent for external peer-review.