Patient feedback and anaesthetists: what are patients assessing and why?

In its document Supporting Information for Appraisal and Revalidation 1 the General Medical Council (GMC) states that doctors need to produce six types of supporting information for appraisal and revalidation, and that one of these is feedback from patients. The purpose behind obtaining feedback is to gather information on how the patients we treat perceive us, in order to guide our future development. From the outset, concerns were expressed by many anaesthetists about the problems they anticipated in obtaining valid feedback. In February 2012, the Royal College of Anaesthetists (RCoA), Faculty of Intensive Care Medicine and Faculty of Pain Medicine published a position statement endorsing the view that all anaesthetists should seek feedback from patients through generic patient feedback questionnaires (PFQs) 2. This document recognised that there are problems specific to anaesthetists in obtaining patient feedback, and further guidance was published by the RCoA in December 2012 3. As revalidation rolls out across the NHS, anaesthetists continue to grapple with the specific difficulties of our specialty in meeting the GMC's requirements. There is increasing emphasis on, and an assumption that we should provide, ‘patient-centred care’, and ‘patient satisfaction’ is seen as an important outcome measure; however, defining what these terms really mean is complex. Heidegger et al.'s review of patient satisfaction, in this issue of Anaesthesia, describes how work in this field has shown that communication and interpersonal skills are the most important factors influencing patients' satisfaction with anaesthesia care; however, whilst a literature search found around 4000 relevant papers, the overwhelming majority of studies did not define what satisfaction was or used validated tools to measure it 4, 5. In the second part of the review, Nübling et al. emphasise the need to use a strict protocol and assess the psychometric measuring qualities of the test, when developing a suitable questionnaire. Given that validated tools are already available to assess doctor's communication skills, assessing this aspect of patient satisfaction ought to be more straightforward, and the GMC has held back from requiring global assessments of satisfaction in favour of assessing communication with, and involvement of, patients. There are basic principles pertaining to obtaining feedback from patients that all doctors should be aware of and with which they should try to comply. The RCoA's latest guidance document 3 outlines ten principles that anaesthetists should consider before seeking feedback, and gives some examples of how some members of the RCoA Working Group on Colleague and Patient Feedback had successfully obtained feedback. In brief: patients should be randomly selected from the doctor's caseload; all patients should be considered eligible, including those for whom English is not their first language; a third party should administer the PFQs; the patients should understand why the feedback is being sought and must feel confident that their doctors will not see what they write; and the completed PFQs should be returned to a third party who will prepare a report. Doctors should see their feedback report before their appraisal, giving them time to reflect on it and to inform their Personal Development Plan. Appraisers should have received training in discussing patient feedback. It is important to remember that the GMC's and other validated PFQs are assessing doctors' communication skills, which involve a combination of conscious and unconscious processes and responses 6, and their ability to inspire trust and to make the patient feel involved in the decision-making about their treatment 7, rather than patients' satisfaction per se or the doctors' clinical performance. Concerns about clinical competence issues would be raised by other means, via other supporting evidence, including – but not exclusively – multi-source feedback from colleagues, adverse incidents and complaints. In primary care, obtaining feedback from patients is already part of the appraisal process, and is relatively easily obtained by having clerical staff randomly distribute PFQs to patients. For many doctors in secondary care, it is also relatively easy to arrange feedback; any doctor with an outpatient clinic should be able to arrange for clerical staff to administer PFQs randomly. Anecdotally, hospitals have taken a variety of approaches: some have taken over the administration of such systems themselves; some have employed private providers to do this; and some doctors have paid private providers themselves. My own Health Board, in the West of Scotland, will issue a doctor copies of a feedback form (Consultation and Relational Empathy (CARE) Measure (see below) 8, and the PFQs are returned to the Human Resources Department, whose staff will prepare a report for the doctor; the actual distribution of the PFQs is the responsibility of the individual doctor. As far as training appraisers in discussing patient feedback with their appraisees is concerned, this seems to be patchy at present. The problems faced by anaesthetists will vary depending on their practice. Those with outpatient clinics, including pain or pre-operative assessment clinics, should find it easiest; however, they have to remember that the feedback they obtain should reflect their whole clinical practice. Anaesthetists whose workload is exclusively general anaesthesia may find it most difficult; obtaining feedback postoperatively, when a patient is still affected by anaesthetic drugs, may be unreliable, and sending PFQs to a patient's home can lead to lower returns. But it must be remembered that it is doctors' ability to explain to patients who they are and what they are going to do, to engender trust and to ensure the patient feels involved in the process, that is being explored. Therefore, feedback can be undertaken on the pre-operative assessment interview. Anaesthetists with primarily paediatric practice, or doctors who work solely in intensive care, will have obvious problems, but PFQs have been designed to seek feedback from parents, carers and relatives. While the principle of random distribution of PFQs is perfectly reasonable, it is possible that in some circumstances it may lead to error. In a busy ward admitting patients for same-day surgery in multiple different theatres, there is enormous potential for the wrong PFQ to be given to a patient, and for a doctor to receive feedback on another doctor's communication skills. Doctors working primarily in one clinical area will find this easier, but ward nursing or clerical staff are often overstretched and the random administration and collection of PFQs will, understandably, not be a priority. One suggestion is that PFQs could be sent out by post. The RCoA guidance suggests that a photograph of the anaesthetist could be attached to increase the chances of the patient's assessing the correct doctor. This may work well, although personal communication with several doctors suggests that not all are happy for their names and photographs to be sent out to patients. Trusts, hospitals, department and individual doctors need to develop their own strategy for obtaining feedback, and have confidence that the feedback received is a reasonable and accurate assessment of doctors' communication skills. There are a number of PFQs available from commercial providers, but free validated PFQs available include the CARE Measure and the GMC's generic feedback form. The CARE Measure PFQ was developed in Scotland, initially for general practitioners, though it has been piloted in anaesthesia and has been shown to discriminate between doctors reliably 8. It is recommended that doctors receive feedback from a minimum of 25 patients using this tool. The GMC's generic feedback forms again assesses communication skills 9; it is suggested that 45 questionnaires should be distributed and a minimum of 34 returns received 10. While either form could be used by anaesthetists, the wording on the forms is not particularly suitable for anaesthesia and might confuse patients. The RCoA has received many enquiries about PFQs, and in response to this, its Revalidation Committee and Patient Liaison Group has modified the GMC's generic form. The wording is now more appropriate for anaesthesia, and this form is currently being piloted in four sites in the UK, with the GMC's approval. The form is currently available on the RCoA's website (see http://www.rcoa.ac.uk/patient-feedback-questionnaire). I have sought feedback using both the CARE Measure form and the RCoA-modified GMC form, and I am awaiting reports from both. Almost all of my patients come in on the morning of surgery – or sometimes during the course of the theatre day – and are seen by numerous healthcare professionals pre-operatively. In order to ensure that they receive the PFQ reasonably soon after I have seen them, and in the hope that they might complete it and assess me rather than another doctor, I ask a nurse, operating department practitioner or ward clerk to distribute and collect the forms shortly after I have undertaken my pre-operative visit. Therefore the process is not random, and I could be criticised for having the potential to choose patients with whom I have established good rapport. I have given the forms to consecutive patients, though this is outside the recommended guidelines. In my obstetric practice, PFQs are given to mothers receiving epidural analgesia in labour and regional anaesthesia for caesarean section. I do ask staff to distribute PFQs after caesarean section, so mothers are assessing both pre-operative and intra-operative care. There is the potential of course that issues outwith my control could impact on the feedback that they give, e.g. intra-operative pain if the obstetrician exteriorises the uterus. As one of the pilot leads I am entering my colleagues' feedback data on a database. I have already encountered one example of a patient (for whom English was not her first language) who rated the doctor poorly but added a text comment that she thought the anaesthetist was very good – I suspect she just did not understand the scoring system and entered her score at the wrong end of the scale! No system will be perfect, hence the need to view an occasional outlier as perhaps having been an error– but doctors can be sensitive souls, and it seems likely that any negative feedback will create a degree of angst. The primary purpose of revalidation is to reassure the public that doctors are up-to-date and fit to practise, and receiving patient feedback is perceived by the GMC and patient representative groups to be integral to the process; it is assumed that if doctors receive poor feedback they will take steps to improve their performance. While the wishes of the GMC and general public may be perceived as sufficient reason to engage in the patient feedback exercise, it is not unreasonable to ask if there is any evidence that this type of exercise does lead to improved performance or more satisfied patients, but this type of evidence is difficult to find. A study of general practice trainees has shown that systematic patient feedback at regular intervals throughout training led to sustained improvement in interpersonal skills, with most improvement seen in the earlier stages of training 11. Whether that can be extrapolated to all doctors receiving feedback once every five years remains to be seen. Patients who view doctors as unfriendly, unhelpful or uncommunicative are more likely to complain, and the majority of claims submitted to the Medical Protection Society involve poor communication 12. If those doctors who needed to improve their communication skills did so, patients might be happier – but so might doctors. Receiving complaints and poor feedback is unpleasant, and some doctors may be completely unaware of the fact that they are perceived to be uncommunicative. Dealing with such feedback therefore requires sensitive handling and this can be both time-consuming and stressful for all concerned. Assuming that some doctors do receive poor patient feedback, and there are no other concerns arising from other supporting information, what action would be taken? It is hoped that the mere existence of a poor report would stir a doctor into improving his/her communication skills. However, some of the feedback may be erroneous, with patients either assessing the wrong doctor, unfairly attributing problems to the wrong specialist or simply misunderstanding the form. Therefore there is the potential to cause unfair anxiety and distress for doctors. In those cases where there really is a communication problem, it is not clear what could be done. This is the type of low-level concern that it is hoped would be dealt with locally, but it is recognised that there is no consistent structure in place to deal with any level of remediation, and it seems possible that patient feedback may increase the number of low-level problems that need to be addressed, and overstretch the limited facilities available for dealing with performance issues 13. The other potential source for harm might be if doctors are distracted from their clinical responsibilities by arranging the distribution and collection of PFQs. As discussed earlier, it is possible to make arrangements for third parties to undertake this, but increasingly, under the financial constraints in which we work, this is not always feasible, and anything that takes the doctor's attention away from clinical care certainly has the potential to cause harm. However, the exercise has to be undertaken only once every five years, in 25–30 patients, so while this is a valid concern it needs to be seen in perspective. The decision has been made that doctors need to obtain feedback from patients, and while there are problems for all doctors, and some specific ones for anaesthetists, they are not insurmountable. The guidance for getting such feedback is merely guidance – it contains reasonable principles that should be followed whenever feasible, but my personal view is that Responsible Officers should be pragmatic about patient feedback initially, when structures are not consistently in place to support the exercise. As hospital organisational structures vary widely, the mechanisms used to obtain feedback are likely to differ, particularly in the early years of this exercise. Like many aspects of revalidation, it is an evolving process and will be fine-tuned as time goes on and as doctors develop more experience in the process. The bigger questions about whether it is worth the energy and expenditure spent on it – whether any aspect of the patient's journey will be improved by it – may perhaps be answered in time. It would be worth bearing in mind that any process that distracts doctors from their primary duty of caring for patients may lead to more harm than good. Similarly, if systems evolve that end up costing a lot of money the process needs to be re-assessed – there is the potential for private companies to see a market, and neither doctors nor patients would wish for money to be directed away from patient care. It is essential that all involved in the process remember that patient feedback is only one of six pieces of supporting evidence, and that it assesses communication skills and the ability to empathise and engender trust. As Heidegger and colleagues have highlighted 4, 5, patient satisfaction is a much broader issue and we are not currently at the stage where we can routinely measure satisfaction with anaesthesia. Further, it is entirely possible that a doctor could score very highly in the proposed assessment exercise and yet have clinical competence issues, or vice versa – patient feedback is only one piece of the jigsaw. It is important to demonstrate that patient feedback is beneficial in some way, either in reassuring the public and/or improving some aspect of care, and that it specifically does not cause harm either to patients or doctors. If patient feedback encouraged doctors to improve their interpersonal skills then this would be a positive benefit. In the meantime, all anaesthetists should strive to obtain feedback, comply with the GMC's recommendations, and accept that this is likely to become part of our working lives. I am a member of RCoA Revalidation Committee and Deputy Chair of the Scottish Academies Revalidation Group; however, the views I have expressed are my own. No external funding declared.