Cumulative sum (CUSUM) analysis to assess caudal block performance among anesthesia residents

Caudal analgesia is a common technique in pediatric anesthesia.1 Anesthesiology residents learn caudal blocks during their pediatric anesthesia rotation, but there is a learning curve and a minimum number of repetitions needed to achieve acceptable failure rates. A study in 2000 looked at the learning curve for caudal block among inexperienced residents using the bootstrap statistical modeling method and concluded 32 procedures are needed to achieve a success rate of 80%.2 The limit of the bootstrapping method is that it relies on the assumption the sample collected is representative of the population, and the prior study's results presents a dilemma because most residents are not able to do 32 caudal blocks during their training. Cumulative sum method (CUSUM) has been used to assess the learning curve of manual procedures among anesthesia residents previously including airway intubation, intravenous catheter placement, arterial line placement, and epidural and spinal placements.3-6 Our study looked at utilizing CUSUM graphs to assess learners' proficiency in caudal blocks, determining how many caudal blocks may be required to achieve competency, and if residents are able to perform enough caudal blocks to achieve an acceptable failure rate during their training. The technical skills of 17 anesthesiology residents who performed caudal blocks were evaluated over an 18-month period during their pediatric anesthesiology rotations using a standardized self-evaluation questionnaire. Residents at our institution start the pediatric rotation after at least 1 year of training in adult general and regional anesthesia. We utilized CUSUM analysis to determine individual learning curves. Acceptable failure rate was set at 15%, and "competency achieved" was defined as the graphical trend for an individual's learning curve falling below two boundary lines. As the failure rate in literature ranged from 15% to 25%,2 the more conservative cutoff was chosen (15%). Cumulative successes/failures were charted against the attempt number for the procedure and control "boundary lines" were determined with certain parameters, including: the acceptable outcome (f0), unacceptable outcome (f1), Type I error rate (α), and Type II error (β). The boundary lines were set at Ho and H1 and factors thereof (e.g., 2H1, 2H0). Acceptable and unacceptable failure rates were set at f0 = 0.15 (15%) and f1 = 0.30 (30%), respectively. The Type I and Type II error rates were set at 0.10, which is standard for these types of analyses. Intermediate values (a, b, P, Q, S) were calculated as shown in Table 1. If a procedure was successful, the next plot point was decreased by s, and if it was not successful, the next plot point was increased by 1-s. In our study, we focused on the clinical effect of caudal blocks. Failure was defined as when additional opioid medications were needed during the procedure, blood or CSF was aspirated, when the supervising attending anesthesiologist determined it to be a failed block, and/or the resident was unable to place the block. Success and failure are mutually exclusive for the purpose of this study. Of the 17 residents included in this study, five residents were able to achieve competency. Of the remaining 12 residents, six residents did not provide a minimum amount of data to establish a trend, and six residents did not achieve competency. Of the five residents who achieved competency by our definitions, the minimum number of repetitions for caudal blocks ranged from 6 to 20 procedures (Figure 1). Limitations to our analysis include the small sample size of residents. The questionnaires were not always consistently filled out by the residents for every caudal block, leading to some residents having a paucity of data or not reaching competency by this study definitions during the study period. These do not hinder our conclusion about the application of this data analysis tool to quantify resident learning curves but do limit our conclusions of competency for this resident cohort. CUSUM analysis can improve the methods of evaluating residents' technical skills across procedures by developing individual learning curves, including caudal procedures. There was wide variability in the number of procedures needed before an individual realized acceptable failure rates. However, individual performance while learning this procedure can be quantified and analyzed with the CUSUM approach, thereby improving methods of evaluating residents' technical skills. Future larger studies will help to further define the trend of obtaining competence for caudal blocks for anesthesia residents and the minimal number of procedures required. Although several residents did not reach competency by our study's definitions during the data collection, governing bodies do not have clear definitions of competency regarding caudal blocks for graduation. Such guidelines should be a point of discussion for governing bodies. None. None. Consents were obtained from each resident who participated in this study. The data that support the findings of this study are available from the corresponding author upon reasonable request.