Semi‐automated measurement of nuchal translucency thickness: blasphemy or oblation to quality?

The quality of nuchal translucency (NT) measurement is widely regarded as the cornerstone of ultrasound screening in the first trimester. It can also be seen as the lowest common denominator that ultrasonologists should be able to achieve. The Fetal Medicine Foundation (FMF) certification for NT measurement was a breakthrough in the practice of obstetric ultrasound. The litany of requirements for an appropriate measurement are recited as the 11th commandment by several thousands of FMF believers and non-believers alike. NT measurement has also become an undisputed genetic test, for which pre- and post- procedure counseling are required, potentially being a matter of life and death for the fetus, either through disclosure of an underlying severe abnormality or through fetal loss related to an invasive procedure triggered by a combination of risk factors, of which NT is the most powerful. ‘Automation’ describes the execution by a machine, usually a computer, of a function that was previously carried out by a human. The aim of replacing the human operator is generally to increase efficiency, reliability, accuracy and/or cost1. The use of automation in medicine is not new and its application to ultrasound seems both logical and feasible, since more challenging pattern and image recognition processes, including cytogenetic and malignancy screening as well as computer-aided and robotic surgery, have been successfully implemented for years. Automation is rarely all-or-nothing, and interaction with human experience and judgement are required for appropriate control1. Two articles published in this issue of the Journal2, 3 and one published ahead of print4 suggest that there might be a benefit from semi-automated measurement of NT. Although the senior author of one of the papers is Kypros Nicolaides, I would like to raise some concerns, acting as a guardian of the Temple but hopefully not as the Pythia of Delphi. Table 1 summarizes the steps and requirements for obtaining an optimal NT measurement, of which the measurement itself is only one, and including the possibility for semi-automated measurement as an alternative to manual measurement5. The success of semi-automated measurement of NT is based on the operator's selection of an appropriate image and manual placement of a region-of-interest (ROI) box over a limited part of the nuchal area, comprising both distal and proximal echogenic lines defining the translucency. It allows for inner-to-inner (calipers placed on the edge of the echogenicity of the NT area—equivalent to manual ‘on-to-on’ measurement) and inner-to-middle (lower caliper placed at the geometrical middle of NT echogenic margin) caliper placement options. The software then allows for the largest, mean or shortest of these measurements to be chosen within the measurement box. Thus, the choice of the best placement of the calipers, while performed automatically by the semi-automated measurement software, is limited by the operator's decision as to placement of the ROI box and the type of measurement to be performed2-4. This human–machine interaction is exposed to two main types of drift from the expected clinical and economic benefits, which can be designated as misuse and abuse of automation. Abuse occurs when the design of automated function does not fit with the clinical expectation and the operator's role becomes a by-product of the automation. This can lead to misuse, i.e. overreliance on automation1. Automated measurements may well be systematically larger than the corresponding manual measurements of NT by an average of 0.2 mm3. Inner-to-inner caliper placement satisfies The FMF requirement to show the whole translucency and nothing but the translucency. However, ultrasound image resolution has improved dramatically over the last 5–10 years and both pre- and post-processing tools are responsible for a sharper image and possibly thicker echogenic margins of the structures under examination, including NT6. The higher the gain the smaller the NT measurement (Figure 1) and the same applies to harmonics6. This probably explains why so many certified sonographers undergoing recent assessment of their measurements have a median value that is smaller than the expected median FMF value, while the 5th and 95th centiles are as expected. Figure 2 shows an example of a team of seven sonologists with up-to-date FMF certification and performing over 5000 first-trimester examinations per year. Some argue that looking at differences in NT below the wavelength of the probe used (0.25 mm for 6 MHz) is in vain3. Indeed, it takes more than the wavelength to determine the axial resolution of the ultrasound measurement; the number of beams used and the way they cross also comes into play, as does the use of harmonics. Questioning the validity of the ultrasound measurement of NT to the tenth of a millimeter on this basis seems as futile as challenging the fact that planes can fly, despite being heavier than air. Semi-automated measurement of nuchal translucency thickness (NT) using the inner–inner caliper placement option (c–e) overestimates appropriate manual measurement using on-to-on placement of the calipers by 0.1 mm (a) and overestimates it by more with a lower gain setting (b). Semi-automated measurement of NT does not change with gain (c–e). Note, the region-of-interest box is not displayed on these images because, once a measurement has been validated by the operator, the box disappears from the screen, leaving only the calipers visible. Images kindly provided by J.-P. Bault. Distribution around the median value of nuchal translucency thickness (NT) measured manually by a team of seven sonologists up-to-date with The Fetal Medicine Foundation (FMF) certification and performing over 5000 first-trimester ultrasound examinations per year. This shows a normal distribution around a median value that is below The FMF median. Distributions kindly provided by J.-P. Bernard. Measurements made semi-automatically seem to remain stable regardless of gain, as shown in Figure 1 using a posteriori variations of gain on the same image and repeating measurements both manually and with semi-automated measurement. However, it is not possible to evaluate rigorously the effect of harmonics on a single image since harmonics cannot be modified once the image is frozen. Crude errors in NT measurements are generated by semi-automated measurement if the ROI box encompasses more of the nuchal area than strictly the margins of NT (Figure 3). In semi-automated measurement of nuchal translucency (NT), positioning of the region-of-interest (ROI) box over an area that includes more than the margins of the NT can lead to erroneous caliper placement and a false reading. Measurements using correct (a) and incorrect (b) positioning of the ROI box in the same image are shown. Note, the region-of-interest box is not shown on these images because, once a measurement has been validated by the operator, the box disappears from the screen, leaving only the calipers visible. Images kindly provided by J.-P. Bault. NT measurement, although critical, only concludes the series of rigorous processes of acquisition of an appropriate image (Table 1). Combining automation and human intervention is therefore more likely than not to be more demanding of human understanding and capabilities. This is one message from Moratalla et al.2, whose paper aims to demonstrate the added value of (semi) automation in decreasing the variability of the measurement of a ‘perfectly’ assessed NT within a range of rather high values (1.5–3 mm) with an overrepresentation of measurements between 2.3 and 2.7 mm. They do not compare values obtained by semi-automated measurement with those obtained manually. Grangé et al.4, reaching the same conclusion, suggest that the benefit of semi-automated measurement could also apply to less experienced operators and to images of lower quality. Abele et al.3 did compare values obtained by manual measurement and by semi-automated measurement3. They also worked on the best available images reviewed by sonologists of different levels of experience and they appropriately considered the gold standard to be the mean of the manual measurements made by two experts. They challenged the dogma of caliper placement by comparing inner-to-inner manual measurement with semi-automated measurement using inner-to-inner as well as inner-to-middle placement, finding that inner-to-inner, i.e. not strictly on-to-on, manual measurement and inner-to-inner semi-automated measurement produced similar results, but that inner-to-middle semi-automated measurement yielded significantly higher measurements compared with manual measurement. Another interesting result of this study is that the inter- and intraobserver variability of both semi-automated measurement and manual measurement of NT by expert examiners were similar, and were substantially lower than those of the less experienced sonologists, even with the semi-automated measurement method. They conclude that semi-automated measurement of NT improves the standardization of NT assessment. One could, however, interpret these results differently and argue that semi-automated measurement is not useful for well-trained operators. Furthermore, variability of semi-automated measurement of NT remains significant in less experienced examiners even when they are working on ‘ideal’ images that have been acquired by experts4. This approach is consistent with that generally used for evaluating new high-tech developments—by comparing results obtained using the new technology with expert ‘visual’ review. However, in such situations the tests are not independent measures and do not reflect the effect of the introduction of the new method of screening on screening performance or cost-effectiveness. My conclusion, therefore, is that manual skills defined by The FMF are necessary and sufficient for reliable and reproducible NT measurement until proven otherwise in an appropriate clinical study. Indeed, who cares about marginally better reproducibility of poor measurements? It also may well be that normal ranges established through cumulative measurements over the last 10 years should be reviewed, retaining the well-established method of manual measurement of NT, i.e. on-to-on. This might seem more logical than falling for the decoy of an arbitrarily set method for which there are no published reference ranges or algorithms for combined risk calculation. Automation requires further research to achieve the grail of the optimal sagittal plane, the major requirement for obtaining valid and reproducible NT measurements and maintaining the quality of the examination7, 8. Hopes put in three-dimensional (3D) imaging have not been fulfilled; even slight rotation of the fetal head on the axis of the body prevents reconstruction of a true sagittal plane from a single 3D sweep. Important clinical issues could arise from the premature and wide distribution of software allowing automated NT measurement, provided as a freebie with ultrasound machines, if the operator not only has to master all steps of the optimal manual procedure but also has to understand the human–machine interaction in order to prevent misuse of automation. Widespread use of semi-automated measurement could also lead to confusion at this critical time for NT, which is only now taking off as part of many national healthcare guidelines, thereby undermining 19 years of effort, exemplary teaching programs and quality assessment.