Data quality, performance, and uncertainty in taxonomic identification for biological assessments

Taxonomic identifications are central to biological assessment; thus, documenting and reporting uncertainty associated with identifications is critical. The presumption that comparable results would be obtained, regardless of which or how many taxonomists were used to identify samples, lies at the core of any assessment. As part of a national survey of streams, 741 benthic macroinvertebrate samples were collected throughout the eastern USA, subsampled in laboratories to ∼500 organisms/sample, and sent to taxonomists for identification and enumeration. Primary identifications were done by 25 taxonomists in 8 laboratories. For each laboratory, ∼10% of the samples were randomly selected for quality control (QC) reidentification and sent to an independent taxonomist in a separate laboratory (total n = 74), and the 2 sets of results were compared directly. The results of the sample-based comparisons were summarized as % taxonomic disagreement (PTD) and % difference in enumeration (PDE). Across the set of QC samples, mean values of PTD and PDE were ∼21 and 2.6%, respectively. The primary and QC taxonomists interacted via detailed reconciliation conference calls after initial results were obtained, and specific corrective actions were implemented (if needed) prior to a 2nd round of comparisons. This process improved consistency (PTD = 14%). Corrective actions reduced the proportion of samples that failed the measurement quality objective for PTD from 71 to 27%. Detailed comparisons of results for individual taxa and interpretation of the potential causes for differences provided direction for addressing problematic taxa, differential expertise among multiple taxonomists, and data entry and recording errors. The taxa that proved most difficult (i.e., had high rates of errors) included many Baetidae, Odonata, Ceratopogonidae, selected groups of Chironomidae, and some Hydropsychidae. We emphasize the importance of experience and training and recommend approaches for improving taxonomic consistency, including documentation of standard procedures, taxonomic data quality standards, and routine and rigorous quality control evaluations.