Classification of acoustic survey data: A comparison between seven teams of experts

Acoustic surveys of mixed fish aggregations often rely on manual allocation of acoustic energy to target species, with auxiliary information such as trawl catches and target strength distributions aiding interpretation. However, different teams of experts may use the same auxiliary data differently. In this experiment, seven teams of experts interpreted the same acoustic data, and we compared their classification to target categories to quantify uncertainty in the manual classification process. The teams largely agreed on the total acoustic energy attributable to organisms, but there was significant variation in how the teams split this energy on different target categories. This was caused by differences in applied thresholds for separating plankton and fish, as well as disagreements in species classification. For all target categories, the variation due to teams was lower than the overall variability across acoustic segments, but when scaled up to the number of segments in a typical survey, the team effect either dominated or was of similar magnitude as the segment variability. These results imply a need for further standardisation and uncertainty estimation of expert evaluations in acoustic surveys involving manual interpretation of echo sounder data.