Enforcing Prediction Consistency Across Orthogonal Planes Significantly Improves Segmentation of FIB-SEM Image Volumes by 2D Neural Networks.

Volume electron microscopy (vEM) is transforming cell biology by generating high-resolution 3D reconstructions of large biological samples. However, segmentation of specific features such as mitochondria from highly heterogenous image volumes remains a bottleneck – even powerful deep learning (DL) approaches reveal various limitations and artifacts [1]. Within vEM technologies, focused ion beam scanning electron microscopy (FIB-SEM) can yield isotropic-voxel data where information in orthogonal planes (xy, xz, yz) is essentially interchangeable; here, we exploit this to develop a two-step DL algorithm for segmentation. First, we train a DL model to segment specific features in 2D image slices. Crucially, to incorporate 3D context into the predicted segmentation of a target volume, we run inference over xy, xz and yz planes, and average the results at each voxel, a procedure we call “ortho-plane inference”. In the second step, we use the target volume and predicted segmentation to train a new 2D model in a weakly supervised setting with “bootstrapping”. Bootstrapping enforces prediction consistency between adjacent voxels of the same object regardless of viewing orientation. This two-step algorithm results in a 23% increase in Intersection-over-Union (IoU) over the best case scenarios for ortho-plane inference without bootstrapping and a 35% IoU increase over “2D stack” inference (Fig 2). Sampling 3D volumes while staying in a 2D regime makes this approach nimble and thus well suited to vEM researchers with limited image and compute resources.