Bridging deep segmentation and metaverse visualization: Cellpose-based 3D brain tumor reconstruction from MRI

Accurate and efficient brain tumor segmentation is critical for diagnosis, treatment planning, and outcome monitoring in neuro-oncology. This study presents an integrated framework that combines deep learning-based tumor segmentation with 3D spatial reconstruction and metaverse-aligned visualization. The Cellpose segmentation model, known for its shape-aware adaptability, was applied to grayscale T1-weighted MRI slices to generate binary tumor masks. These 2D masks were reconstructed into 3D surface meshes using the marching cubes algorithm, enabling the computation of clinically relevant spatial parameters including centroid, surface area, bounding box dimensions, and mesh extents. The resulting tumor models were embedded into a global coordinate system and visualized across orthogonal planes, simulating extended reality (XR) environments for immersive anatomical exploration. Quantitative evaluation using DICE, Intersection over Union (IoU), and Positive Predictive Value (PPV) validated the segmentation accuracy, with DICE scores exceeding 0.85 in selected cases. The reconstructed tumors exhibited surface areas ranging from ~45,000 to ~74,000 voxel² units and extended across more than 200 units along the Y and Z axes. Although volumetric values were not computed due to open mesh geometry, the spatial profiles provided a reliable foundation for integration into metaverse platforms. This pipeline offers a lightweight and scalable approach for bridging conventional 2D tumor imaging with immersive 3D applications, paving the way for advanced diagnostic, educational, and surgical planning tools.