3D Gaussian Splatting and Differentiable Spatial Computing: Real-Time Radiance Field Rendering, Dynamic Scene Deformation, and Mobile XR Acceleration

Abstract—Photorealistic novel-view synthesis and real-time 3D scene reconstruction are fundamental to spatial computing, virtual and augmented reality (XR), autonomous robotics, and interactive digital twins. While Neural Radiance Fields (NeRFs) achieved landmark visual fidelity by parameterizing continuous volumetric radiance fields through implicit coordinate-based Multi-Layer Perceptrons (MLPs), their deployment in interactive real-time systems is severely bottlenecked by ray-marching volume rendering: querying deep neural networks hundreds of times per pixel limits rendering framerates to sub-real-time speeds (<15 FPS) on workstation GPUs and causes prohibitive thermal energy dissipation on mobile edge head-mounted displays (HMDs). 3D Gaussian Splatting (3DGS) establishes a transformative explicit radiance field paradigm that reconciles state-of-the-art photorealism with ultra-high framerate rasterization (>150 FPS at 1080p). By parameterizing 3D scenes as millions of continuous, anisotropic 3D Gaussians governed by 3D covariance matrices, spherical harmonics (SH) color coefficients, and opacity values, 3DGS executes tile-based differentiable rasterization via fast hardware-accelerated GPU sorting. However, scaling 3DGS to real-world dynamic physical environments introduces critical challenges: catastrophic memory scaling (>1.5 GB per scene), temporal visual artifacts during topological deformations and non-rigid motion, severe multi-view anti-aliasing aliasing over varying camera scales, and strict mobile compute constraints on embedded mobile XR silicon. This paper presents an exhaustive, mathematically formal technical investigation and empirical benchmark of 3D Gaussian Splatting and differentiable spatial computing architectures. We formalize the projection of 3D ellipsoidal covariance matrices into 2D camera screen coordinates via local affine Jacobian approximations, model continuous spatio-temporal deformation fields via neural hexagonal planes and 4D dynamic Gaussians, and derive entropy-constrained vector quantization codecs that compress explicit scene representations by over 88%. Furthermore, we construct an enterprise benchmarking testbed across static real-world scenes (Mip-NeRF 360, Tanks and Temples) and dynamic non-rigid datasets (D-NeRF) evaluated across high-end desktop GPUs (NVIDIA RTX 4090) and embedded mobile XR chipsets (Snapdragon XR2 Gen 2). Our empirical findings demonstrate that compressed 3DGS sustains 148 FPS at 1080p resolution while achieving 28.4 dB PSNR, outperforming NeRF baselines by 12.4x in rendering throughput while slashing storage footprint from 1.42 GB down to 164 MB. Finally, we formulate open challenges in all-optical spatial computing, open-vocabulary semantic Gaussian segmentation, and physical simulation coupling via differentiable continuum mechanics.