Delta: A Lightweight Monocular Depth Estimation Model via Knowledge Distillation

This is the authors' abstract. We don't add key points for this paper.

Deploying accurate monocular depth estimation models on mobile devices is challenging due to computational constraints. We present Delta, a lightweight model achieving near real-time inference (average 186 ms on an iPhone 13) through an efficient knowledge distillation framework leveraging the state-of-the-art teacher model, Depth Anything V2. Delta employs a MobileViT-XS encoder to capture global context with minimal overhead, paired with a custom lightweight MiniDPT decoder. Training utilizes a composite loss function that integrates scale-invariant logarithmic, gradient matching, feature matching, and attention matching components for robust knowledge transfer. A proof-of-concept mobile application demonstrates the model's inference speed. Comprehensive evaluations affirm its suitability for applications such as augmented reality and navigation. Source code, ONNX models, and demo application: https://github.com/GitGud-f/delta