Precomputation-Optimized Lakehouse Architecture for Online Analytical Processing Tasks

Managing diverse data formats and improving query processing efficiency at cloud service centers is crucial in the era of interconnected heterogeneous devices. Lakehouse effectively manages heterogeneous data but faces challenges in maintaining SQL query performance and data independence in large-scale analyses. Precomputation, which stores intermediate results in advance, reduces query latency by a space-for-time trade-off. This paper addresses the challenges of matching and rewriting results in lakehouse environments. By combining pre-computation with the low-cost and dynamically scalable storage characteristics of cloud storage, we introduce an optimized lakehouse architecture with an in-memory index and dynamic task scheduling to enhance OLAP performance while balancing data center resources. Experiments demonstrate improvements in query performance ranging from 6.1 % to 64.9% over native lakehouse architectures and reductions in resource occupation from the original 75.6% to 17.3% with preloading and 25.0% with dynamic scheduling.