Sunrise Ops: Database Engine Architectural Evolution and Empirical Limits under High-Throughput Ingestion (2022-2026)

This technical report presents a longitudinal empirical study (2022–2026) evaluating the mechanical behavior, hardware scalability barriers, and architectural evolution of modern database engines: traditional relational systems (PostgreSQL 16, MySQL 8.4 LTS, Firebird 5.0), document stores (MongoDB 7.0 WiredTiger), and log-structured merge-tree (LSM-tree) wide-column systems (ScyllaDB 6.x Seastar). The research formalizes an engineering progression across three operational stages: STAGE I (2022–2023): Deterministic, low-level manual profiling (EXPLAIN ANALYZE BUFFERS, B-Tree lock contention, and cache pollution during logical backups). STAGE II (2024–2026): Modern asynchronous concurrent orchestration (LSM-trees, WiredTiger eviction stalls, and .NET 10 channel-based concurrency). STAGE III (2026): Unified polyglot persistence guidelines bridging transactional consistency, distributed stream ingestion, and HNSW vector indexing. Through empirical benchmarks involving 500,000 batch payloads and high-concurrency aggregations, we connect Little's Law and Neil Gunther's Universal Scalability Law (USL) with physical hardware bottlenecks (PCIe bus limits, NVMe IOPS, and L3 cache coherency). Our findings reveal that while thread-per-core LSM architectures excel at continuous asynchronous stream ingestion, PostgreSQL with raw binary protocols (COPY) outperforms distributed engines on single-node hardware with a remarkably compact client memory footprint (160.3 MB).