Answers are grounded in this paper and cite their sources.
Over the last few years, the increasing demand on processing streaming data with high throughput and low latency has led to the development of specialized stream processing engines (SPE). Although existing SPEs show high performance in evaluating stateless operations and stateful operations with small windows, their performance degrades significantly when calculating exact answers for complex aggregate queries with huge windows. Examples include correlated aggregations, quantile and ordering statistic computation. Meanwhile, modern database systems have demonstrated the ability of processing complex analytical tasks efficiently over very large datasets, using technologies such as vertical storage, vectorized query execution, etc. This suggests the feasibility of leveraging database systems to assist SPEs to process complex aggregate queries to reduce their evaluation latency.
No takes yet. Share an insight, caveat, or question.
Yuanzhen Ji (2013) studied this question.