Efficient Real-Time Quantile Regression for Streaming Data

Abstract Quantile regression (QR) is a powerful tool for modeling the conditional quantiles of a response variable given a set of covariates, offering a comprehensive portrayal of their relationship without assuming a specific distribution for the error term. This makes QR particularly suitable for analyzing skewed or heterogeneous data, with wide-ranging applications in fields such as economics and finance. However, traditional QR methods are often impractical for stream data, where data blocks arrive sequentially and demand real-time updates. To address this, we propose a Sequential Quantile Regression (SQR) algorithm that significantly enhances computational efficiency by converting the check loss optimization problem into a least squares problem. The SQR algorithm leverages a Bayesian framework for sequential updates, using the posterior distribution from previous data as the prior for new data. This approach allows for rapid, efficient updates suitable for real-time processing of stream data. We extend the SQR algorithm to Composite Quantile Regression (CQR) and provide theoretical analysis demonstrating the unbiasedness, asymptotic normality, and linear convergence of the SQR estimator. Simulation studies and real data analysis show that SQR offers favorable estimation and inferential performance while being significantly faster than existing methods. Our results highlight the potential of SQR for various applications requiring real-time data processing.