Designing Non-uniform Locally Repairable Codes for Wide Stripes under Skewed File Accesses

Large-scale storage systems increasingly deploy erasure coding to provide high-reliability and low-cost data storage. To achieve further storage savings while maintaining repair efficiency, enterprises and academia explore wide stripes using locally repairable codes (LRCs) and several constructions are proposed. However, we find that these wide stripe LRCs neglect the file access information within the stripe, which matters as these LRCs assign uniform local group sizes while a wide stripe comprises many files with non-uniform or skewed access frequencies. Consequently, the repair efficiency is compromised. To this end, we design a novel wide stripe LRC construction called non-uniform LRC by fully considering the skewed file accesses in a wide stripe. Our non-uniform LRC assigns small group sizes for hot files in the stripe to achieve low repair cost and large group sizes for cold files to realize low storage overhead. It performs well under general coding parameters, adapts to irregular file sizes, provides high fault tolerance, and can be flexibly deployed to various data placements in real-world clustered storage systems. We implement our non-uniform LRC in a distributed storage prototype. Evaluations show that it greatly reduces the degraded read time and improves the node repair rate compared with the state-of-the-art wide stripe LRCs.