Using a database management system (DBMS) is essential to ensure the data integrity and reliability of large, multidimensional data sets. However, loading multiterabyte data into a DBMS is a time-consuming and error-prone task that the authors have tried to automate by developing the sqlLoader pipeline-a distributed workflow system for data loading.
No takes yet. Share an insight, caveat, or question.
Szalay et al. (2008) studied this question.