NII Technical Report (NII-2006-012E)：Building a Terabyte-scale Web Data Collection "NW1000G-04" in the NTCIR-5 WEB Task | Synapse
April 1, 2026Open Access
NII Technical Report (NII-2006-012E)：Building a Terabyte-scale Web Data Collection "NW1000G-04" in the NTCIR-5 WEB Task
Key Points
To build a large-scale web data collection for use in the NTCIR-5 WEB task.
Developed the NW1000G-04 web data collection
Ensured the collection reached terabyte-scale size
Compiled detailed statistics on the data collection
Successfully created the NW1000G-04 collection at terabyte scale
Provided comprehensive statistics related to the data collection
Abstract
We built a terabyte-scale web data collection, NW1000G-04, which was used in the NTCIR-5 WEB task. This paper describes the process of building the collection and some statistics of it in detail.