Effectiveness of Different Ground Truths at Detecting Lead Service Pipes Using Machine Learning

Replacing all existing lead service pipes (SPs) is an important strategic goal for the water industry because lead SPs represent a clear health risk. For example, the UK water industry has been instructed by the Drinking Water Inspectorate to be lead-free by 2050, and the USEPA has assigned $3 billion to tackle lead. The first key challenge in achieving this goal is determining whether a property’s service pipes are made of lead. Previous research into the application of machine learning (ML) to this problem has often made use of proxy problems to tackle the core goal of identifying lead SPs. The main proxies used have been lead concentration sampling and asset records. This paper will evaluate these proxies by using them to train ML models as target classifications and comparing the results with those from a model trained on excavations and trial holes based on a large real-world data set from a UK water company. The results challenge existing assumptions about the suitability of these proxies for training data-driven machine learning models. They show that models trained on these proxies can perform no better than random chance in determining if a property has a lead SP on this real-world data set. Instead, data need to be collected from excavations and trial holes before an effective model can be trained. The conclusions drawn will help to direct future research in this area and improve the effectiveness of lead SP replacement efforts by reducing the number of false positives that need to be visited.