Training-Data Coverage and Generalization in Vehicle Asking-Price Prediction: Categorical Group and Numerical Range Holdouts

An error estimate calculated while testing a machine learning model depends on what kinds of examples are represented in its training. A vehicle price model may be tested on a new listing while having been trained on many similar listings. Alternatively, it may face a category or numerical region that was absent from training. This study measures how those forms of coverage change the performance of a fixed vehicle price pipeline. A LightGBM model was evaluated on 20,801 marketplace listings using random listing-level validation, complete mileage and vehicle age range holdouts, and complete make/model and manufacturer holdouts. Random validation produced mean absolute error (MAE) of USD 3,015. MAE increased to USD 4,171–4,403 under numerical range holdouts and USD 7,676–7,921 under categorical-group holdouts. A separate comparison held the test listings and number of training rows constant. In that design, withholding selected interior mileage or age ranges raised MAE by 15%–19%, whereas withholding the 100,000-mile-and-above or age-nine-and-above tails raised it by 82%–89%. Dollar and percentage errors emphasize different vulnerable parts of the training gaps: low-mileage and recent vehicles produce larger dollar misses, whereas high-mileage and old vehicles produce larger percentage misses. Sensitivity checks demonstrated changes in size but not the direction of the main penalties. For this dataset and the associated pipeline, reliability of the model’s prediction depended not only on how much training data existed, but also on which parts of the vehicle attributes it covered.