Empirical preprocessing methods and their impact on NIR calibrations: a simulation study

Abstract The extraction of chemical information from dense particulate suspensions, such as industrial slurries and biological suspensions, using near‐infrared (NIR) spectroscopic measurements is complicated by sample‐to‐sample path length variations due to light scattering. Empirical preprocessing techniques such as multiplicative scatter correction (MSC), extended MSC and derivatives have been applied to remove these effects and in some cases have shown promise. While the performance of these techniques and other related approaches is known to depend on the nature and extent of the variations and on the measurement configuration, detailed investigations into the efficacy of these approaches under various conditions have not been previously undertaken. The main obstacle to carrying out such investigations has been the lack of, and the difficulty in obtaining, an accurate and comprehensive experimental data set. In this work, simulations that generate ‘actual’ measurements were carried out to obtain ‘experimental’ spectroscopic data on particulate systems. This was achieved by solving the exact transport equation for light propagation. A model system comprising four chemical components with one consisting of spherical submicron particles was considered. Total diffuse transmittance and reflectance data generated through simulations for moderate particle concentrations were used as the basis for examining the effect of particle size variations and measurement configurations on the efficacy of a number of preprocessing techniques in enhancing the performance of partial least squares (PLS) models for predicting the concentration of one of the non‐scattering chemical species. Additionally, a form of extended multiplicative signal correction based on considerations arising from fundamental light scattering theory is proposed and found to perform better than the other techniques for the cases considered in the study. Copyright © 2005 John Wiley & Sons, Ltd.