How Much Does the Classifier Matter? A Cluster-Aware Comparison of Six Machine Learning Algorithms for Land Use Land Cover Mapping of Taipei, Taiwan, Using Google Earth Engine and Python

Comparative studies of machine learning classifiers for Land Use Land Cover (LULC) mapping often recommend one algorithm, yet their validation pixels are usually drawn from a few hand-digitised polygons and are far from independent. This study tested whether such differences survive that dependence. Six classifiers, namely Classification and Regression Tree (CART), Random Forest, Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM), a multilayer perceptron and a support vector machine (SVM) with a radial basis function kernel, were applied to four top-of-atmosphere bands of a 2020 annual Sentinel-2 median composite of Taipei City, Taiwan, at 10~m. All 110 reference polygons were kept whole in both the hold-out split and the tuning folds. Pairwise differences were tested with the standard and the cluster-corrected McNemar test, and accuracies and class areas were estimated with the Olofsson stratified estimator on the effective sample size. Hold-out overall accuracy ranged only from 0.927 (CART) to 0.956 (SVM). The 1,829 validation pixels came from 30 polygons and, with a design effect of 71-76, were equivalent to only 24-26 independent observations. Eight of the fifteen pairwise differences were significant under the standard test, but none remained significant after the cluster correction. Mapped areas nevertheless differed markedly between models, from 71.7 to 102.1~km² for built-up land and from 10.9 to 32.2~km² for swamp; for built-up land, the between-model spread was as large as the sampling uncertainty of a single map. The six classifiers are therefore statistically indistinguishable in accuracy, and the choice among them should rest on computational cost, which differed by a factor of about 330 at inference, and on the stability of the areas they produce.