Feature selection in cluster analysis for functional data with application to the labour market in Poland

The aim of this article is to apply methods of cluster analysis to functional data in order to determine if there are homogeneous groups among the provinces of Poland in terms of 20 features related to the labour market. In some existing studies all available features are used for cluster analysis, even though some of them may be insignificant for the task. Irrelevant features have a negative impact on the complexity and performance of clustering algorithms. In this study, before clusters of provinces were identified, features significantly differentiating provinces were selected. Our new feature selection criterion involves the use of coefficients of functional distance correlation and functional Hilbert-Schmidt correlation between the features characterizing provinces and their labels. The resulting clusters were nearly identical regardless of whether all 20 features were used or just the 9 significant ones, which indicates that irrelevant features do not contribute much to cluster identification. The methodology proposed in this article is flexible and can be successfully used in other research areas including, inter alia, poverty, socio-economic development or disability.