Representation learning for entity alignment (EA) aims to identify entities in different knowledge graphs (KGs) that refer to the same real-world object by comparing their embedding similarity. Although many EA models perform well on synthetic benchmark datasets, this performance does not always transfer to real-world, incomplete, and domain-specific data. A systematic comparison between synthetic benchmarks and original heterogeneous datasets is still limited. Many EA models also restrict the alignment search space to validation entities, limiting coverage of real KG content. Within this setting, our results show that embedding-based EA models continue to face generalization challenges in realistic large-scale KG search spaces. We evaluate several competitive EA models-commonly tested on benchmarks such as DBP15K-on multiple real-world heterogeneous datasets. The experiments reveal a performance decrease when moving beyond synthetic benchmarks, indicating that current models do not fully capture the characteristics of real data. We also analyze semantic similarity and profiling features of the datasets to help explain these differences. This study outlines practical limitations of embedding-based EA methods and provides insights for developing approaches that better handle the variability and complexity found in real-world KG alignment tasks.
Raoufi et al. (Tue,) studied this question.