Variation in language phylogenies may result from variation in concept translation

Phylogenetic reconstruction in historical linguistics now typically relies on cognates sets assembled from multilingual wordlists. While more and more scholars now trust in the robustness of the algorithms underlying the reconstruction of cognate-based language phylogenies, few studies have actually tested to which degree initial choices during data preparation can influence their outcome. Here, we provide first tests that focus on the role that concept translation—the initial stage of wordlist compilation, in which a list of concepts is translated into the target languages prior to identifying cognate sets—plays in phylogenetic reconstruction. Based on a newly compiled comparative dataset consisting of seven wordlists from five language families in which the same language varieties were coded by different authors, we investigate to which degree differences in concept translation lead to differences in phylogenetic reconstruction. Our results show that despite considerable differences in concept translation, lexical distances still show a considerably high correlation across all datasets. However, when comparing individual phylogenies reconstructed with the help of Bayesian inference, we find considerable differences ranging between 0.02 and 0.44 in normalized quartet distances computed from posterior tree samples. An additional cluster analysis that we introduce shows that larger differences in phylogenies do not necessarily correspond to high disagreement in the larger subgroups. A detailed inspection of individual concept translation differences in the Indo-European and Tupian wordlists in our sample further confirms that concept translation differences may specifically impact subgrouping decisions in lower clades of the tree, while major groupings are often unaffected.