Neural information retrieval systems have achieved remarkable success for high-resource languages, yet their application to morphologically complex, lower-resource languages like Turkish remains underexplored. While dense bi-encoders dominate current Turkish IR systems, late-interaction models which preserve token-level representations for fine-grained semantic matching have not been systematically evaluated for Turkish retrieval tasks. We present TurkColBERT, the first comprehensive benchmark comparing dense encoders and late-interaction models on Turkish information retrieval. We introduce a two-stage adaptation pipeline: first, we fine-tune english-based (ettin, bert-hash) and modern multilingual encoders (mmBERT) on Turkish NLI and STS tasks; second, we convert them into ColBERT-style retrievers using PyLate, trained on MS MARCO-TR. We evaluated 10 models across five diverse Turkish BEIR datasets spanning scientific, financial, and argumentative domains. Notably, these models show exceptional parameter efficiency: our 1.0M parameter colbert-hash-nano-tr model is 600 times smaller than the 600M parameter turkish-e5-large dense encoder, yet retains over 71% of its average mAP performance. Our results show that 3-5x smaller late-interaction models significantly outperform dense encoders, with ColmmBERT-base-TR achieving up to +13.8% improvement in mAP on domain-specific tasks. To assess production-readiness, we benchmarked indexing algorithms. Our analysis shows that MUVERA+Rerank is, on average, 3.33x faster than the standard PLAID index while also achieving a +1.7% relative improvement in mAP. This demonstrates that efficient, low-latency systems are practical, with models like ColmmBERT-base-TR achieving query times as low as 0.54 ms using MUVERA. We publicly release all model checkpoints, training configurations, and evaluation scripts to facilitate future Turkish IR research. However, our study is limited to moderately sized datasets (≤ 50K documents) and translated benchmarks, which may not fully reflect real-world Turkish retrieval scenarios. Additionally, while MUVERA-based retrieval shows promising latency, further evaluation is needed to assess scalability on large-scale production systems.
Ezerceli et al. (Thu,) studied this question.