Mathematical Epistemology - On the nature of language

Key Points

• The research aims to investigate how language syntax and semantics interact through deep learning models.
• Utilized implicit deep learning models to learn Q-K-V embeddings and contextual token representations.
• Examined six types of parameterization to characterize interactions between syntax and semantics.
• Focused on capturing both syntactic co-occurrence and semantic alignment.
• Identified different cognitive regimes based on the parameterizations of syntax and semantics interactions.
• Demonstrated a nuanced understanding of how language features can be learned together via embeddings.