Machine learning systems must adapt when the conditions under which they are deployed differ from the conditions under which they were trained or calibrated. This dissertation studies adaptation across two settings that are usually treated separately. The first is domain adaptation under distribution shift, where the central question is how to transfer across multiple related domains without collapsing their structure. The second is in-context learning in large language models, where a pretrained model must specialize to new tasks or long contexts under strict computational constraints. The dissertation develops five methods organized into these two lines of work. For structured domain adaptation, it introduces Graph-Relational Domain Adaptation, which aligns domains according to an explicit graph; Taxonomy-Structured Domain Adaptation, which preserves hierarchical relations among domains; and Variational Domain Indexing, which infers interpretable domain structure directly from data when no explicit structure is provided. For efficient in-context learning in large language models, it introduces Implicit In-Context Learning, which replaces explicit demonstrations with activation-space context vectors, and K-Token Merging, which compresses contiguous token embeddings into shorter latent representations for efficient long-context processing. Together, these chapters argue that adaptation is a broad framework that includes adaptation across structured domains, adaptation with inferred domain identities, and adaptation through efficient in-context conditioning under efficiency constraints.
Zihao Xu (Thu,) studied this question.