Machine learning models are increasingly adapted in settings where supervision is incomplete, biased, or indirect, and where data distributions, tasks, and behavioral requirements evolve. In such regimes, optimizing the task objective alone often leads to spurious shortcut learning, catastrophic forgetting, or undesirable behavioral drift. This thesis studies how introducing additional structure into the learning process, beyond standard empirical risk minimization, enables more reliable model adaptation across vision and language domains. In visual recognition, the thesis develops structured representation learning methods that improve data efficiency, interpretability, and robustness. We introduce a weakly supervised framework for learning spatially grounded semantic representations via alignment and disentanglement regularization, improving few-shot generalization without requiring strong localization supervision. We further address incidental correlation learning in part-based models by explicitly separating foreground and background factors and enforcing invariance, thereby improving robustness to background shifts and domain perturbations. The thesis also proposes a replay-free continual object detection framework that mitigates catastrophic forgetting and prevents confusion between previously learned objects and the background during incremental updates. Finally, the thesis extends these principles to foundation model fine-tuning. It introduces an alignment-aware fine-tuning framework that incorporates feedback from external verification models through a policy-gradient–based regularization term, enabling control over behavioral properties such as safety and factuality while preserving task performance. Together, these contributions demonstrate that reliable adaptation requires learning signals beyond task loss, enabling robust, interpretable, and aligned models under distribution shift and repeated fine-tuning.
Gaurav Bhatt (Thu,) studied this question.