This paper presents JoeyLLM, an initiative to build a regionally aligned foundation model for Australian infrastructure. Early experiments with fine-tuning existing models revealed that cultural alignment cannot be achieved through data alone, but requires control over architecture, training, and deployment. We document the transition from adapting open models to building a model from scratch, and propose a framework for developing Civic Foundation Models (CFMs) for sovereign AI.
Altenburg et al. (Wed,) studied this question.