FinOps Governance and Carbon-Aware Workload Scheduling for Hyperscale AI Training

Training and fine-tuning foundation models across hyperscale cloud data centers consumes megawatt-hours of electrical energy and generates millions of dollars in monthly cloud infrastructure bills. Historically, enterprise machine learning orchestration treated cloud resource provisioning and sustainability accounting as post-hoc reporting exercises divorced from real-time cluster scheduling. As a result, delay-tolerant batch training jobs, hyperparameter sweeps, and nightly distillation pipelines routinely execute on expensive on-demand GPU instances during peak regional electricity grid demand powered by fossil-fuel peaker plants. This paper presents a joint FinOps and Carbon-Aware (GreenOps) Kubernetes scheduling architecture for multi-region cloud AI clusters. Integrating real-time marginal operating emissions rates (MOER) from regional power grids (Sukprasert et al., 2024; Wiesner et al., 2021) with dynamic cloud spot-instance interruption hazard models and asynchronous fast-checkpointing (Mohan et al., 2021; Thorpe et al., 2023), the scheduler formulates job placement as a constrained Pareto optimization over monetary cost, carbon emissions, and deadline slack. We validate the architecture through an in-depth production case study combining Microsoft Azure Carbon-Aware Kubernetes scheduling patterns and Spotify Cloud FinOps unit-economics telemetry across three North American and European cloud regions. Over a 90-day evaluation spanning 18,400 GPU-days of PyTorch FSDP and DeepSpeed workloads, our scheduler reduces net cloud GPU expenditure by 46.8% through hazard-aware spot multiplexing, lowers Scope 2 operational carbon emissions by 38.4% via spatiotemporal workload shifting, and maintains a 99.4% training deadline completion rate with less than 2.1% checkpoint recovery overhead. Research Domain & Specialization Track Master of Science in Information Technology (MSIT) — Cloud FinOps, Sustainable Computing, and AI Cluster Scheduling. Core Technical Highlights Empirical Production Case Studies: Rigorous architectural blueprints and telemetry evaluation across enterprise cloud deployments. Four-Tier Reference Architecture: Formal specification from edge ingress to distributed hardware execution pools. Mathematical Formulation: Formally derived algorithmic and optimization models with closed-loop verification. Multi-Modal Benchmarking: Comparative empirical baselines under fault-injection and synthetic workload scaling. Author Information Author: Sonu Kumar SinghRole: Senior Consultant Affiliation: Independent Researcher Publication Status: Preprint / Working Paper (MSIT AI & Cloud Systems Research Series) Reproducibility & Open Science All included architecture diagrams, telemetry schemas, and benchmark curves are open-access under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.