AutoUMA: Accelerated ML Compiler Support for Hardware Accelerators based on Automatic Kernel Library Generation

The rapid development of domain-specific deep neural network (DNN) accelerators, particularly spatial architectures such as systolic arrays, has become essential for improving power–performance efficiency in edge computing. However, despite advances in hardware design tools, deploying deep learning models on custom spatial accelerators remains a major bottleneck. Existing machine learning (ML) compilers are primarily designed for GPUs and CPUs, making the integration of new accelerators complex and labor-intensive due to diverse programming interfaces and architecture-specific execution models. This lack of a unified deployment interface limits the scalability and generality of current compiler toolchains. In this work, we address this challenge by shifting the abstraction from accelerator-specific interfaces to kernel functions, the fundamental building blocks for executing DNN workloads. We present AutoUMA, an automated compilation framework that externalizes kernel generation and introduces an AutoScheduler to automatically derive optimized kernels from accelerator mappings. The scheduler performs structured transformations, including multi-level loop tiling, memory transfer staging, and instruction emission, and incorporates a guard mechanism to eliminate redundant memory operations. In addition, we introduce AutoUMA ’s compiler retargeting pipeline that leverages kernel functions as a uniform, coarse-grained deployment interface and abstracts low-level modifications through high-level accelerator descriptions. We evaluate our framework on the Gemmini accelerator and demonstrate that it significantly reduces integration effort while generating efficient kernel libraries. Our approach achieves speedups of up to 4.4× over manually developed static libraries in low-level C and delivers performance competitive with expert-optimized kernel implementations using high-level scheduling languages.