Preliminary Results of the MLPerf BERT Inference Benchmark on AMD Instinct GPUs

In recent years, Artificial Intelligence (AI) has reshaped various facets of our day-to-day lives. To evaluate both hardware and software deployment of Machine Learning (ML) applications, it is necessary to measure system-wide performance and accuracy. MLPerf, a Machine Learning Benchmark from MLCommons, provides a standard for measuring Machine Learning Performance on hardware. To our knowledge, there is little known public effort on porting and optimizing these inference benchmark models on AMD GPUs. This paper focuses on the effort to port MLPerf BERT, one of the MLPerf Inference Benchmarks, to run on AMD Instinct MI210 and MI250 Accelerators. We describe the challenges encountered, solutions applied, and the successful porting of the benchmark resulting in submission of the code to the MLPerf community codebase. This paper derives from unpublished work completed as part of the 2023 Student Cluster Competition (SCC23). We present the first unofficial results for running the MLPerf BERT Inference Benchmark using our optimization strategies on AMD accelerators, specifically the MI250 and MI210. Additionally, as a result of this effort, the CM Automation framework now supports AMD ROCm for PyTorch and ONNX Runtime platforms.