What question did this study set out to answer?

This work aims to audit the performance of FP16 arithmetic in Apple's Neural Engine to identify and address silent numerical failures.

June 1, 2026Open Access

Silent Numerical Failures in On-Device ML Converters: A Systematic Audit of FP16 Overflow in Apple Neural Engine Deployment

Key Points

This work aims to audit the performance of FP16 arithmetic in Apple's Neural Engine to identify and address silent numerical failures.
Conducted a systematic audit of coremltools converter in Apple's Neural Engine.
Identified five specific operations with overflow issues during computation: softplus, mish, reduce_log_sum_exp, log_softmax, and logcumsumexp.
Developed and implemented numerically stable alternatives and conducted regression tests.
Five operations were found to silently yield incorrect results on ANE due to FP16 overflow.
Proposed numerically stable versions of operations were developed, enhancing the reliability of the coremltools.
The findings are critical for key AI models like YOLO, BERT, ViT, and MobileNetV3, affecting their deployment on Apple devices.

Abstract

Apple's Neural Engine (ANE) executes neural network operations in FP16 arithmetic, making it susceptible to overflow and underflow failures. This work presents a systematic audit of Apple's coremltools converter and identifies five operations that silently produce incorrect results on ANE: softplus, mish, reduceₗogₛumₑxp, logₛoftmax, and logcumsumexp. We derive numerically stable implementations, provide quantitative failure analysis, and submit fixes with regression tests to the coremltools repository. The findings impact major model families including YOLO, BERT, ViT, MobileNetV3, and CTC-based speech recognition systems.

Silent Numerical Failures in On-Device ML Converters: A Systematic Audit of FP16 Overflow in Apple Neural Engine Deployment

Key Points

Abstract

Cite This Study