Cooking in the Dark: Performance Evaluation of Matrix-Vector Operations in Neural Networks under Homomorphic Encryption

Homomorphic Encryption (HE) is an emerging technology that enables Privacy-Preserving Machine Learning (PPML). By providing the capability to compute onencrypted data, it facilitates the evaluation of Neural Networks (NNs) in client-server scenarios, where sensitive data is sent to an untrusted server for inference.As NNs rely heavily on matrix–vector multiplications, we evaluate several naive methods for implementing these operations within the constraints of HE. We demonstrate that the diagonal method is a highly versatile approach and the fastest overall. However, the traditional row-wise method offers the lowest memory footprint, making it suitable for memory-constrained environments.