Distributed Neural Networks using TensorFlow over Multicore and Many-Core Systems

This paper focuses on distributed deep learning models that simulate the HAR (Human Activity Recognition) data set from the UCI machine learning Repository. The proposed deep learning LSTM (Long Short-Term Memory) model works with the TensorFlow framework using the Python 3 programming language which supports the distributed architecture. In order to simulate the distributed deep learning models over different multicore and many-core systems, two hardware platforms are built; the first one is equipped with a Raspberry Pi cluster with 16 Pi 3 model B+ boards which each having 1 GB of RAM and 32 GB flash storage. The second platform is houses an Octa-core Intel Xeon CPU system with a 16MB Cache, 32 GB RAM and 2 TB SSD primary storage with 10 TB HDD secondary storage. In this paper, the performance of the distributed LSTM model over multicore and many-core systems is presented in terms of execution speed and efficiency of prediction accuracy upon varying number of deep layers with corresponding hidden nodes. In this experiment, a 3 x 3 distributed LSTM model has been used, which furnishes higher prediction accuracy with faster computation time than the models that different number of layers provide.