ADaPS-FC: Enhancing Performance Through Channel Division on Edge Computing Platforms

The proliferation of Artificial Intelligence (AI) applications has driven a substantial demand for deploying Neural Networks (NNs) across a wide range of device platforms. Modern networks have become so computationally intensive that even relatively simple Convolution Neural Networks (CNNs) often require costly specialized hardware for efficient execution. To mitigate this challenge, techniques that distribute inference workloads across interconnected, resource-constrained devices have become increasingly important. While existing approaches typically rely on empirical models or support only limited partitioning dimensions, we introduce ADaPS-FC, a novel framework for optimally distributing CNN inference workloads across heterogeneous embedded devices. Our analytical model partitions the height, width, and channel dimensions of 4D CNN tensors while also exploring partition configurations that enable layer fusion. In addition, it introduces weight-division strategies for fully connected layers and small output operators to improve both memory usage and latency, fully accounting for each device’s computational capabilities and inter-device communication overhead. This work extends the previously proposed ADaPS method by addressing the limitations encountered in the later, non-divisible layers of CNN models. To efficiently navigate the large search space of possible partitionings, ADaPS-FC employs a hybrid optimization algorithm that combines Alpha–Beta pruning with dynamic programming for the earlier layers, where height or width division is feasible. For the later layers, where the prior method was unable to apply spatial partitioning, the framework introducess weight (channel) division to enable further distribution opportunities. We evaluate ADaPS-FC across multiple CNN architectures deployed on interconnected heterogeneous hardware, using configurations ranging from two to four devices. Experimental results show that ADaPS-FC improves inference time by 1.3 × on average over previous state-of-the-art.