Edge-Fog-Cloud systems utilize offloading to shift workload from less powerful edge nodes to more powerful central nodes in fog and cloud. However, existing approaches for offloading decisions rely on simplified execution time estimations, such as static values or linear time-per-byte (TPB) approximations. However, many tasks show a highly variable execution time behavior, with more complicated relationships to the original request, leading to suboptimal scheduling of tasks and hardware usage.We propose a method for predicting the execution time for different execution nodes, based on a lightweight AI model. The approach is based on a multi-output feedforward neural network, able to predict the execution time for all heterogeneous compute nodes in a single pass. Numerical and categorical features are extracted from the request as model input. The resulting prediction was then integrated into a cost model for the final offloading decision. The proposed concept was then evaluated on different task categories: Tasks with a more linear relationship, such as applying an image filter, tasks with a more complex relationship, such as GPS routing, as well as hybrid complexity tasks, where the execution time is not only influenced by the request size, but also more complex relationships. For tasks with a hybrid relationship, the proposed approach was able to decrease the overall offloading cost by 14.66% compared to using TPB approximation. This includes the sub 0.3ms prediction latency. Even for tasks whose execution time depends nearly linearly on the request size, the proposed approach was equal to TPB. On a mixed-task dataset, the overall cost was decreased by 5.59% compared to TPB.
Kreutzer et al. (Wed,) studied this question.