Context-aware transformer models for accounting-based inventory prediction and supply chain cost optimization

Accounting for increasing complexity in supply chain operations, predictive analytics is essential for optimizing costs and managing inventory efficiently. However, limitations in conventional machine learning (ML) and deep learning (DL) models hinder their ability to capture non-linear dependencies in high-dimensional tabular data. This study aims to overcome these challenges by employing transformer-based models enhanced with attention mechanisms for robust feature extraction and meta-learning. In this work, transformer-based tabular learning is studied to predict inventory and optimize cost with mixed operational and financial variables as driven by accounting to supply chain analytics. Conventional machine learning frameworks do not tend to capture nonlinear interactions between categorical drivers of cost, demand fluctuation and time. To deal with this we compare TabTransformer and FT-Transformer architectures to the baseline models such as Random Forests, gradient boosting and recurrent neural networks. The outcomes of the experiments demonstrate that the models based on transformers always work better than the traditional methods. FT-Transformer performs best in terms of inventory prediction (RMSE = 4.44, MAE = 2.95, R2 = 0.94), becoming much better than tree-based and recurrent baselines. Transformer Encoder + MLP model provides the best results in cost optimization (RMSE = 2.60, MAE = 4.78, R2 = 0.98) and the best representation of accounting-driven cost relationships. The robustness is confirmed by cross-validation, sensitivity analysis, and ablation tests which determine how the model behaves when its parameters are changed. SHAP and LIME interpretability analysis highlight important cost and inventory drivers, which can be used to act in decision-making. These results suggest that transformer-based tabular models offer viable, consistent, and explainable solutions to current supply chain analytics and still have a practical implementation level.