Fractional Optimizers Meet Fractal Activation Functions: An Empirical Study of Multi-Scale Optimization in Neural Networks

Fractional optimization methods and fractal activation functions are two independent directions for improving neural network training. Fractional optimizers extend first-order optimization through fractional derivatives and memory effects, whereas fractal activations introduce multi-scale nonlinear representations based on self-similar Weierstrass- and Blancmange-type functions. Here, we investigate their interaction within a unified experimental framework. We evaluate fractional optimizer families on Ackley and Himmelblau benchmark surfaces, in standard form and with additive Weierstrass-type perturbations, and then in feed-forward neural networks with conventional and fractal activations on ten classification datasets. The comparison includes standard methods, Herrera-type local scaling optimizers, explicit and adaptive memory-based fractional optimizers, and other representative literature methods. Fractional optimization and fractal activations show useful but selective pairings. Herrera-type local fractional scaling performs well with selected fractal activations in network training, while explicit Grünwald–Letnikov memory is competitive on the controlled surfaces without being preferentially favoured by the perturbation: the unmodified baselines rank first on both additive variants. Adaptive memory improves plain memory substitution in several cases, supporting controlled fractional memory as a promising direction rather than a universal replacement.