Fractals in Neural Networks: Introducing a New Class of Activation Functions

We introduce a class of activation functions for neural networks based on the Blancmange curve and the Weierstrass function. These activation functions are designed to be qualitatively similar to standard functions such as tanh, while incorporating fractal, self-affine, and self-similar patterns. The goal is to force the neural network to produce non-smooth outputs, preventing excessive smoothing in predictions and thus increasing expressivity.We evaluate this new class of activation functions on standard classification problems and demonstrate how fractal properties of activation functions influence the expressivity of neural networks.