Simplified Hardware Implementation of the Softmax Activation Function

In this paper a simplified hardware implementation of a CNN softmax layer is proposed. Initially the softmax activation function is analyzed in terms of required accuracy and certain optimizations are proposed. Subsequently the proposed hardware architecture is evaluated in terms of the introduced approximation error. Finally the proposed circuits are synthesized in a 90-nm 1.0 V CMOS standard-cell library using Synopsys Design Compiler. Comparisons reveal significant reduction up to 47% and 43% for certain cases, in terms of area × delay product over prior art. Area savings are achieved with no performance penalty.