Deep Residual Networks with Exponential Linear Unit

The depth of convolutional neural networks is a crucial ingredient for reduction in test errors on benchmarks like ImageNet and COCO. However, training a neural network becomes difficult with increasing depth. Problems like vanishing gradient and diminishing feature reuse are quite trivial in very deep convolutional neural networks.