Performance Analysis of Various Activation Functions in Generalized MLP Architectures of Neural Networks

The activation function used to transform the activation level of a unit (neuron) into an outputsignal. There are a number of common activation functions in use with artificial neural networks(ANN). The most common choice of activation functions for multi layered perceptron (MLP) isused as transfer functions in research and engineering. Among the reasons for this popularity areits boundedness in the unit interval, the function’s and its derivative’s fast computability, and anumber of amenable mathematical properties in the realm of approximation theory. However,considering the huge variety of problem domains MLP is applied in, it is intriguing to suspect thatspecific problems call for single or a set of specific activation functions. The aim of this study is toanalyze the performance of generalized MLP architectures which has back-propagation algorithmusing various different activation functions for the neurons of hidden and output layers. Forexperimental comparisons, Bi-polar sigmoid, Uni-polar sigmoid, Tanh, Conic Section, and RadialBases Function (RBF) were used.