AUTHOR=Habib Gousia , Qureshi Shaima 

TITLE=GAPCNN with HyPar: Global Average Pooling convolutional neural network with novel NNLU activation function and HYBRID parallelism

JOURNAL=Frontiers in Computational Neuroscience

VOLUME=Volume 16 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2022.1004988

DOI=10.3389/fncom.2022.1004988

ISSN=1662-5188

ABSTRACT=Abstract: With the increasing demand for deep learning in the last few years, CNNS haven been widely used in many
application areas and gained interest in many classifications, regression, and image recognition tasks. The training of
these deep neural networks is compute-intensive and takes even days or weeks to train the model from scratch. The
compute-intensive nature of these deep neural networks sometimes limits the practical implementation of the CNNS in
real-time applications. Therefore, the computational speedup in these networks is of utmost importance, which gains
interest towards DNN acceleration. Much research is going on to meet the computational requirement and make its
feasibility for real-time applications. Because of simplicity, data parallelism is used primarily, but it performs worst
sometimes. In most cases, researchers prefer model parallelism instead of data parallelism, but it isn’t always the best
choice. Therefore, in this work, we will implement a hybrid of both data and model parallelism for improving the
computational speed without compromising accuracy. There is only a 1.5% accuracy drop in our proposed work with an
increased speed up of 3.62X. Also, the novel activation function Normalized nonlinear activation unit NNLU is proposed,
to introduce the nonlinearity in the model. The activation unit is non-saturated and helps avoid the model’s over-fitting.
The activation unit is free from the vanishing gradient problem. Also in the proposed CNN model, the fully connected
The layer is entirely replaced by the Global average pooling layers (GAP) to have enhanced accuracy and computational
performance of the model. The model, when tested on a bio-medical image data-set, achieved an accuracy of 98.89% and
required a training time of only 1 sec. The model performs the classification of medical images into different categories of
Tumours such as glioma, meningioma, and pituitary tumours. The model is compared with existing network models and
is observed that the proposed model outperforms as far as classification accuracy and computational speed is concerned.
Also, the accuracy results are observed on the varying optimizer, a varying range of learning rates, and several epochs.