The CNN: The Architecture Behind Artificial Intelligence Development
DOI:
https://doi.org/10.47611/jsrhs.v12i4.5579Keywords:
Convolutional Neural Network, Convolutional Layer, Transfer Function, Artificial Intelligence, Machine LearningAbstract
The convolutional neural network (CNN) is a multilayer network architecture that is capable of training itself using an advanced algorithm to produce increasingly accurate results. The CNN is especially effective in spatial image recognition and is used in a multitude of fields, such as image recognition in the medical industry, and image segmentation in the security field. The ability of the CNN to show impressive results is seen in its multilayer composition.
This multilayer network consists of the convolutional layer, the pooling layer, the activation layer, and the fully connected layer. Here, the convolutional layer, the pooling layer, and the activation layer have their own parameters which will influence the flow of input data throughout the CNN until it produces an ultimate output in the connected layer. As such, this paper will delve into the individual characteristics of each layer, and introduce its relationship with not only its immediate surrounding layers but with the entirety of the CNN. Additionally, this paper will stress important modules and parameters that can help improve the layers.
Downloads
References or Bibliography
Ali, S. H., Al-Sultan, H. A., & Al Rubaie, M. T. (2022). Fifth industrial revolution. International Journal of Business, Management and Economics, 3(3), 196–212. https://doi.org/10.47747/ijbme.v3i3.694
Assael, Y., Sommerschield, T., Shillingford, B., Bordbar, M., Pavlopoulos, J., Chatzipanagiotou, M., Androutsopoulos, I., Prag, J., & de Freitas, N. (2022). Restoring and attributing ancient texts using deep neural networks. Nature, 603(7900), 280–283. https://doi.org/10.1038/s41586-022-04448-z
Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., Santamaría, J., Fadhel, M. A., Al-Amidie, M., & Farhan, L. (2021). Review of Deep Learning: Concepts, CNN Architectures, challenges, applications, Future Directions. Journal of Big Data, 8(1). https://doi.org/10.1186/s40537-021-00444-8
Callaghan, C. (2022). The Fifth Industrial Revolution: An Unfolding Knowledge Productivity Revolution. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4307753
Campbell, M., Hoane, A. J., & Hsu, F (2002) Deep Blue, Artificial Intelligence, 134(2), 57-83,, https://doi.org/10.1016/S0004-3702(01)00129-1.
Carbone, M. R. (2022). When not to use machine learning: A perspective on potential and limitations. MRS Bulletin, 47(9), 968–974. https://doi.org/10.1557/s43577-022-00417-z
Crafts, N. F. R (1996). The First Industrial Revolution: A Guided Tour for Growth Economists. The American Economic Review, 86(2), 197–201. http://www.jstor.org/stable/2118122
Deiana, A. M., Tran, N., Agar, J., Blott, M., Di Guglielmo, G., Duarte, J., Harris, P., Hauck, S., Liu, M., Neubauer, M. S., Ngadiuba, J., Ogrenci-Memik, S., Pierini, M., Aarrestad, T., Bahr, S., Becker, J., Berthold, A.-S., Bonventre, R. J., Bravo, T. E. M., … Weng, O. (2021, October 25). Applications and techniques for fast machine learning in science. arXiv.org. https://arxiv.org/abs/2110.13041
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022). Activation functions in Deep learning: A comprehensive survey and benchmark. Neurocomputing, 503, 92–108. https://doi.org/10.1016/j.neucom.2022.06.111
Ford, P. (2020, April 2). Our fear of Artificial Intelligence. MIT Technology Review. https://www.technologyreview.com/2015/02/11/169210/our-fear-of-artificial-intelligence/
Geetha, T. V., & Sendhilkumar, S. (2023). Machine learning applications. Machine Learning, 295–315. https://doi.org/10.1201/9781003290100-12
Gassenmaier, S., Küstner, T., Nickel, D., Herrmann, J., Hoffmann, R., Almansour, H., Afat, S., Nikolaou, K., & Othman, A. E. (2021). Deep learning applications in Magnetic Resonance Imaging: Has the future become present? Diagnostics, 11(12), 2181. https://doi.org/10.3390/diagnostics11122181
Gholamalinezhad, H., & Khosravi, H. (2020, September 16). Pooling methods in Deep Neural Networks, a review. arXiv.org. https://arxiv.org/abs/2009.07485
Groumpos, P. P. (2021). A critical historical and scientific overview of all industrial revolutions. IFAC-PapersOnLine, 54(13), 464–471. https://doi.org/10.1016/j.ifacol.2021.10.492
IBM100 - Deep Blue. (2023). https://www.ibm.com/ibm/history/ibm100/us/en/icons/deepblue/
Hao, W., Yizhou, W., Yaqin, L., & Zhili, S. (2020). The role of activation function in CNN. 2020 2nd International Conference on Information Technology and Computer Application (ITCA). https://doi.org/10.1109/itca52113.2020.00096
Hashemi, M. (2019). Enlarging smaller images before inputting into convolutional neural network: Zero-padding vs. interpolation. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0263-7
Islam, M. A., Wimmer, H., & Rebman, C. M. (2021). Examining sigmoid vs relu activation functions in deep learning. Interdisciplinary Research in Technology and Management, 432–437. https://doi.org/10.1201/9781003202240-68
Lewis, B. (2018, February 8). Predictive maintenance for more resilient self-service vending. https://www.insight.tech/retail/predictive-maintenance-for-more-resilient-self-service-vending
Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., Hubert, T., Choy, P., de Masson d’Autume, C., Babuschkin, I., Chen, X., Huang, P.-S., Welbl, J., Gowal, S., Cherepanov, A., … Vinyals, O. (2022). Competition-level code generation with AlphaCode. Science, 378(6624), 1092–1097. https://doi.org/10.1126/science.abq1158
Mahima, R., Maheswari, M., Roshana, S., Priyanka, E., Mohanan, N., & Nandhini, N. (2023). A comparative analysis of the most commonly used activation functions in deep neural network. 2023 4th International Conference on Electronics and Sustainable Communication Systems (ICESC). https://doi.org/10.1109/icesc57686.2023.10193390
Malallah, H., Zeebaree, S. R., Zebari, R. R., Sadeeq, M. A., Ageed, Z. S., Ibrahim, I. M., Yasin, H. M., & Merceedi, K. J. (2021). A comprehensive study of kernel (issues and concepts) in different operating systems. Asian Journal of Research in Computer Science, 16–31. https://doi.org/10.9734/ajrcos/2021/v8i330201
Malleswaran, M., Vaidehi, V., & Deborah, S. A. (2011). CNN based GPS/INS data integration using new dynamic learning algorithm. 2011 International Conference on Recent Trends in Information Technology (ICRTIT). https://doi.org/10.1109/icrtit.2011.5972270
Milosevic, N. (2020). Convolutions and Convolutional Neural Networks. Introduction to Convolutional Neural Networks. https://doi.org/10.1007/978-1-4842-5648-0_12
Mohajan, H. (2019, October 21). The Second Industrial Revolution has brought modern social and economic developments. Munich Personal RePEc Archive. https://mpra.ub.uni-muenchen.de/98209/
Mohajan, H. (2021). Third Industrial Revolution Brings Global Development. 7. 239-251.
Monkam, P., Qi, S., Ma, H., Gao, W., Yao, Y., & Qian, W. (2019). Detection and classification of pulmonary nodules using Convolutional Neural Networks: A survey. IEEE Access, 7, 78075–78091. https://doi.org/10.1109/access.2019.2920980
Nedeljkovic, D., & Jakovljevic, Z. (2022). CNN based method for the development of cyber-attacks detection algorithms in industrial control systems. Computers & Security, 114, 102585. https://doi.org/10.1016/j.cose.2021.102585
Newborn, M. (1997). Kasparov versus Deep Blue. https://doi.org/10.1007/978-1-4612-2260-6
Noble, S. M., Mende, M., Grewal, D., & Parasuraman, A. (2022). The Fifth Industrial Revolution: How Harmonious Human–machine collaboration is triggering a retail and service [r]evolution. Journal of Retailing, 98(2), 199–208. https://doi.org/10.1016/j.jretai.2022.04.003
Onyema, E. M., Almuzaini, K. K., Onu, F. U., Verma, D., Gregory, U. S., Puttaramaiah, M., & Afriyie, R. K. (2022). Prospects and challenges of using machine learning for academic forecasting. Computational Intelligence and Neuroscience, 2022, 1–7. https://doi.org/10.1155/2022/5624475
Özdemir, C. (2023). Avg-topk: A new pooling method for Convolutional Neural Networks. Expert Systems with Applications, 223, 119892. https://doi.org/10.1016/j.eswa.2023.119892
Prasad, P. S., & R. Upadhyay, A. (2012). Design of hybrid kernel and the performance improvement of the operating system. International Journal of Engineering and Technology, 4(2), 162–165. https://doi.org/10.7763/ijet.2012.v4.340
Sarker, I.H. AI-Based Modeling: Techniques, Applications and Research Issues Towards Automation, Intelligent and Smart Systems. Sn Computer Science. 3, 158 (2022). https://doi.org/10.1007/s42979-022-01043-x
Shaheen, F., Verma, B., & Asafuddoula M. (2016). Impact of Automatic Feature Extraction in Deep Learning Architecture, 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Gold Coast, QLD, Australia, 2016, pp. 1-8, doi: 10.1109/DICTA.2016.7797053.
Sharma, N., & Jakovljevic, Z. (2018, June 8). An analysis of convolutional neural networks for Image Classification. Procedia Computer Science. https://www.sciencedirect.com/science/article/pii/S1877050918309335
Schmelzer, R. (2022, October 12). Should we be afraid of ai?. Forbes. https://www.forbes.com/sites/cognitiveworld/2019/10/31/should-we-be-afraid-of-ai/?sh=344b91414331
Schmidt, J., Marques, M. R., Botti, S., & Marques, M. A. (2019). Recent advances and applications of machine learning in solid-state materials science. Npj Computational Materials, 5(1). https://doi.org/10.1038/s41524-019-0221-0
Tan, T.-B., & Shang-su, W. (2017). The Fourth Industrial Revolution Explained. In PUBLIC POLICY IMPLICATIONS OF THE FOURTH INDUSTRIAL REVOLUTION FOR SINGAPORE (pp. 5–7). S. Rajaratnam School of International Studies. http://www.jstor.org/stable/resrep17650.5
Taye, M. M. (2023). Theoretical Understanding of Convolutional Neural Network: Concepts, Architectures, Applications, Future Directions. Computation, 11(3), 52. https://doi.org/10.3390/computation11030052
Thakur, D. (2019, September 4). ML Basics #4: Replace negatives with Zeros!. dhruvs space RSS. https://dhruvs.space/posts/ml-basics-issue-4/
Theodoridis, S. (2015). Parameter learning. Machine Learning, 327–402. https://doi.org/10.1016/b978-0-12-801522-3.00008-2
The Strategic Foresight Initiative. (2013). The Third Industrial Revolution. In Envisioning 2030: US Strategy for the Coming Technology Revolution (pp. 15–22). Atlantic Council. http://www.jstor.org/stable/resrep03584.8
Varoquaux, G., & Cheplygina, V. (2022). Machine Learning for Medical Imaging: Methodological Failures and recommendations for the future. Npj Digital Medicine, 5(1). https://doi.org/10.1038/s41746-022-00592-y
Wandelt, B. D., & Bailer-Jones, C. A. L. (2008). Precision parameter estimation and machine learning. AIP Conference Proceedings. https://doi.org/10.1063/1.3059073
Xu, C., & Wang, H. (2022). Research on a convolution kernel initialization method for speeding up the convergence of CNN. Applied Sciences, 12(2), 633. https://doi.org/10.3390/app12020633
Zafar, A., Aamir, M., Mohd Nawi, N., Arshad, A., Riaz, S., Alruban, A., Dutta, A. K., & Almotairi, S. (2022). A comparison of pooling methods for Convolutional Neural Networks. Applied Sciences, 12(17), 8643. https://doi.org/10.3390/app12178643
Published
How to Cite
Issue
Section
Copyright (c) 2023 Yechan Lee
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright holder(s) granted JSR a perpetual, non-exclusive license to distriute & display this article.
