Reconstructing Static Memories from the Brain with EEG Feature Extraction and Generative Adversarial Networks
Keywords:Convolutional Neural Networks, General Adversarial Networks, EEG, Image Reconstruction, Machine Learning
Forensic tools and facial recognition techniques have implemented state memory reconstruction in the court of law. Recent research has incorporated machine learning into this field in order to fulfill the rising demand for this service. However, the suggested solutions require expensive cutting-edge equipment such as fMRI and CT scanners. The goal of the present study is to recreate images using user electroencephalography (EEG) input patterns. We combined a discriminator and generator network in order to decrease blur, face distortion, and erroneous features in the ImageNet dataset. An EEG feature matrix was produced using a convolutional neural network (CNN) encoder from spectrogram inputs, which are visual representations of the spectrum of electrical frequencies observed at each electrode. The encoder produced an 85% accuracy when linking spectrograms to the labels of the relevant images. The feature matrix was passed into the GAN during training. The viability of image reconstruction and coherent image representation from the brain were both proved feasible to a dependable degree due to our GAN output images’ resemblance to the original dataset. The present work could find applications in future studies as a non-invasive, more affordable option to recreate memories from brain signals.
References or Bibliography
Shimizu, H., & Srinivasan, R. (2022, January 1). Improving classification and reconstruction of imagined images from EEG signals. bioRxiv. Retrieved July 5, 2022, from https://www.biorxiv.org/content/10.1101/2022.06.01.494379v1.full#ref-10
G. Shen, K. Dwivedi, K. Majima, T. Horikawa, and Y. Kamitani, “End-to-End Deep Image Reconstruction From Human Brain Activity,” Frontiers in Computational Neuroscience, vol. 13. Frontiers Media SA, Apr. 12, 2019. doi: 10.3389/fncom.2019.00021.
R. Alazrai, A. Al-Saqqaf, F. Al-Hawari, H. Alwanni, and M. I. Daoud, “A Time-Frequency Distribution-Based Approach for Decoding Visually Imagined Objects Using EEG Signals,” IEEE Access, vol. 8, pp. 138955–138972, 2020, doi: 10.1109/ACCESS.2020.3012918.
P. Bobrov, A. Frolov, C. Cantor, I. Fedulova, M. Bakhnyan, and A. Zhavoronkov, “Brain-Computer Interface Based on Generation of Visual Images,” PLoS ONE, vol. 6, no. 6, p. e20674, Jun. 2011, doi: 10.1371/journal.pone.0020674.
S. Palazzo, C. Spampinato, J. Schmidt, I. Kavasidis, D. Giordano, and M. Shah, “Correct block-design experiments mitigate temporal correlation bias in EEG classification,” arXiv:2012.03849 [cs, q-bio], Nov. 2020, Accessed: Jul. 05, 2022. [Online]. Available: https://arxiv.org/abs/2012.03849
V. Sorin, Y. Barash, E. Konen, and E. Klang, “Creating Artificial Images for Radiology
Applications Using Generative Adversarial Networks (GANs) – A Systematic Review,” Academic Radiology, vol. 27, no. 8. Elsevier BV, pp. 1175–1185, Aug. 2020. Doi: 10.1016/j.acra.2019.12.024.
S. Wakita, T. Orima, and I. Motoyoshi, “Photorealistic reconstruction of visual texture from EEG signals,” Frontiers, 01-Jan-1AD. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fncom.2021.754587/full. [Accessed: 05-Jul-2022].
T. A. Izzuddin, N. M. Safri, and M. A. Othman, “Compact convolutional neural network (CNN) based on SincNet for end-to-end motor imagery decoding and analysis,” Biocybernetics and Biomedical Engineering, vol. 41, no. 4. Elsevier BV, pp. 1629–1645, Oct. 2021. doi: 10.1016/j.bbe.2021.10.001.
K. Ogórek, P. Poryzała, και P. Strumiłło, ‘EEG Based Image Reconstruction Using Transformers’, Biocybernetics and Biomedical Engineering – Current Trends and Challenges, 2021.
D. Vivancos, MindBigData "IMAGENET" of The Brain, 30-Jun-2022. [Online]. Available: http://www.mindbigdata.com/opendb/imagenet.html. [Accessed: 07-Jul-2022].
Development and validation of an EEG-based real-time emotion recognition system using edge ai computing platform with Convolutional Neural Network System-on-chip design. IEEE Xplore. (n.d.). Retrieved July 7, 2022, from https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8890664&tag=1.
How to Cite
Copyright (c) 2023 Matthew Zhang
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright holder(s) granted JSR a perpetual, non-exclusive license to distriute & display this article.