ntvНаучно-технический вестник информационных технологий, механики и оптикиScientific and Technical Journal of Information Technologies, Mechanics and Optics2226-14942500-0373Университет ИТМО10.17586/2226-1494-2023-23-4-734-742ntv-203Research ArticleКОМПЬЮТЕРНЫЕ СИСТЕМЫ И ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИCOMPUTER SCIENCEСверхвысокое разрешение изображения магнитно-резонансной томографии головного мозга с использованием дискретного косинусного преобразования и сверточной нейронной сетиBrain MRT image super resolution using discrete cosine transform and convolutional neural networkhttps://orcid.org/0000-0002-8603-5954СингхП.SinghP.

Сингх Пуджа — магистр, исследователь

sc 57225030639

Нью-Дели, 110006

Pooja Singh — Magister, Researcher

sc 57225030639

New Delhi, 110006

pujasingh0409@gmail.comhttps://orcid.org/0000-0002-3720-8716ГанотраД.GanotraD.

Ганотра Динеш — PhD, доцент

sc 6506229541

Нью-Дели, 110006

Dinesh Ganotra — PhD, Associate Professor

sc 6506229541

New Delhi, 110006

dinesh_ganotra@hotmail.comДелийский технический университет Индиры Ганди для женщинИндияIndira Gandhi Delhi Technical University for WomenIndia202314122024234734742Copyright © Сингх П., Ганотра Д., 20242024Сингх П., Ганотра Д.Singh P., Ganotra D.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ntv.elpub.ru/jour/article/view/203

Изображения с высоким разрешением (High Resolution, HR) имеют широкое применение, например при проведении видеоконференций, дистанционного зондирования, медицинской визуализации и других. С использованием алгоритмов сверхвысокого разрешения появилась возможность решить несколько проблем магнитно-резонансной томографии изображений мозга, связанных с низкой чувствительностью, значительным частотным шумом, а также низким разрешением. Чтобы устранить данные проблемы, предложен метод улучшения качества сингулярного кадра на основе сверточной нейронной сети (Convolutional Neural Network, CNN) с дискретным косинусным преобразованием (Discrete Cosine Transform, DCT). Метод состоит из двух этапов, включающих обучение и тестирование. На этапе обучения изображения HR и низкого разрешения (Low Resolution, LR) используются в качестве входных данных и проходят предварительную обработку для создания блоков изображений. Для извлечения признаков из блоков LR и HR применены гистограмма и DCT. Извлеченным признакам присваивается идентификатор класса. CNN извлекает функции DCT, назначает идентификатор класса и получает свой экстрактор функций для окончательного ввода. Входное изображение LR на этапе тестирования повторно делится на блоки [2 × 2], с помощью гистограммы оценивается каждый блок и функции DCT. Каждый вектор признаков передается в нейронную сеть, полученные результаты сравниваются с набором векторов признаков, которые были записаны, в дополнение к идентификатору класса и назначены определенному вектору. Для генерации изображения сверхвысокого разрешения с изображением LR соответствующий блок HR заменяется на блок LR. Полученные результаты показали, что эффективность предложенного метода исходного набора данных достигла значений отношения пикового сигнала к шуму (PSNR) и среднеквадратичной ошибки (RMSE) 22,4 и 19,5 соответственно. Второй набор данных показал значения PSNR и RMSE, равные 20,1 и 25,5, а третий набор — 45,7 и 12,3. Таким образом, представленный метод работает лучше, чем нейронная сеть пространственной модуляции канала сверхвысокого разрешения и метод повышения разрешения.

High Resolution (HR) images have numerous applications, such as video conferencing, remote sensing, medical imaging, etc. Furthermore, a few challenges with the super resolution algorithms of magnetic resonance brain images are now obtainable, namely, low sensitivity, signifcant frequency noise as well as poor resolution. To fx these problems, a Convolutional Neural Network (CNN) based Discrete Cosine Transform (DCT) singular frame quality improvement method is described. There are two stages in this proposed method, involving training and testing. During the training stage, the HR, and Low Resolution (LR) pictures are employed as input, and they are preprocessed to create blocks of images. The histogram and DCT are used for extracting the features from the LR and HR blocks, and these extracted features are assigned with class id. The CNN, which extracts the features and allocates class id, receives its feature extractor as its fnal input. An LR input image is once more divided into [2 × 2] blocks during the testing stage, so each block histogram and DCT feature are estimated. Each feature vector is fed into the neural network as well as the results are contrasted with a set of feature vectors that have been recorded, in addition to the class id that has been allocated to a certain vector. In order to generate a Super resolution image with an LR image, a relevant HR block is then swapped out for this LR block. These results indicated that the initial dataset can achieve 22.4 and 19.5 Peak Signal to Noise Ratio (PSNR) and Root Mean Square Error (RMSE) values while measuring the effectiveness of this proposed method using RMSE and PSNR. Then, the second dataset illustrates that the PSNR and RMSE values are 20.1 and 25.5. For the third dataset, the values are 45.7 and 12.3, respectively. However, the presented method works better than the neural method of Super Resolution Channel Spatial Modulation Network and resolution enhancement technique.

высокое разрешениенизкое разрешениедискретное косинусное преобразованиеповышение разрешениясреднеквадратическая ошибкаPSNRсверточная нейронная сетьhigh resolutionlow resolutiondiscrete cosine transformresolution enhancementRMSEPSNRconvolutional neural networkReferencesChen Q., Huang J., Feris R., Brown L.M., Dong J., Yan S. Deep domain adaptation for describing people based on fne-grained clothing attributes. Proc. of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 5315–5324. https://doi.org/10.1109/cvpr.2015.7299169Chen Q., Huang J., Feris R., Brown L.M., Dong J., Yan S. Deep domain adaptation for describing people based on fne-grained clothing attributes. Proc. of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 5315–5324. https://doi.org/10.1109/cvpr.2015.7299169Denton E.L., Chintala S., Fergus R. Deep generative image models using a Laplacian pyramid of adversarial networks. Advances in Neural Information Processing Systems, 2015, vol. 28.Denton E.L., Chintala S., Fergus R. Deep generative image models using a Laplacian pyramid of adversarial networks. Advances in Neural Information Processing Systems, 2015, vol. 28.Cui Z., Chang H., Shan S., Zhong B., Chen X. Deep network cascade for image super-resolution. Lecture Notes in Computer Science, 2014, vol. 8693, pp. 49–64. https://doi.org/10.1007/978-3-319-10602-1_4Cui Z., Chang H., Shan S., Zhong B., Chen X. Deep network cascade for image super-resolution. Lecture Notes in Computer Science, 2014, vol. 8693, pp. 49–64. https://doi.org/10.1007/978-3-319-10602-1_4Farhadifard F., Abar E., Nazzal M., Ozkaramanh H. Single image super resolution based on sparse representation via directionally structured dictionaries. Proc. of the 22nd Signal Processing and Communications Applications Conference (SIU), IEEE, 2014, pp. 1718–1721. https://doi.org/10.1109/siu.2014.6830580Farhadifard F., Abar E., Nazzal M., Ozkaramanh H. Single image super resolution based on sparse representation via directionally structured dictionaries. Proc. of the 22nd Signal Processing and Communications Applications Conference (SIU), IEEE, 2014, pp. 1718–1721. https://doi.org/10.1109/siu.2014.6830580Ahmed J., Memon R.A., Waqas M., Mangrio M.I., Ali S. Selective sparse coding based coupled dictionary learning algorithm for single image super-resolution. Proc. of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), 2018, pp. 1–5. https://doi.org/10.1109/icomet.2018.8346357Ahmed J., Memon R.A., Waqas M., Mangrio M.I., Ali S. Selective sparse coding based coupled dictionary learning algorithm for single image super-resolution. Proc. of the 2018 International Conference on Computing, Mathematics and Engineering Technologies (iCoMET), 2018, pp. 1–5. https://doi.org/10.1109/icomet.2018.8346357Choi J.H., Kim J.H., Cheon M., Lee J.S. Deep learning-based image super-resolution considering quantitative and perceptual quality. Neurocomputing, 2020, vol. 398, pp. 347–59. https://doi.org/10.1016/j.neucom.2019.06.103Choi J.H., Kim J.H., Cheon M., Lee J.S. Deep learning-based image super-resolution considering quantitative and perceptual quality. Neurocomputing, 2020, vol. 398, pp. 347–59. https://doi.org/10.1016/j.neucom.2019.06.103Dong C., Loy C.C., He K., Tang X. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, vol. 38, no. 2, pp. 295–307. https://doi.org/10.1109/TPAMI.2015.2439281Dong C., Loy C.C., He K., Tang X. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, vol. 38, no. 2, pp. 295–307. https://doi.org/10.1109/TPAMI.2015.2439281Dong C., Loy C.C., Tang X. Accelerating the super-resolution convolutional neural network. Lecture Notes in Computer Science, 2016, vol. 9906, pp. 391–407. https://doi.org/10.1007/978-3-319-46475-6_25Dong C., Loy C.C., Tang X. Accelerating the super-resolution convolutional neural network. Lecture Notes in Computer Science, 2016, vol. 9906, pp. 391–407. https://doi.org/10.1007/978-3-319-46475-6_25Ayas S., Ekinci M. Single image super resolution using dictionary learning and sparse coding with multi-scale and multi-directional Gabor feature representation. Information Sciences, 2020, vol. 512, pp. 1264–1278. https://doi.org/10.1016/j.ins.2019.10.040Ayas S., Ekinci M. Single image super resolution using dictionary learning and sparse coding with multi-scale and multi-directional Gabor feature representation. Information Sciences, 2020, vol. 512, pp. 1264–1278. https://doi.org/10.1016/j.ins.2019.10.040Gu S., Zuo W., Xie Q., Meng D., Feng X., Zhang L. Convolutional sparse coding for image super-resolution. Proc. of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1823–1831. https://doi.org/10.1109/iccv.2015.212Gu S., Zuo W., Xie Q., Meng D., Feng X., Zhang L. Convolutional sparse coding for image super-resolution. Proc. of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1823–1831. https://doi.org/10.1109/iccv.2015.212Dosovitskiy A., Springenberg J.T., Brox T. Learning to generate chairs with convolutional neural networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1538–1546. https://doi.org/10.1109/cvpr.2015.7298761Dosovitskiy A., Springenberg J.T., Brox T. Learning to generate chairs with convolutional neural networks. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015, pp. 1538–1546. https://doi.org/10.1109/cvpr.2015.7298761Mathieu M., Couprie C., LeCun Y. Deep multi-scale video prediction beyond mean square error. Proc. of the 4th International Conference on Learning Representations (ICLR), 2016.Mathieu M., Couprie C., LeCun Y. Deep multi-scale video prediction beyond mean square error. Proc. of the 4th International Conference on Learning Representations (ICLR), 2016.Alec R., Luke M., Soumith C. Unsupervised representation learning with deep convolutional generative adversarial networks. Proc. of the International Conference on Learning Representations (ICLR), 2015, pp. 1–16.Alec R., Luke M., Soumith C. Unsupervised representation learning with deep convolutional generative adversarial networks. Proc. of the International Conference on Learning Representations (ICLR), 2015, pp. 1–16.Dong C., Loy C.C., He K., Tang X. Learning a deep convolutional network for image super-resolution. Lecture Notes in Computer Science, 2014, vol. 8692, pp. 184–199. https://doi.org/10.1007/978-3-319-10593-2_13Dong C., Loy C.C., He K., Tang X. Learning a deep convolutional network for image super-resolution. Lecture Notes in Computer Science, 2014, vol. 8692, pp. 184–199. https://doi.org/10.1007/978-3-319-10593-2_13Aharon M., Elad M., Bruckstein A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, vol. 54, no. 11, pp. 4311– 4322. https://doi.org/10.1109/tsp.2006.881199Aharon M., Elad M., Bruckstein A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, vol. 54, no. 11, pp. 4311– 4322. https://doi.org/10.1109/tsp.2006.881199Rueda A., Malpica N., Romero E. Single-image super-resolution of brain MR images using overcomplete dictionaries. Medical Image Analysis, 2013, vol. 17, no. 1, pp. 113–132. https://doi.org/10.1016/j.media.2012.09.003Rueda A., Malpica N., Romero E. Single-image super-resolution of brain MR images using overcomplete dictionaries. Medical Image Analysis, 2013, vol. 17, no. 1, pp. 113–132. https://doi.org/10.1016/j.media.2012.09.003Wang H., Jiang K. Research on image super-resolution reconstruction based on transformer. Proc. of the 2021 IEEE International Conference on Artifcial Intelligence and Industrial Design (AIID), 2021, pp. 226–230. https://doi.org/10.1109/aiid51893.2021.9456580Wang H., Jiang K. Research on image super-resolution reconstruction based on transformer. Proc. of the 2021 IEEE International Conference on Artifcial Intelligence and Industrial Design (AIID), 2021, pp. 226–230. https://doi.org/10.1109/aiid51893.2021.9456580Liu H., Guo Q., Wang G., Gupta B.B., Zhang C. Medical image resolution enhancement for healthcare using nonlocal self-similarity and low-rank prior. Multimedia Tools and Applications, 2019, vol. 78, no. 7, pp. 9033–9050. https://doi.org/10.1007/s11042-017-5277-6Liu H., Guo Q., Wang G., Gupta B.B., Zhang C. Medical image resolution enhancement for healthcare using nonlocal self-similarity and low-rank prior. Multimedia Tools and Applications, 2019, vol. 78, no. 7, pp. 9033–9050. https://doi.org/10.1007/s11042-017-5277-6Liu J., Malekzadeh M., Mirian N., Song T.A., Liu C., Dutta J. Artifcial intelligence-based image enhancement in PET imaging: Noise reduction and resolution enhancement. PET Clinics, 2021, vol. 16, no. 4, pp. 553–576. https://doi.org/10.1016/j.cpet.2021.06.005Liu J., Malekzadeh M., Mirian N., Song T.A., Liu C., Dutta J. Artifcial intelligence-based image enhancement in PET imaging: Noise reduction and resolution enhancement. PET Clinics, 2021, vol. 16, no. 4, pp. 553–576. https://doi.org/10.1016/j.cpet.2021.06.005Dabbaghchian S., Ghaemmaghami M.P., Aghagolzadeh A. Feature extraction using discrete cosine transform and discrimination power analysis with a face recognition technology. Pattern Recognition, 2010, vol. 43, no. 4, pp. 1431–1440. https://doi.org/10.1016/j.patcog.2009.11.001Dabbaghchian S., Ghaemmaghami M.P., Aghagolzadeh A. Feature extraction using discrete cosine transform and discrimination power analysis with a face recognition technology. Pattern Recognition, 2010, vol. 43, no. 4, pp. 1431–1440. https://doi.org/10.1016/j.patcog.2009.11.001Liew W.S., Tang T.B., Lin C.H., Lu C.K. Automatic colonic polyp detection using integration of modifed deep residual convolutional neural network and ensemble learning approaches. Computer Methods and Programs in Biomedicine, 2021, vol. 206, pp. 106114. https://doi.org/10.1016/j.cmpb.2021.106114Liew W.S., Tang T.B., Lin C.H., Lu C.K. Automatic colonic polyp detection using integration of modifed deep residual convolutional neural network and ensemble learning approaches. Computer Methods and Programs in Biomedicine, 2021, vol. 206, pp. 106114. https://doi.org/10.1016/j.cmpb.2021.106114Timofte R., De V., Van Gool L. Anchored neighborhood regression for fast example-based super-resolution. Proc. of the IEEE International Conference on Computer Vision, 2013, pp. 1920–1927. https://doi.org/10.1109/iccv.2013.241Timofte R., De V., Van Gool L. Anchored neighborhood regression for fast example-based super-resolution. Proc. of the IEEE International Conference on Computer Vision, 2013, pp. 1920–1927. https://doi.org/10.1109/iccv.2013.241Haris M., Shakhnarovich G., Ukita N. Deep back-projectinetworks for single image super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, vol. 43, no. 12, pp. 4323– 4337. https://doi.org/10.1109/tpami.2020.3002836Haris M., Shakhnarovich G., Ukita N. Deep back-projectinetworks for single image super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, vol. 43, no. 12, pp. 4323– 4337. https://doi.org/10.1109/tpami.2020.3002836Niu B., Wen W., Ren W., Zhang X., Yang L., Wang S., Zhang K., Cao X., Shen H. Single image super-resolution via a holistic attention network. Lecture Notes in Computer Science, 2020, vol. 12357, pp. 191–207. https://doi.org/10.1007/978-3-030-58610-2_12Niu B., Wen W., Ren W., Zhang X., Yang L., Wang S., Zhang K., Cao X., Shen H. Single image super-resolution via a holistic attention network. Lecture Notes in Computer Science, 2020, vol. 12357, pp. 191–207. https://doi.org/10.1007/978-3-030-58610-2_12Lan R., Sun L., Liu Z., Lu H., Pang C., Luo X. MADNet: a fast and lightweight network for single-image super resolution. IEEE Transactions on Cybernetics, 2021, vol. 51, no. 3, pp. 1443–1453. https://doi.org/10.1109/tcyb.2020.2970104Lan R., Sun L., Liu Z., Lu H., Pang C., Luo X. MADNet: a fast and lightweight network for single-image super resolution. IEEE Transactions on Cybernetics, 2021, vol. 51, no. 3, pp. 1443–1453. https://doi.org/10.1109/tcyb.2020.2970104

The authors declare that there are no conflicts of interest present.