ntvНаучно-технический вестник информационных технологий, механики и оптикиScientific and Technical Journal of Information Technologies, Mechanics and Optics2226-14942500-0373Университет ИТМО10.17586/2226-1494-2022-22-3-594-599ntv-277Research ArticleКОМПЬЮТЕРНЫЕ СИСТЕМЫ И ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИCOMPUTER SCIENCEВеб-приложение для быстрой оценки субъективных ответов с использованием обработки естественного языкаWeb app for quick evaluation of subjective answers using natural language processinghttps://orcid.org/0000-0001-7048-7289Минакши АнурагТ.Meenakshi AnuragTh.

Минакши Анураг Талор — PhD, доцент

Пуна, 411001

Thalor Meenakshi Anurag — PhD, Associate Professor

Pune, 411001

meenakshi.thalor@aissmsioit.orghttps://orcid.org/0000-0002-3252-9983Прадип Б.М.Pradeep B.M.

Прадип Б. Мане — PhD, директор

Пуна, 411001

Mane Pradeep B. — PhD, Principal

Pune, 411001

pbmane6829@gmail.comhttps://orcid.org/0000-0002-1667-4814ВишакаМ.VishakaM.

Вишака Мандж — бакалавр, студент

Пуна, 411001

Mandge Vishaka — B.E., Student

Pune, 411001

mandge1999@gmail.comAISSMS Институт информационных технологийИндияAISSMS Institute of Information TechnologyIndia202217122024223594599Copyright © Минакши Анураг Т., Прадип Б. М., Вишака М., 20242024Минакши Анураг Т., Прадип Б. М., Вишака М.Meenakshi Anurag T., Pradeep B. M., Vishaka M.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ntv.elpub.ru/jour/article/view/277

В современном цифровом пространстве развитие образования связано с продвижением компьютерных технологий. Происходит цифровизация образования: проводятся онлайн-классы, онлайн-экзамены и т. д. В период проведения экзамена учащиеся оцениваются педагогом по ответам на заданные вопросы. На настоящий момент доступно множество инструментов оценки успеваемости учащегося с использованием вопросов с несколькими вариантами ответов. Такой подход обеспечивает мгновенную оценку, однако инструментов, дающих возможность оценки субъективных ответов учащихся еще недостаточно. В работе представлено веб-приложение для решения этой проблемы. Приложение позволяет автоматизировать процесс проверки субъективных ответов и генерировать результаты с применением методов обработки естественного языка, таких как семантическое сопоставление ключевых слов, лексический анализ и косинусное сходство. Экспериментальные результаты показали, что ответы, оцененные педагогами, и предложенная система дают близкие результаты. Полученная точность ответов составила 97 %. Представленная система не только сокращает рабочее время, но и позволяет устранить традиционный метод проведения исключительно субъективных экзаменов с использованием бумажных документов. В результате сокращается время получения результатов и время проверки. Уменьшается утечка информации и повышается объективность оценки. 

In current digital climate, education sector is evolving as the computer technology advances. Education is being digitized: online classes, online examination methods are conducted, etc. During examination, students are assessed by their answers having given for the question set by a teacher. Today many tools are available to assess the performance of a student using multi choice questions tools which provide instant evaluation, but there are available very limited and operational tools where subjective type answer of students are evaluated. This paper presents a web-based application to address this challenge. It automates the process of subjective answers checking and generates results through using natural language processing methods, like keyword matching semantic, lexical analysis and cosine similarity. Experiments show that appreciated by the teacher result and the system estimation does not have much difference which signifies that the system evaluates answers with a 97 % accuracy. The presented system not only reduces manpower but also eliminates the traditional method of conducting exclusively subjective exams using paper documents. It also eliminates the delays in the paper checking, result generation process. The cases of information leak are being reduced and the objectivity of the assessment is being increased.

косинусное сходствоизвлечение информациисопоставление ключевых словобработка естественного языкаcosine similarityinformation extractionkeyword matchingnatural language processingReferencesLakshmi V., Ramesh V. Evaluating students’ descriptive answers using natural language processing and artificial neural networks. International Journal of Creative Research Thoughts — IJCRT, 2017, vol. 5, no. 4, pp. 3168–3173.Lakshmi V., Ramesh V. Evaluating students’ descriptive answers using natural language processing and artificial neural networks. International Journal of Creative Research Thoughts — IJCRT, 2017, vol. 5, no. 4, pp. 3168–3173.Patil S.M., Patil S. Evaluating student descriptive answers using natural language processing. International Journal of Engineering Research & Technology (IJERT), 2014, vol. 3, no. 3, pp. 1716–1718.Patil S.M., Patil S. Evaluating student descriptive answers using natural language processing. International Journal of Engineering Research & Technology (IJERT), 2014, vol. 3, no. 3, pp. 1716–1718.Yin W., Kann K., Yu M., Schütze H. Comparative study of CNN and RNN for natural language processing. ArXiv, 2017, arXiv:1702.01923. https://doi.org/10.48550/arXiv.1702.01923Yin W., Kann K., Yu M., Schütze H. Comparative study of CNN and RNN for natural language processing. ArXiv, 2017, arXiv:1702.01923. https://doi.org/10.48550/arXiv.1702.01923Kate A., Kamble S., Bodkhe A., Joshi M. Conversion of Natural Language Query to SQL Query. Proc. of the 2nd International conference on Electronics, Communication and Aerospace Technology (ICECA), 2018, pp. 488–491. https://doi.org/10.1109/ICECA.2018.8474639Kate A., Kamble S., Bodkhe A., Joshi M. Conversion of Natural Language Query to SQL Query. Proc. of the 2nd International conference on Electronics, Communication and Aerospace Technology (ICECA), 2018, pp. 488–491. https://doi.org/10.1109/ICECA.2018.8474639Thangarasu S., Sasikala D. Extracting knowledge from XML document using tree-based association rules. Proc. of the International Conference on Intelligent Computing Applications (ICICA), 2014, pp. 134–137. https://doi.org/10.1109/ICICA.2014.37Thangarasu S., Sasikala D. Extracting knowledge from XML document using tree-based association rules. Proc. of the International Conference on Intelligent Computing Applications (ICICA), 2014, pp. 134–137. https://doi.org/10.1109/ICICA.2014.37Mati D.N., Ajdari J., Raufi B., Hamiti M., Selimi B. A Systematic mapping study of language features identification from large text collection. Proc. of the 8th Mediterranean Conference on Embedded Computing (MECO), 2019, pp. 8760042. https://doi.org/10.1109/MECO.2019.8760042Mati D.N., Ajdari J., Raufi B., Hamiti M., Selimi B. A Systematic mapping study of language features identification from large text collection. Proc. of the 8th Mediterranean Conference on Embedded Computing (MECO), 2019, pp. 8760042. https://doi.org/10.1109/MECO.2019.8760042Cai J., Li J., Li W., Wang J. Deeplearning model used in text classification. Proc of the 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 2018, pp. 123–126. https://doi.org/10.1109/ICCWAMTIP.2018.8632592Cai J., Li J., Li W., Wang J. Deeplearning model used in text classification. Proc of the 15th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 2018, pp. 123–126. https://doi.org/10.1109/ICCWAMTIP.2018.8632592Tan M., Santos C., Xiang B., Zhou B. Improved representation learning for question answer matching. Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), 2016, pp. 464–473. https://doi.org/10.18653/v1/P16-1044Tan M., Santos C., Xiang B., Zhou B. Improved representation learning for question answer matching. Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), 2016, pp. 464–473. https://doi.org/10.18653/v1/P16-1044Lan Y., Wang S., Jiang J. Knowledge base question answering with a matching-aggregation model and question-specific contextual relations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, vol. 27, no. 10, pp. 1629–1638. https://doi.org/10.1109/TASLP.2019.2926125Lan Y., Wang S., Jiang J. Knowledge base question answering with a matching-aggregation model and question-specific contextual relations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, vol. 27, no. 10, pp. 1629–1638. https://doi.org/10.1109/TASLP.2019.2926125Grappy A., Grau B., Falco M-H., Ligozat A-L., Robba I., Vilnat A. Selecting answers to questions from Web documents by a robust validation process. Proc. of the IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, 2011, pp. 55–62. https://doi.org/10.1109/WI-IAT.2011.210Grappy A., Grau B., Falco M-H., Ligozat A-L., Robba I., Vilnat A. Selecting answers to questions from Web documents by a robust validation process. Proc. of the IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, 2011, pp. 55–62. https://doi.org/10.1109/WI-IAT.2011.210Lan Y., Wang S., Jiang J. Knowledge base question answering with a matching-aggregation model and question specific contextual relations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, vol. 27, no. 10, pp. 1629–1638. https://doi.org/10.1109/TASLP.2019.2926125Lan Y., Wang S., Jiang J. Knowledge base question answering with a matching-aggregation model and question specific contextual relations. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, vol. 27, no. 10, pp. 1629–1638. https://doi.org/10.1109/TASLP.2019.2926125Zhang W., Ming Z., Zhang Y., Liu T., Chua T. Capturing the semantics of key phrases using multiple languages for question retrieval. IEEE Transactions on Knowledge and Data Engineering, 2016, vol. 28, no. 4, pp. 888–900. https://doi.org/10.1109/TKDE.2015.2502944Zhang W., Ming Z., Zhang Y., Liu T., Chua T. Capturing the semantics of key phrases using multiple languages for question retrieval. IEEE Transactions on Knowledge and Data Engineering, 2016, vol. 28, no. 4, pp. 888–900. https://doi.org/10.1109/TKDE.2015.2502944Pakray P., Pal S., Bandyopadhyay S., Gelbukh A. Automatic answer validation system on English language. Proc. of the 3rd International Conference on Advanced Computer Theory and Engineering, 2010, pp. V6329–V6333. https://doi.org/10.1109/ICACTE.2010.5579166Pakray P., Pal S., Bandyopadhyay S., Gelbukh A. Automatic answer validation system on English language. Proc. of the 3rd International Conference on Advanced Computer Theory and Engineering, 2010, pp. V6329–V6333. https://doi.org/10.1109/ICACTE.2010.5579166Thalor M.A. A descriptive answer evaluation system using cosine similarity technique. Proc. of the 4th International Conference on Communication information and Computing Technology (ICCICT), 2021. https://doi.org/10.1109/ICCICT50803.2021.9510170Thalor M.A. A descriptive answer evaluation system using cosine similarity technique. Proc. of the 4th International Conference on Communication information and Computing Technology (ICCICT), 2021. https://doi.org/10.1109/ICCICT50803.2021.9510170Mandge V.A., Thalor M.A. Revolutionize cosine answer matching technique for question answering system. Proc. of the International Conference on Emerging Smart Computing and Informatics (ESCI), 2021, pp. 335–339. https://doi.org/10.1109/ESCI50559.2021.9396864Mandge V.A., Thalor M.A. Revolutionize cosine answer matching technique for question answering system. Proc. of the International Conference on Emerging Smart Computing and Informatics (ESCI), 2021, pp. 335–339. https://doi.org/10.1109/ESCI50559.2021.9396864

The authors declare that there are no conflicts of interest present.