ntvНаучно-технический вестник информационных технологий, механики и оптикиScientific and Technical Journal of Information Technologies, Mechanics and Optics2226-14942500-0373Университет ИТМО10.17586/2226-1494-2024-24-6-1044-1048ntv-411Research ArticleКОМПЬЮТЕРНЫЕ СИСТЕМЫ И ИНФОРМАЦИОННЫЕ ТЕХНОЛОГИИCOMPUTER SCIENCEОсобенности организации игрового взаимодействия асимметричных агентов с использованием графовых нейронных сетейAspects of organizing game interactions among asymmetric agents using graph neural networkshttps://orcid.org/0000-0002-2938-0575ИсаковА. О.IsakovA. O.

Исаков Артём Олегович - аспирант, 

Санкт-Петербург, 197101

Artem O. Isakov - PhD Student,

Saint Petersburg, 197101

aoisakov@itmo.ruhttps://orcid.org/0000-0002-2855-9207ПерегородиевД. Е.PeregorodievD. E.

Перегородиев Данил Евгеньевич - студент,

Санкт-Петербург, 197101

Danil E. Peregorodiev - Student,

Saint Petersburg, 197101

deperegorodiev@itmo.ruhttps://orcid.org/0000-0003-1886-2867ТомиловИ. В.TomilovI. V.

Томилов Иван Вячеславович - студент,

Санкт-Петербург, 197101

Ivan V. Tomilov - Student,

Saint Petersburg, 197101

ivan-tomilov3@yandex.ruhttps://orcid.org/0000-0002-1361-6037ГусароваН. Ф.GusarovaN. F.

Гусарова Наталия Федоровна - кандидат технических наук, старший научный сотрудник, доцент,

Санкт-Петербург, 197101

Natalia F. Gusarova - PhD, Senior Researcher, Associate Professor,

Saint Petersburg, 197101

natfed@list.ruhttps://orcid.org/0000-0001-7417-6947ГолубевА. А.GolubevA. A.

Голубев Александр Андреевич - аспирант,

Санкт-Петербург, 197101

Alexander A. Golubev - PhD Student,

Saint Petersburg, 197101

9459539@gmail.comУниверситет ИТМОРоссияITMO UniversityRussian Federation20242912202424610441048Copyright © Исаков А.О., Перегородиев Д.Е., Томилов И.В., Гусарова Н.Ф., Голубев А.А., 20242024Исаков А.О., Перегородиев Д.Е., Томилов И.В., Гусарова Н.Ф., Голубев А.А.Isakov A.O., Peregorodiev D.E., Tomilov I.V., Gusarova N.F., Golubev A.A.Данная работа распространяется под лицензией Creative Commons Attribution 4.0.This work is licensed under a Creative Commons Attribution 4.0 License.https://ntv.elpub.ru/jour/article/view/411Введение

Введение. Рассмотрена структура представления графа межагентных связей для повышения эффективности взаимодействия агентов в кооперативных состязательных играх с использованием графовых нейронных сетей.

Метод

Метод. Выполнена сравнительная оценка метрик и матриц смежности для графов связей, задаваемых с применением геометрической и семантической метрик близости.

Основные результаты

Основные результаты. Показано, что семантическая близость более эффективна при построении графа межагентных связей, а применение орграфов обеспечивает гибкое управление информационными потоками.

Обсуждение

Обсуждение. Предложенные закономерности важно учитывать при организации многоагентного обучения с подкреплением в широком диапазоне областей применения.

The article considers the structures of representation of the graph of inter-agent connections for increasing the efficiency of agent interaction in cooperative competitive games using graph neural networks. A comparative assessment of metrics and adjacency matrices for graphs of connections defined using geometric and semantic metrics of proximity is performed. It is shown that semantic proximity is more effective in constructing a graph of inter-agent connections, and the use of oriented graphs ensures flexible management of information flows. The proposed patterns are important to consider when organizing multi-agent reinforcement learning in a wide range of application areas.

теория графовграфовые нейронные сетиобучение с подкреплениеммногоагентные системыкооперативно-состязательное поведениеgraph theorygraph neural networksreinforcement learningmulti-agent systemscooperative-competitive behaviorРабота выполнена при поддержке Министерства науки и высшего образования Российской Федерации, госзадание № 2019-1339.The Ministry of Science and Higher Education of the Russian Federation: State Assignment No. 2019-1339.ReferencesYang S. Hierarchical graph multi-agent reinforcement learning for traffic signal control // Information Sciences. 2023. V. 634. P. 55–72. https://doi.org/10.1016/j.ins.2023.03.087Yang S. Hierarchical graph multi-agent reinforcement learning for traffic signal control. Information Sciences, 2023, vol. 634, pp. 55–72. https://doi.org/10.1016/j.ins.2023.03.087Veličković P. Everything is connected: Graph neural networks // Current Opinion in Structural Biology. 2023. V. 79. P. 102538. https://doi.org/10.1016/j.sbi.2023.102538Veličković P. Everything is connected: Graph neural networks. Current Opinion in Structural Biology, 2023, vol. 79, pp. 102538. https://doi.org/10.1016/j.sbi.2023.102538Khemani B., Patil S., Kotecha K., Tanwar S. A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions // Journal of Big Data. 2024. V. 11. N 1. P. 18. https://doi.org/10.1186/s40537-023-00876-4Khemani B., Patil S., Kotecha K., Tanwar S. A review of graph neural networks: concepts, architectures, techniques, challenges, datasets, applications, and future directions. Journal of Big Data, 2024, vol. 11, no. 1, pp. 18. https://doi.org/10.1186/s40537-023-00876-4Nie M., Chen D., Wang D. Reinforcement learning on graphs: A survey // IEEE Transactions on Emerging Topics in Computational Intelligence. 2023. V. 7. N 4. P. 1065–1082. https://doi.org/10.1109/tetci.2022.3222545Nie M., Chen D., Wang D. Reinforcement learning on graphs: A survey. IEEE Transactions on Emerging Topics in Computational Intelligence, 2023, vol. 7, no. 4, pp. 1065–1082. https://doi.org/10.1109/tetci.2022.3222545Zhou J., Cui G., Hu S., Zhang Z., Yang C., Liu Z., Wang L., Li C., Sun M. Graph neural networks: A review of methods and applications // AI Open. 2020. V. 1. P. 57–81. https://doi.org/10.1016/j.aiopen.2021.01.001Zhou J., Cui G., Hu S., Zhang Z., Yang C., Liu Z., Wang L., Li C., Sun M. Graph neural networks: A review of methods and applications. AI Open, 2020, vol. 1, pp. 57–81. https://doi.org/10.1016/j.aiopen.2021.01.001Wu Z., Pan S., Chen F., Long G., Zhang C., Yu P.S. A comprehensive survey on graph neural networks // IEEE Transactions on Neural Networks and Learning Systems. 2020. V. 32. N 1. P. 4–24. https://doi.org/10.1109/tnnls.2020.2978386Wu Z., Pan S., Chen F., Long G., Zhang C., Yu P.S. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2020, vol. 32, no. 1, pp. 4–24. https://doi.org/10.1109/tnnls.2020.2978386Bhatti U.A., Tang H., Wu G., Marjan S., Hussain A. Deep learning with graph convolutional networks: An overview and latest applications in computational intelligence // International Journal of Intelligent Systems. 2023. V. 2023. P. 8342104. https://doi.org/10.1155/2023/8342104Bhatti U.A., Tang H., Wu G., Marjan S., Hussain A. Deep learning with graph convolutional networks: An overview and latest applications in computational intelligence. International Journal of Intelligent Systems, 2023, vol. 2023, pp. 8342104. https://doi.org/10.1155/2023/8342104Wu L., Cui P., Pei J., Zhao L., Guo X. Graph neural networks: foundation, frontiers and applications // Proc. of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2022. P. 4840–4841. https://doi.org/10.1145/3534678.3542609Wu L., Cui P., Pei J., Zhao L., Guo X. Graph neural networks: foundation, frontiers and applications. Proc. of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022, pp. 4840–4841. https://doi.org/10.1145/3534678.3542609Meng L., Shao Y., Yuan L., Lai L., Cheng P., Li X., Yu W., Zhang W., Lin X., Zhou J. A survey of distributed graph algorithms on massive graphs // ACM Computing Surveys. 2024. V. 57. N 2. P. 1–39. https://doi.org/10.1145/3694966Meng L., Shao Y., Yuan L., Lai L., Cheng P., Li X., Yu W., Zhang W., Lin X., Zhou J. A survey of distributed graph algorithms on massive graphs. ACM Computing Surveys, 2024, vol. 57, no. 2, pp. 1–39. https://doi.org/10.1145/3694966Vrahatis A.G., Lazaros K., Kotsiantis S. Graph Attention Networks: A Comprehensive Review of Methods and Applications // Future Internet. 2024. V. 16. N 9. P. 318. https://doi.org/10.3390/fi16090318Vrahatis A.G., Lazaros K., Kotsiantis S. Graph Attention Networks: A Comprehensive Review of Methods and Applications. Future Internet, 2024, vol. 16, no. 9, pp. 318. https://doi.org/10.3390/ fi16090318Munikoti S., Agarwal D., Das L., Halappanavar M., Natarajan B. Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications // IEEE Transactions on Neural Networks and Learning Systems. 2023. V. 35. N 11. P. 15051–15071. https://doi.org/10.1109/tnnls.2023.3283523Munikoti S., Agarwal D., Das L., Halappanavar M., Natarajan B. Challenges and opportunities in deep reinforcement learning with graph neural networks: A comprehensive review of algorithms and applications. IEEE Transactions on Neural Networks and Learning Systems, 2023, vol. 35, no. 11, pp. 15051–15071. https://doi.org/10.1109/tnnls.2023.3283523Verbavatz V., Barthelemy M. Betweenness centrality in dense spatial networks // Physical Review E. 2022. V. 105. N 5. P. 054303. https://doi.org/10.1103/physreve.105.054303Verbavatz V., Barthelemy M. Betweenness centrality in dense spatial networks. Physical Review E, 2022, vol. 105, no. 5, pp. 054303. https://doi.org/10.1103/physreve.105.054303Tsalouchidou I., Baeza-Yates R., Bonchi F., Liao K., Sellis T. Temporal betweenness centrality in dynamic graphs // International Journal of Data Science and Analytics. 2020. V. 9. N 3. P. 257–272. https://doi.org/10.1007/s41060-019-00189-xTsalouchidou I., Baeza-Yates R., Bonchi F., Liao K., Sellis T. Temporal betweenness centrality in dynamic graphs. International Journal of Data Science and Analytics, 2020, vol. 9, no. 3, pp. 257– 272. https://doi.org/10.1007/s41060-019-00189-xWills P., Meyer F.G. Metrics for graph comparison: a practitionerʼs guide // PLoS ONE. 2020. V. 15. N 2. P. e0228728. https://doi.org/10.1371/journal.pone.0228728Wills P., Meyer F.G. Metrics for graph comparison: a practitionerʼs guide. PLoS ONE, 2020, vol. 15, no. 2, pp. e0228728. https://doi.org/10.1371/journal.pone.0228728O’Bray L., Horn M., Rieck B., Borgwardt K. Evaluation metrics for graph generative models: Problems, pitfalls, and practical solutions // Proc. of the ICLR 2022 — 10th International Conference on Learning Representations. 2022.O’Bray L., Horn M., Rieck B., Borgwardt K. Evaluation metrics for graph generative models: Problems, pitfalls, and practical solutions. Proc. of the ICLR 2022 — 10th International Conference on Learning Representations, 2022.

The authors declare that there are no conflicts of interest present.