International Journal of Engineering Research in Computer Science and Engineering (IJERCSE)

Delineation of Agents for Games using Deep Reinforcement Learning

Author : Pranav Batra ¹ Sambhav Jain ² Asst. Prof. Yamini Ratawal ³

Date of Publication :10th September 2020

Abstract: Artificial Intelligence (AI) agents should have a precise representation of their surroundings and should be able to retain information based on their past experiences and from sensory inputs. It is done by developing smart self-learning agents using various Deep Reinforcement Learning algorithms such as Q-learning (DQN) and SARSA learning (DSN). These algorithms have helped game developers in removing the monotonous nature of games helping in the augmentation of the gaming industry. Intelligent agents can also be used in game development as these agents inhibit human-like behaviour which can help with the game balancing and assessment, almost identical to the performance of game developers. First Person Shooter (FPS) games have seen a significant gain in popularity due to increased difficulty and lack of predictability. This paper is an attempt in doing the same by reviewing various existing works in this field and some of the popular techniques of developing intelligent Agents.

Reference :

1. L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” Journal of artificial intelligence research 4, pp.237-285, May 1996.
2. V. Zambaldi, D. Raposo, A. Santoro, V. Bapst, Y. Li, I. Babuschkin, K. Tuyls et al. "Deep
3. reinforcement learning with relational inductive biases," International Conference on Learning Representations, 2018.
4. K. Clary, E. Tosch, J. Foley, and D. Jensen. "Let's Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments," arXiv preprint arXiv:1904.06312, 2019.
5. D. Silva, F. Leno, P. Hernandez-Leal, B. Kartal, and M. E. Taylor, "Uncertainty-Aware Action Advising for Deep Reinforcement Learning Agents," AAAI, pp. 5792-5799, 2020.
6. L. Gruenwoldt, S. Danton, and M. Katchabaw, "Creating reactive non player character artificial intelligence in modern video games," Proceedings of the 2005 GameOn North America Conference, pp. 10-17, 2005
7. S. S. Esfahlani, J. Butt, and H. Shirvani, "Fusion of artificial intelligence in neuro-rehabilitation video games," IEEE Access 7, pp. 102617-102627, 2019.
8. W. Westera, R. Prada, S. Mascarenhas, P. A. Santos, J. Dias, M. Guimarães, K. Georgiadis et al., "Artificial intelligence moving serious gaming: Presenting reusable game AI components," Education and Information Technologies 25, no. 1, pp. 351-380, 2020.
9. R. Thawonmas, J. Togelius, and G. N. Yannakakis, "Artificial general intelligence in games: Where play meets design and user experience," 2019.
10. B. J. Pohl, A. Harris, M. Balog, M. Clausen, G. Moran, and R. Brucks, "Fortnite: supercharging CG animation pipelines with game engine technology," Proceedings of the ACM SIGGRAPH Digital Production Symposium, pp. 1-4, 2017.
11. Entertainment, Sony Interactive. "Playstation vr," Website, (2017), Retrieved April 10 2020.
12. A. Dobrovsky, U. M. Borghoff, and M. Hofmann, "Improving adaptive gameplay in serious games through interactive deep reinforcement learning," Cognitive infocommunications, theory and applications, Springer, Cham, pp. 411-432, 2019.
13. Y. Zhao, I. Borovikov, F. D. M. Silva, A. Beirami, J. Rupert, C. Somers, J. Harder et al., "Winning Isn't Everything: Enhancing Game Development with Intelligent Agents," IEEE Transactions on Games, 2020.
14. J. Suarez, Y. Du, P. Isola, and I. Mordatch, "Neural MMO: A massively multiagent game environment for training and evaluating intelligent agents," arXiv preprint arXiv:1903.00784, 2019.
15. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing atari with deep reinforcement learning," arXiv preprint arXiv:1312.5602, 2013.
16. A. Taïga, W. Fedus, M. C. Machado, A. Courville, and M. G. Bellemare, "Benchmarking bonus-based exploration methods on the arcade learning environment," arXiv preprint arXiv:1908.02388, 2019.
17. G. Lample, and D. S. Chaplot, "Playing FPS games with deep reinforcement learning," Thirty-First AAAI Conference on Artificial Intelligence, February 2017.
18. A. Khan, N. Naeem, Z. A. M. Z. Asghar, A. U. Din, and Atif Khan, "Playing first-person shooter games with machine learning techniques and methods using the VizDoom Game-AI research platform." Entertainment Computing 34, 100357, 2020.
19. M. McPartland, and M. Gallagher, "Reinforcement learning in first person shooter games," IEEE Transactions on Computational Intelligence and AI in Games, vol. 3(1), pp. 43-56, 2010.
20. B. B. Vivien, "Features and Performance of Sarsa Reinforcement Learning Algorithm with Eligibility Traces and Local Environment Analysis for Bots in First Person Shooter Games." PhD diss., Waseda University, 2020.
21. M. H. Olyaei, H. Jalali, A. Olyaei, and A. Noori, "Implement Deep SARSA in Grid World with Changing Obstacles and Testing Against New Environment," Fundamental Research in Electrical Engineering, Springer, pp. 267-279, Singapore, 2019.
22. K. Shao, Z. Tang, Y. Zhu, N. Li, and D. Zhao, "A survey of deep reinforcement learning in video games," arXiv preprint arXiv:1912.10944, 2019.
23. D. Choi, T. Konik, N. Nejati, C. Park, and P. Langley, "A believable agent for first-person perspective games." 3rd Artificial Intelligence and Interactive Digital Entertainment International Conference, 2007.
24. Y. Tian, Q. Gong, W. Shang, Y. Wu, and C. L. Zitnick, "Elf: An extensive, lightweight and flexible research platform for real-time strategy games," Advances in Neural Information Processing Systems, pp. 2659-2669, 2017.
25. O. Vinyals, T. Ewalds, S. Bartunov, P. Georgiev, A. S. Vezhnevets, M. Yeo, A. Makhzani et al, "Starcraft ii: A new challenge for reinforcement learning," arXiv preprint arXiv:1708.04782, 2017.
26. M. Mora-Cantallops, and MÁ. Sicilia, "MOBA games: A literature review," Entertainment computing, vol. 26, pp. 128-138, 2018.
27. H. V. Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double q-learning," Thirtieth AAAI conference on artificial intelligence, March, 2016.
28. F. G. Glavin, & M. G. Madden, "DRE-Bot: A hierarchical First-Person Shooter bot using multiple Sarsa (λ) reinforcement learners," 2012 17th International Conference on Computer Games (CGAMES), pp. 148-152, IEEE, July 2012
29. Games, Epic. Unreal tournament 2004. Atari, 2004.
30. S. Arzt, G. Mitterlechner, M. Tatzgern, and T. Stütz, "Deep Reinforcement Learning of an Agent in a Modern 3D Video Game."
31. W. Konen, "Reinforcement learning for board games: The temporal difference algorithm," Research Center CIOP (Computational Intelligence, Optimization and Data Mining), TH Köln–Cologne University of Applied Sciences, Tech. Rep, 2015.