• Applying Markov decision processes to 2D real time games

      Hartley, Thomas; Mehdi, Qasim; Gough, Norman (University of Wolverhampton, School of Computing and Information Technology, 2004)
      This paper presents the outcomes of a research project into the field of artificial intelligence (AI) and computer game AI. The project considered the problem of applying AI techniques to computer games. Current commercial computer games tend to use complex scripts to control AI opponents. This can result in poor and predictable gameplay. The use of academic AI techniques is a possible solution to overcome these shortcomings. This paper describes the process of applying Markov decision processes (MDPs) using the value iteration algorithm to a 2D real time computer game. We also introduce a new stopping criterion for value iteration, which has been designed for use in computer games and we discuss results from experiments conducted on the MDPs AI engine. This paper also outlines conclusions about how successful MDPs are in relation to a real computer game AI engine and how useful they might be to computer games developers.
    • Using value iteration to solve sequential decision problems in games

      Hartley, Thomas; Mehdi, Qasim; Gough, Norman (University of Wolverhampton, School of Computing and Information Technology, 2004)
      Solving sequential decision problems in computer games, such as non-player character (NPC) navigation, can be quite a complex task. Current games tend to rely on scripts and finite state machines (FSM) to control AI opponents. These approaches however have shortcomings; as a result academic AI techniques may be a more desirable solution to solve these types of problems. This paper describes the process of applying the value iteration algorithm to an AI engine, which can be applied to a computer game. We also introduce a new stopping criterion called game value iteration, which has been designed for use in 2D real time computer games and we discuss results from experiments conducted on the AI engine. We also outline our conclusions which state that the value iteration and the newly introduced game value iteration algorithms can be successfully applied to intelligent NPC behaviour in computer games; however there are certain problems, such as execution speed, which need to be addressed when dealing with real time games.