Decision making in simplified land combat models - on design and implementation of software modules playing the games of Operation Lucid and Operation Opaque

Abstract

We present the work done on the stochastic games Operation Lucid and Operation Opaque during FFI project 722, “Synthetic decision making”. These games, designed as simplified land combat simulation models, are defined and some of their properties described. We give a theoretical and practical treatment of the problem of evaluating performance in these games, including mathematically sound performance measures and a successful method for reducing the effect of stochastic noise in the games. The core of the report consists of a general design based on constraint programming for software agents playing the games of Operation, and two applications of this design, using neural nets and fuzzy logic, respectively. The agent design presented is successful in combining the best points of a brute-force and a more human-like approach to game playing, and makes it possible for software agents to play well in spite of the very high complexity of the games. The applications demonstrate the practical utility of this design. Special issues pertaining to the information imperfection of Operation Opaque are also addressed. Some main conclusions of the work are: 1) Our agent design is useful for applying and combining artificial intelligence techniques. 2) Reinforcement learning algorithms are suitable for learning in this noisy domain, while direct gradient-based parameter optimisation is not. 3) Representation of domain knowledge can significantly improve performance.