Knowledge-based and data-driven behavioral modeling techniques in engagement simulation

Zhi Zhu, Tao Wang, Hessam Sarjoughian, Weiping Wang, Yuehua Zhao

Research output: Contribution to journalArticlepeer-review


As knowledge and data increase in scale and complexity, it is more difficult to apply these two key assets to achieve optimal effectiveness in engagement simulation. The aim of this study was to investigate the techniques of knowledge and data integration with respect to the development of smart agents to predict accurate behaviors in tactical engagements. To reduce the complexity of combat behavior representation, with respect to the functions, we represented subject matter expert operational knowledge by proposing multiple levels of cascaded hierarchical structure, namely, the function decision tree, to increase the readability and maintainability of the behavioral model. For decision points in a behavioral model, smart agents can be trained based on data samples collected from rounds of constructive simulations which provide validated physical models and tactical principles. As a proof of concept, we constructed a simulation testbed of multi-warhead ballistic missile penetration, which generated 129,600 constructive simulations over a total of 84 h. Thereafter, we selected 5817 data samples (i.e. ~4.5% of the simulations) using an operational metric of total rewards exceeding 100. The data samples are used to train an artificial neural network and then this network is used to develop a deep reinforcement learning agent. The results revealed that the training process iterated nearly 17,000 epochs until the policy loss decreased to an acceptable low value. The smart agent increased the ratio of ballistic missile target hits by 18.96%, a significant increase when compared with the traditional rule-based behavioral model.

Original languageEnglish (US)
Pages (from-to)1069-1089
Number of pages21
Issue number10
StatePublished - Oct 2023


  • Agent-based systems
  • artificial intelligence
  • deep reinforcement learning
  • defense systems

ASJC Scopus subject areas

  • Software
  • Modeling and Simulation
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Knowledge-based and data-driven behavioral modeling techniques in engagement simulation'. Together they form a unique fingerprint.

Cite this