TY - JOUR
T1 - The automated model of comprehension version 4.0 – Validation studies and integration of ChatGPT
AU - Corlatescu, Dragos Georgian
AU - Watanabe, Micah
AU - Ruseti, Stefan
AU - Dascalu, Mihai
AU - McNamara, Danielle S.
N1 - Publisher Copyright: © 2024 Elsevier Ltd
PY - 2024/5
Y1 - 2024/5
N2 - Modeling reading comprehension processes is a critical task for Learning Analytics, as accurate models of the reading process can be used to match students to texts, identify appropriate interventions, and predict learning outcomes. This paper introduces an improved version of the Automated Model of Comprehension, namely version 4.0. AMoC has its roots in two theoretical models of the comprehension process (i.e., the Construction-Integration model and the Landscape model), and the new version leverages state-of-the-art Large Language models, more specifically ChatGPT, to have a better contextualization of the text and a simplified construction of the underlying graph model. Besides showcasing the usage of the model, the study introduces three in-depth psychological validations that argue for the model's adequacy in modeling reading comprehension. In these studies, we demonstrated that AMoC is in line with the theoretical background proposed by the Construction-Integration and Landscape models, and it is better at replicating results from previous human psychological experiments than its predecessor. Thus, AMoC v4.0 can be further used as an educational tool to, for example, help teachers design better learning materials personalized for student profiles. Additionally, we release the code from AMoC v4.0 as open source in a Google Collab Notebook and a GitHub repository.
AB - Modeling reading comprehension processes is a critical task for Learning Analytics, as accurate models of the reading process can be used to match students to texts, identify appropriate interventions, and predict learning outcomes. This paper introduces an improved version of the Automated Model of Comprehension, namely version 4.0. AMoC has its roots in two theoretical models of the comprehension process (i.e., the Construction-Integration model and the Landscape model), and the new version leverages state-of-the-art Large Language models, more specifically ChatGPT, to have a better contextualization of the text and a simplified construction of the underlying graph model. Besides showcasing the usage of the model, the study introduces three in-depth psychological validations that argue for the model's adequacy in modeling reading comprehension. In these studies, we demonstrated that AMoC is in line with the theoretical background proposed by the Construction-Integration and Landscape models, and it is better at replicating results from previous human psychological experiments than its predecessor. Thus, AMoC v4.0 can be further used as an educational tool to, for example, help teachers design better learning materials personalized for student profiles. Additionally, we release the code from AMoC v4.0 as open source in a Google Collab Notebook and a GitHub repository.
KW - Automated model of comprehension
KW - ChatGPT
KW - Large language models
KW - Natural language processing
KW - Reading comprehension
UR - http://www.scopus.com/inward/record.url?scp=85184661324&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85184661324&partnerID=8YFLogxK
U2 - 10.1016/j.chb.2024.108154
DO - 10.1016/j.chb.2024.108154
M3 - Article
SN - 0747-5632
VL - 154
JO - Computers in Human Behavior
JF - Computers in Human Behavior
M1 - 108154
ER -