Seminar in Psychometrics
A Latent Dirichlet Allocation Model of Action Patterns
Date and time: March 8, 2022 (3:40 PM CET)
Place Remotely on Zoom, projected to K4, MFF UK
Abstract: Action pattern data are process data often recorded in a computer-based large-scale testing setting and extracted from log files. The action pattern data portray different actions that test takers use to solve a given item. This research uses unsupervised and supervised latent Dirichlet allocation (LDA) topic modeling on action pattern data from a large-scale assessment. Topic modeling, which includes the LDA model, is a machine learning framework to rapidly discover latent topics from large quantities of open-ended qualitative textual data quantitatively. In this research, action pattern data from a large-scale assessment are treated as qualitative textual data to be analyzed with LDA. These latent topics amount to thematic annotations of a collection of documents referred to as a corpus. For the qualitative action pattern data, the LDA model treats documents (here a student’s set of action patterns on an item) as being represented by a random mixture over latent topics where a distribution over words represents each latent topic. As the results of this study demonstrate, the latent topics derived from the action pattern data can provide helpful insight into different cognitive processes and key actions that lead to item success or failure. For instance, this research provides evidence of classes of problem-solving strategies derived from topic distributions of action pattern data and how these strategies are predictive of item success or failure.
References.
Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77-84.
doi: 10.1145/2133806.2133826
Tang, X., et al. (2020). Latent feature extraction for process data via multidimensional scaling. Psychometrika, 85(2), 378-397.
doi: 10.1007/s11336-020-09708-3
Cintron, D. W., & Montrosse-Moorhead, B. Integrating Big Data Into Evaluation: R Code for Topic Identification and Modeling.
American Journal of Evaluation (2021).
doi: 10.1177/10982140211031640
