COMPS Group is at IMPS 2026 in Seoul

For the full conference program, see the IMPS 2026 webpage.

Presentations & Symposium at IMPS2026

• Tuesday 12:45pm · Room 103, School of Business
  Jun Wu
  Identifiability of Partial Mastery DINA Models Show abstract

  Partial mastery (PM) cognitive diagnostic models (CDMs) extend traditional CDMs by replacing binary latent attribute mastery indicators with continuous mastery scores for multidimensional latent attributes. In PM-CDMs, each subject is characterized by a fixed continuous mastery score vector, from which item-specific binary attribute profiles are independently and identically generated. This formulation allows PM-CDMs to bridge classical CDMs and continuous latent variable models. Despite growing interest in PM-CDMs, their identifiability properties have not yet been formally established. In this work, we provide the first systematic study of identifiability for partial mastery deterministic input noisy “and” gate (PM-DINA) models. We derive sufficient conditions for identifiability that are directly analogous to known results for traditional CDMs. To illustrate the core argument, we analyze a minimal example with four items and two latent attributes, using symbolic computation to verify that the Jacobian of the model parameterization is nonzero at a generic point. By combining tools from analysis and algebraic statistics, we show that this local result implies finite-to-one identifiability of the item parameters and the marginal distributions of the relevant latent attributes. Finally, we show that if the Q-matrix contains such identifiable local structures for all attribute pairs, these results combine to yield identifiability of the full PM-DINA model. Our findings establish a theoretical foundation for estimation and inference in partial mastery CDMs.

  
  Related manuscript: Wu, J., Martinková, P., Erosheva, E. (2026). Identifiability of partial-mastery cognitive diagnostic models. arXiv. doi: 10.48550/arXiv.2607.16593
• Tuesday, 5:00pm · B132, The Commons
  Jan Netík
  Response-Free Rasch Difficulty Prediction via Option Interaction Modeling Show abstract

  Modeling the item difficulty from its wording alone is commonly approached by regressing difficulty estimates on hand-crafted item text features. This tradition has recently been challenged by transformer-based models—neural networks pretrained to represent natural language and then fine-tuned, using a small linear layer (a "regression head"), to predict the difficulty directly from the item wording (Peters et al., 2025). Despite achieving generally better results, the predictive performance remains modest, especially for multiple-choice reading comprehension items, where difficulty arises from the interplay of passage, question, and options rather than from surface text features—motivating further methodological work. Netík and Martinková (2026) recently demonstrated that an auxiliary question-answering (QA) objective can regularize the training and benefit the prediction, but a large portion of variance still remains unexplained. In this work, we propose a model that builds on the transformer architecture with the QA head and show how difficulty can be obtained through a transform linking the model outputs to the Rasch difficulty scale. To predict difficulty, we model the behavior of a simulated population of test takers. We train the QA model (a "Reader") on roughly 90,000 multiple-choice reading comprehension items (the RACE++ dataset)—using either only the information about which option is correct, or a probability distribution over the options given by a simulated population of test takers. Each option is represented jointly with the passage and question, and the per-option scores are converted to option-choice probabilities in exactly the functional form of Bock's (1972) nominal response model—so the competition among the options for the test taker's choice is modeled explicitly. Because the Reader's probability of a correct response then corresponds to a Rasch item response function marginalized over a population with some assumed ability density, we derive a closed-form approximation that inverts the Reader's log-odds of a correct response onto the Rasch difficulty scale, introducing two new parameters, a slope and an intercept. The complete model is trained with two loss functions—a mean squared error loss for the Rasch difficulty and an auxiliary Kullback–Leibler divergence loss aligning the Reader's predicted option distribution with the empirically observed one. To illustrate the approach, we use 791 items from the Cambridge Multiple-Choice Questions Reading Dataset (CMCQRD), with Rasch difficulties linked across four subpopulations on a common scale and with observed test-taker response distributions. Because the observed responses come from subpopulations differing in ability, the transform carries one intercept per subpopulation; we estimate the transform parameters by ordinary least squares (OLS) regression. We evaluate and compare under passage-grouped nested cross-validation. The proposed model with the closed-form inversion reaches an overall Spearman correlation with the linked Rasch difficulty of 0.559, exceeding the common fine-tuned transformer-with-regression-head baseline (0.422) by +0.138. Interestingly, further fine-tuning the RACE++-trained model on the CMCQRD data adds no significant benefit; only the transform parameters, which link the Reader output to a given population's ability distribution, need to be estimated—and they are easily obtained through OLS, which suggests that the approach is well generalizable. We hypothesize that further gains lie in simulating the population's option choices more faithfully, as passing the observed response distributions through the same inversion reaches a correlation of 0.975.

  
  Related manuscript: Netík, J., & Martinková, P. (2026). Response-free item difficulty modelling for multiple-choice items with fine-tuned transformers: Component-wise representation and multi-task learning. arXiv. doi: 10.48550/arXiv.2605.16991
• Wednesday, 3:30pm · B147, The Commons
  Michaela Vařejková
  Addressing Covariate Measurement Differences in Generalized Kernel Equating under the NEC Design Show abstract

  In non-equivalent groups designs, covariates may be used to support test score equating when anchor items are unavailable. Within the kernel equating framework, such situations are handled using generalized kernel equating (GKE) under the non-equivalent groups with covariates (NEC) design. The validity of this approach relies on the assumption of population-invariant conditional distributions of test scores given covariates. This assumption, however, may be violated when one or more covariates are measured using different test forms, potentially leading to biased equated scores. This work investigates a sequential equating strategy within the GKE–NEC framework, in which covariates measured on multiple test forms are equated prior to their inclusion in the primary test score equating. GKE is applied first to align the covariate distributions across forms, and subsequently to equate the target test scores conditional on the equated covariates. This procedure aims to improve the comparability of covariate distributions across groups and to strengthen the credibility of the population-invariant conditional distributions assumption. The proposed approach is evaluated through a simulation study varying covariate test difficulty, association between covariates and test scores, and sample size. Results show that standard GKE under the NEC design performs well when covariates are measured on comparable scales, but may be associated with increased bias when covariate measurement differs across groups. Applying the sequential equating strategy reduces bias, with a modest increase in the standard error of equating. The practical applicability of the approach is illustrated using data from a national upper secondary school leaving examination.

  
  Related manuscript: Vařejková, M., Martinková, P., & Potužníková, E. (2026). Sequential generalized kernel equating: Providing comparable scores across multiple test forms with nonequivalent groups and differently measured covariates. arXiv. doi: 10.48550/arXiv.2605.28559
• Thursday 1:45pm – 3:00pm · B126, Ballroom
  Patrícia Martinková (ICS CAS) and Gabriel Wallin (Lancaster University)
  Symposium: Measurement Non-Invariance Show abstract

  Measurement invariance and differential item functioning (DIF) remain central concerns in psychometrics, particularly as testing contexts grow more complex and data structures more heterogeneous. This symposium brings together four methodological contributions that extend DIF and invariance analysis beyond traditional frameworks. Dr. Hladká presents a social network-based extension of logistic regression DIF methods, identifying structured, test-wide patterns of non-invariance via community detection algorithms. Dr. Martinková examines treatment-related DIF in longitudinal designs, comparing plug-in and fully joint latent-variable approaches to account for baseline measurement error in estimating item-level treatment effects. Dr. Wallin proposes a framework for DIF detection under asymmetric IRT models with unknown groups and anchor items, integrating mixture modeling and regularized estimation within a penalized marginal likelihood framework. Finally, Ms. Cichrová addresses continuous DIF using spline-based varying-coefficient models under monotonicity constraints, highlighting computational and inferential trade-offs when modeling nonlinear covariate effects. Collectively, these presentations advance contemporary approaches to modeling non-invariance by incorporating latent heterogeneity, continuous covariates, network structure, measurement error, and asymmetric response processes. The symposium highlights methodological innovations that improve the detection, interpretation, and practical handling of complex DIF patterns in psychological and educational measurement.

  
  
  • Thursday 1:45pm · B126, Ballroom
    Adéla Hladká
    Beyond Individual DIF: A Social Network Framework for Identifying Structured Patterns of Non-Invariance Show abstract

    Measurement invariance is a critical assumption in psychological and educational testing, yet violations often occur in structured ways across subsets of items. Traditional differential item functioning (DIF) detection methods typically evaluate items one at a time by comparing examinees matched on the same ability level, which can miss systematic, test-wide patterns of non-invariance. We present a novel approach that combines classic logistic regression DIF detection with a network-based framework to identify clusters of items exhibiting related non-invariance. In this method, items are represented as nodes in a network, and edges capture similarity in item-level statistics derived from group comparisons. Pre-specified item blocks—corresponding to testlets or hypothesized item clusters—allow for targeted testing of local dependence (LD) differences between groups. Community detection algorithms are then applied to identify clusters of items with shared DIF and LD patterns. By focusing on structured patterns rather than independent signals, this approach provides a more nuanced understanding of measurement invariance violations. The framework also provides intuitive visualizations of item relationships and flagged DIF clusters, helping applied researchers with test construction, fairness evaluation, and diagnostic assessment of item-level dependencies.

  • Thursday 2:00pm · B126, Ballroom
    Patrícia Martinková
    Addressing Measurement Error in Treatment-Related Differential Item Functioning Show abstract

    Treatment effects are traditionally evaluated by comparing total test scores across groups. However, examining treatment effects at the item level may reveal influences on specific skills that remain undetected when analyzing aggregated scores. Conceptually, this corresponds to analyzing differential item functioning in change (DIF-C), or treatment-related DIF, to identify item-level heterogeneous treatment effects. Existing approaches to modeling DIF-C typically include pretest ability as a covariate, operationalized as either a standardized total score or a latent ability score estimated from an item response theory (IRT) model. These methods generally rely on a two-step procedure in which baseline ability is estimated first and subsequently treated as fixed. This motivates examining how baseline measurement uncertainty may affect estimation and inference for item-level treatment effects. When full item-level pretest data are available, multigroup structural equation models (SEMs) that jointly model pretest and posttest responses within a unified latent-variable framework offer a natural extension. Such models enable simultaneous estimation of latent ability, measurement error, change processes, and item-specific treatment effects, thereby incorporating baseline uncertainty directly into the DIF-C analysis. We compare plug-in approaches with different ability-score estimates and fully joint latent-variable models through a simulation study evaluating bias, Type I error, and power under varying levels of measurement precision and treatment heterogeneity. An empirical illustration uses longitudinal data from the German National Educational Panel Study and showcases the benefits of evaluating tracking effects at the item level.

    
    Related manuscript: Martinková, P., Hladká, A., & Potužníková, E. (2020). Is academic tracking related to gains in learning competence? Using propensity score matching and differential item change functioning analysis for better understanding of tracking implications. Learning and Instruction, 66, 101286. doi: 10.1016/j.learninstruc.2019.101286
  • Thursday 2:15pm · B126, Ballroom
    Gabriel Wallin
    DIF Analysis with Unknown Groups and Anchor Items for Asymmetric IRT Models Show abstract

    Item response theory (IRT) models traditionally assume symmetric item characteristic curves, yet empirical response processes are often asymmetric. While asymmetric IRT models have been shown to improve interpretability and theoretical coherence, differential item functioning (DIF) methodology has largely remained tied to symmetric model specifications. This mismatch risks confounding true non-invariance with model misspecification. The problem becomes even more challenging when focal and reference groups are unobserved and anchor items are unknown. Currently, no framework jointly addresses asymmetric item response functions, latent group membership, and data-driven anchor selection. We propose a novel DIF detection framework that integrates asymmetric IRT modeling with latent class analysis and regularized estimation. Specifically, we embed a complementary log-log (CLL) IRT model within a mixture structure, where class-specific difficulty shifts capture DIF and the asymmetric link accounts for skewed response behaviour. We impose an L1 penalty on class-specific difficulty shifts and estimate all parameters via penalized marginal maximum likelihood. This approach enables simultaneous estimation of item parameters, latent classes, and DIF effects without pre-specified anchors. A computationally efficient proximal EM algorithm is developed to handle the non-smooth penalty. Our simulation study demonstrates accurate recovery of measurement and structural parameters, and strong respondent classification and DIF detection performance. Additionally, we present an analysis of data from a large-scale educational test to illustrate the methodology. The proposed framework provides a unified and practically applicable solution for latent DIF detection under asymmetric IRT models.

    
    Related manuscript: Wallin, G., & Huang, Q. (2026). Latent Impact and Differential Item Functioning Analysis for Asymmetric IRT Models. arXiv. doi: 10.48550/arXiv.2605.05684.
  • Thursday 2:30pm · B126, Ballroom
    Michaela Cichrová
    Continuous DIF under Monotonicity Constraints: Methodological Challenges and Solutions Show abstract

    Differential item functioning (DIF) is most commonly assessed with respect to a categorical grouping variable, such as gender. However, in many applications it is of substantive interest to examine DIF with respect to a continuous covariate, for example, age. Although continuous covariates can be artificially discretized and analyzed using standard DIF methods, this approach entails a loss of information and may fail to capture potentially more complex effects. We consider spline-based varying-coefficient models for testing DIF with respect to a continuous covariate. These models allow item parameters to vary smoothly across the covariate while retaining sufficient flexibility to capture complex, potentially nonlinear effects. At the same time, the resulting item response functions must satisfy monotonicity with respect to the latent ability, which introduces substantial computational and inferential challenges. To address these challenges, we investigate monotone tensor-product smooths combined with post-hoc ANOVA-style decompositions to separate main and interaction effects. We further consider likelihood-based model comparisons between non-nested specifications, using bootstrap procedures for valid inference under shape constraints. Through a comprehensive simulation study, we compare alternative estimation and testing strategies under data-generating mechanisms representing non-DIF, uniform, and non-uniform DIF. Performance is evaluated in terms of Type I error control, statistical power, and accuracy of the estimated item response surfaces. The results demonstrate how monotonicity constraints affect DIF detection and clarify the methodological trade-offs that arise when combining flexible smooth interaction models with shape constraints.

  Related manuscripts:
  • Cichrová, M., Hladká, A., & Martinková, P. (2026). Refining effect size measures and classification for differential item functioning: Toward unified guidelines across methods. British Journal of Mathematical and Statistical Psychology. doi: 10.1111/bmsp.70060
  • Hladká, A., Martinková, P., & Brabec, M. (2026). New iterative algorithms for estimation of item functioning. Journal of Educational and Behavioral Statistics, 51(1), 175—205. doi: 10.3102/10769986241312354
  • Hladká, A. & Martinková, P., & Magis, D. (2024). Combining item purification and multiple comparison adjustment methods in detection of differential item functioning. Multivariate Behavioral Research, 59(1), 46—61. doi: 10.1080/00273171.2023.2205393
  • Hladká, A., & Martinková, P. (2020). difNLR: Generalized logistic regression models for DIF and DDF detection. The R Journal, 12(1), 300—323. doi: 10.32614/RJ-2020-014
  • Martinková, P., Hladká, A., & Potužníková, E. (2020). Is academic tracking related to gains in learning competence? Using propensity score matching and differential item change functioning analysis for better understanding of tracking implications. Learning and Instruction, 66, 101286. doi: 10.1016/j.learninstruc.2019.101286
  • Drabinová, A., & Martinková, P. (2017). Detection of differential item functioning with non-linear regression: Non-IRT approach accounting for guessing. Journal of Educational Measurement, 54(4), 498—517. doi: 10.1111/jedm.12158
  • Martinková, P., Drabinová, A., Liaw, Y.-L., Sanders, E. A., McFarland, J. L., & Price, R. M. (2017). Checking equity: Why DIF analysis should be a routine part of developing conceptual assessments. CBE—Life Sciences Education, 16(2), rm2. doi: 10.1187/cbe.16-10-0307
• Friday, 9:45am · B146, The Commons
  Marie Turčičová
  Statistical Modeling for Timing Just-in-Time Adaptive Interventions Show abstract

  Digital health interventions have become an important tool for supporting long-term behavior change by providing personalized guidance and real-time feedback. In a longitudinal study on wellbeing and digital technologies, individuals aiming to reduce weight and improve their lifestyle are monitored using repeated measures of body weight, wearable indicators (e.g., physical activity, sleep), and self-reported mental status. A key methodological challenge is detecting and preventing disengagement over extended follow-up periods, thereby enabling participants to sustain lifestyle changes beyond the formal intervention phase. We propose a statistical modeling framework for predicting substantial behavioral discrepancy and optimizing the timing of just-in-time adaptive interventions (JITAIs). The goal is to identify early signals of insufficient physical activity and declining engagement from heterogeneous, multivariate time series data and to determine when supportive or educational messages should be delivered to maximize intervention effectiveness while minimizing participant burden. Our framework replaces heuristic intervention rules with data-driven statistical decision models by estimating the probability that an intervention is needed, feasible, and likely to be effective. Using Bayesian hierarchical models, it accommodates repeated measurements, irregularly observed and partially missing data, and substantial between-person heterogeneity while borrowing strength across participants. The resulting probabilistic predictions provide a solid basis for personalized intervention timing.

  
  Related manuscript: Turčičová, M., Martinková, P. (2026). Asymptotically exact threshold for detecting anomalies in multivariate Gaussian data with application to time series. arXiv. doi: 10.48550/arXiv.2607.15637

See also other COMPS Group publications!