The article “Rejoinder to McNeish and Mislevy: What Does Psychological Measurement Require?” by Klaas Sijtsma, Jules L. Ellis, and Denny Borsboom provides a detailed response to criticisms and discussions raised by McNeish and Mislevy regarding the role and application of the sum score in psychometric practices. The authors address core concerns while emphasizing the need for a balance between advanced psychometric techniques and practical, transparent approaches.
Background
The rejoinder builds on discussions about the use of the sum score versus more sophisticated latent variable models like item response theory (IRT) in psychological measurement. The authors argue for the responsible use of both methods, stressing that the sum score remains a useful tool in contexts where simplicity and transparency are essential. The conversation highlights a growing divide between psychology and psychometrics, urging collaboration to strengthen theoretical and practical foundations in the field.
Key Insights
- The Role of Psychometrics Education and Outreach: The authors emphasize the need for enhanced education and outreach to help researchers responsibly use advanced methods like IRT. They highlight that while training is essential, direct collaboration between researchers and psychometricians is crucial for practical application.
- Sum Score Versus Latent Variable Models: The article defends the sum score as a practical and transparent scoring method, particularly in scenarios where granular precision is unnecessary. The authors discuss its limitations but argue for its continued relevance alongside latent variable models.
- Bridging Psychology and Psychometrics: A key theme is the growing gap between psychology and psychometrics. The authors call for better integration of psychological theory with psychometric practices to enhance the validity and utility of measurement tools.
Significance
This work underscores the importance of aligning psychometric advancements with practical needs in psychological testing. By advocating for a dual approach that incorporates both sum scores and latent variable models, the authors address concerns about oversimplification without dismissing the value of transparency. The discussion contributes to ongoing debates about measurement theory, education, and application in psychology.
Future Directions
The authors suggest several areas for further research, including improving collaboration between psychology and psychometrics, exploring methods to balance simplicity and precision in scoring, and investigating the implications of machine learning and data-driven approaches in measurement and prediction. They also highlight the need for more robust theoretical development in psychological attributes.
Conclusion
The article effectively addresses concerns raised by McNeish and Mislevy, advocating for a balanced perspective on psychometric methods. By emphasizing education, outreach, and collaboration, the authors make a compelling case for bridging the divide between psychology and psychometrics to enhance the field’s theoretical and practical contributions.
Reference
Sijtsma, K., Ellis, J. L., & Borsboom, D. (2024). Rejoinder to McNeish and Mislevy: What Does Psychological Measurement Require? Psychometrika, 89, 11175–1185 (2024). https://doi.org/10.1007/s11336-024-10004-7
Frequently Asked Questions
What is item response theory?
Item Response Theory (IRT) is a modern psychometric framework that models the relationship between a person’s latent ability and their probability of answering test items correctly. Unlike classical test theory, IRT provides item-level analysis, enables computerized adaptive testing, and allows test scores to be compared across different test forms.
How does computerized adaptive testing work?
Computerized adaptive testing (CAT) uses IRT to select test items in real-time based on the test-taker’s responses. After each answer, the algorithm estimates ability and selects the next item that provides maximum information at that ability level. This typically achieves the same measurement precision as a fixed test using 50-80% fewer items.
People Also Ask
What is psychometrics: the science of psychological measurement?
The discipline of psychometrics emerged from two distinct yet complementary intellectual traditions. The first, championed by figures such as Charles Darwin, Francis Galton, and James McKeen Cattell, emphasized the study of individual differences and sought to develop systematic methods for their quantification. The second, rooted in the psychophysical research of Johann Friedrich Herbart, Ernst Heinrich Weber, Gustav Fechner, and Wilhelm Wundt, laid the foundation for the empirical investigation of human perception, cognition, and consciousness. Together, these two traditions converged to form the scientific underpinnings of modern psychological measurement.
Read more →What are refining reliability with attenuation-corrected estimators?
Jari Metsämuuronen’s (2022) article introduces a significant advancement in how reliability is estimated within psychological assessments. The study critiques traditional methods for their tendency to yield deflated results and proposes new attenuation-corrected estimators to address these limitations. This review examines the article’s contributions and its implications for improving measurement precision.
Read more →What is optimizing item parameter estimation for the generalized graded unfolding model?
Roberts and Thompson (2011) conducted a thorough analysis of item parameter estimation methods within the Generalized Graded Unfolding Model (GGUM). Their work focused on the performance of the Marginal Maximum A Posteriori (MMAP) procedure compared to other approaches, including Marginal Maximum Likelihood (MML) and Markov Chain Monte Carlo (MCMC). By conducting simulation studies, the authors provided evidence for MMAP’s effectiveness in addressing challenges associated with item parameter estimation.
Read more →Why is background important?
The rejoinder builds on discussions about the use of the sum score versus more sophisticated latent variable models like item response theory (IRT) in psychological measurement. The authors argue for the responsible use of both methods, stressing that the sum score remains a useful tool in contexts where simplicity and transparency are essential. The conversation highlights a growing divide between psychology and psychometrics, urging collaboration to strengthen theoretical and practical foundations in the field.
How does key insights work in practice?
The Role of Psychometrics Education and Outreach: The authors emphasize the need for enhanced education and outreach to help researchers responsibly use advanced methods like IRT. They highlight that while training is essential, direct collaboration between researchers and psychometricians is crucial for practical application. Sum Score Versus Latent Variable Models: The
Why does significance matter in psychology?
This work underscores the importance of aligning psychometric advancements with practical needs in psychological testing. By advocating for a dual approach that incorporates both sum scores and latent variable models, the authors address concerns about oversimplification without dismissing the value of transparency. The discussion contributes to ongoing debates about measurement theory, education, and application in psychology.
