Eric K. H. Chan, Bruno D. Zumbo, Ira Darmawanti, Olievia P. Mulyana
Validity is a fundamental issue in the development and evaluation of psychometric instruments in health care. With an aim towards informing validation practice, we investigated the reporting of validity evidence in the field of health care, focusing on a journal that, by design and scope, is meant to influence health outcomes researchers, technology developers, regulators, policy makers, and health economists: Value in Health, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) official journal. A systematic search of papers since the journal’s inception (January 1998) to December 2010 was conducted using the official journal website. Only empirical psychometric validation papers were included in the analysis. The coding was conducted by two of the authors independently. A total of 68 articles met our inclusion criteria. The number and percentage of reports of the broad categories of the sources of validity evidence included internal consistency reliability (69.1 %), construct (50.0 %), discriminant (33.8 %), convergent (33.8 %), content (25.0 %), criterion (20.6 %), response processes (4.4 %), and consequences (2.9 %). Researchers conducting validation studies are not relying on only one source of validity evidence at the exclusion of all others and hence representing a broad perspective on psychometric validity. However, some sources of validity evidence such as response processes and consequences are rarely reported. Readers and practitioners should not uncritically accept the often-stated conclusion that “the instrument is valid”. © Springer International Publishing Switzerland 2014.
Measurement, Evaluation, and Research Methodology (MERM) Program, Department of Educational and Counseling Psychology and Special Education, The University of British Columbia, 2125 Main Mall, Vancouver, V6T 1Z4, BC, Canada; Department of Educational Psychology and Guidance, State University of Surabaya, Ketintang Baru XIV/2 East Java, Surabaya, 60231, Indonesia
