Original Research

The development of the Conceptual Construct Validity Appraisal Checklist

Mario R. Smith, Erica Munnik
African Journal of Psychological Assessment | Vol 5 | a121 | DOI: https://doi.org/10.4102/ajopa.v5i0.121 | © 2023 Mario R. Smith, Erica Munnik | This work is licensed under CC Attribution 4.0
Submitted: 01 October 2022 | Published: 01 November 2023

About the author(s)

Mario R. Smith, Department of Psychology, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa
Erica Munnik, Department of Psychology, Faculty of Community and Health Sciences, University of the Western Cape, Cape Town, South Africa

Abstract

The validation of instruments is integral to sound psychometric assessment practices. Construct validity, as the apex indicator, is established largely based on data-reduction techniques. This bias overlooks the importance of expert-based content validity and alternate approaches to construct validity. The construct definition, object classification, attribute classification, rater identification, scale formation, and enumeration and reporting (C-OAR-SE) framework provides an alternative to traditional approaches to construct validation. However, the theory lacks articulation into an appraisal instrument that can form an integral part of the methodology used for validation studies. There is a need to focus on the conceptual clarity of constructs and its subsequent impact on operationalisation as a prerequisite to data reduction. This study reports on the development of the Conceptual Construct Validity Appraisal Checklist (CCVAC), based on the C-OAR-SE framework. The CCVAC assesses conceptual construct validity as the logical underpinning of instruments. The CCVAC has three sections, assessing theoretical definitions, operational definitions and scoring, respectively. The measure produces subsection and section scores and a global interpretation matrix. The construction followed a rigorous five-step process. The CCVAC was piloted on the Emotional Social Screening Tool for School Readiness (E3SR). The CCVAC performed well and appears non-sensitive to theoretical frameworks. The E3SR achieved a moderate level of construct validity as estimated by the CCVAC. The kappa statistic (0.55) indicated a moderate inter-rater agreement. All ethics principles were adhered to.

Contribution: The CCVAC is theoretically grounded and provides a quantifiable methodology to objectively assess conceptual construct validity. The CCVAC makes the methodology underpinning construction explicit and produces quantifiable outcomes.


Keywords

conceptual definition; construct validity; operational definition; scale development; measurement

Sustainable Development Goal

Goal 9: Industry, innovation and infrastructure

Metrics

Total abstract views: 1318
Total article views: 1235


Crossref Citations

No related citations found.