Commentary

Ensuring ethical test use in South Africa: The role of Assessment Standards South Africa

Sumaya Laher, Marié de Beer, David J.F. Maree, David A. Bischof

Received: 14 Oct. 2024; Accepted: 20 Jan. 2025; Published: 17 Mar. 2025

Copyright: © 2025. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Historically, many psychological tests in use in South Africa are imported from Western contexts, creating challenges in applying them across South Africa’s diverse cultural and linguistic groups. The Employment Equity Act No 55 of 1998 prohibits the use of psychological assessments unless they are scientifically validated, reliable and unbiased for use with South Africans. Furthermore, the Health Professions Council of South Africa (HPCSA) plays a key regulatory role in ensuring that only qualified psychology professionals use these tests. The HPCSA’s Professional Practice Committee is currently responsible for classifying tests as psychological or not. The Professional Practice Committee does not evaluate the quality of tests. Recognising gaps in the regulatory framework, the three organisations most involved with psychological assessment in practice (the Psychological Society of South Africa [PsySSA], the Society for Industrial and Organisational Psychology of South Africa [SIOPSA] and the Association of Test Publishers [ATP]) collaborated to form Assessment Standards South Africa (ASSA). As a non-governmental body, ASSA oversees the quality of assessments, ensuring that they meet local and international standards. Assessment Standards South Africa’s guidelines emphasise transparency, ethical standards and the importance of local research to ensure the relevance of tests. Through initiatives such as the Assessment Standards Test Review System, ASSA has streamlined test certification and review processes, promoting responsible and ethical use of psychological and other assessment devices in South Africa. Assessment Standards South Africa offers a best practice model for test reviews to ensure responsible and ethical use of tests in South Africa.

Keywords: bias; ethics; psychology; psychological assessment; reliability; test review; validity.

Introduction

In South Africa, the review of psychological tests is governed by professional guidelines to ensure fairness, reliability and cultural appropriateness. Given South Africa’s diverse population, one of the main challenges is ensuring that psychological tests are suitable for all cultural and linguistic groups. Historically, many tests were imported from Western contexts, leading to concerns about their applicability in South Africa. As a result, there has been a growing emphasis on developing and validating locally relevant tests or adapting international tests to suit the local context. This includes language translation, cultural adaptation and ensuring that constructs being measured are meaningful across different groups. According to the Employment Equity Act No 55 (Government Gazette, 1998), psychological testing and similar assessments are prohibited unless they are scientifically validated, reliable, fair and unbiased. This legislation is crucial in a diverse country such as South Africa, where cultural, linguistic and socio-economic differences can impact the fairness and applicability of psychological assessments (Laher, 2024).

The Professional Board for Psychology (PBP) at the Health Professions Council of South Africa (HPCSA) plays a crucial role in regulating psychological testing and assessments in South Africa. As of February 2019, the Psychometrics Committee was responsible for overseeing test classification, education and training in psychometrics and psychological assessment. Its tasks included among others the development of training standards related to psychometrics and psychological assessments. These guidelines inform the accreditation of qualifications, university programmes, internships, national board examinations and continuing professional development. Ethical guidelines for test use and standards for evaluating the suitability of psychological tests are also formulated by the committee (HPCSA, 2019).

In addition, the committee is responsible for classifying devices, instruments, methods or techniques aimed at assessing emotional, cognitive and behavioural processes; personality, intellectual abilities; psychopathology and interest. Classification entails verifying whether a test is measuring a psychological construct or not. This applies to both South African and imported instruments. The committee reports its findings to the Professional Board and annually publishes a list of classified psychological tests and psychometric instruments (HPCSA, 2019, 2021, 2022).

If a test measures a psychological construct, it is published on the list of classified tests. All tests that appear on the list of classified tests may only be used by appropriately qualified psychology professionals who are registered with the HPCSA (Foxcroft et al., 2023). There are no costs attached to test classification applications to the PBP (HPCSA, 2019). As of 2024, the Psychometrics Committee was incorporated into the Professional Practice Committee at the PBP at the HPCSA, and this committee oversees classification applications.

The ethical guidelines, as outlined in Chapter 5 of Annexure 12 of the Health Professions Act, emphasise the importance of cultural sensitivity and the avoidance of outdated or obsolete tests (Government Gazette, 2006). Practitioners must be aware of the limitations of the assessments that they use, especially in diverse contexts, and must strive to use current and relevant tests for the South African context (Foxcroft & De Kock, 2023; Hassem et al., in press; HPCSA, 2010). However, with the HPCSA only classifying tests as psychological or not, there was no oversight on whether tests and other similar assessment devices were reliable, valid, unbiased and fair for use in South Africa.

Assessment Standards South Africa – A brief history

Assessment Standards South Africa (ASSA) was established in response to the need for an independent body to oversee the quality of assessment instruments in the country. Officially launched in 2022, ASSA is a non-governmental and non-profit organisation that aims to set and maintain high standards for assessments. The formation of ASSA was driven by recommendations from a task team from within the psychology and psychometrics fraternity that identified gaps in the existing regulatory framework with regard to certifying tests (psychological and non-psychological) in terms of appropriateness for use in the South African context.

Assessment Standards South Africa’s structure comprises directors from various industry stakeholders, including the Society for Industrial and Organisational Psychology of South Africa (SIOPSA), the Psychological Society of South Africa (PsySSA) and the Association of Test Publishers (ATP). This diverse representation ensures that ASSA’s standards and processes are informed by a broad range of perspectives and expertise. The organisation includes roles such as an editor, project manager and subject matter experts, all working towards the common goal of promoting responsible and ethical test standards. Assessment Standards South Africa aims to be self-sustaining through fees for test reviews, ensuring its independence and ability to continue its vital work.

The core activities of ASSA revolve around setting minimum standards for all testing activities in South Africa. This includes maintaining a website and online platform that provides information on best practices, successful test review results and access to testing information (see, https://www.assa.co.za/). Assessment Standards South Africa’s activities are not limited to psychological tests but extend to all assessment procedures, as specified in the Employment Equity Act (Government Gazette, 1998). When a test meets the criteria for classification as a psychological test, ASSA informs the applicants about the necessary submission to the Professional Practice Committee at the PBP at the HPCSA.

Developing the Assessment Standards South Africa test review guidelines

The development of the ASSA test review guidelines followed a structured, multi-year process to improve psychological assessment practices in South Africa. In 2018, key stakeholders from the HPCSA PBP, ATP, PsySSA and the International Test Commission held panel discussions at the SIOPSA and PsySSA conferences to explore the need for a formal review process for psychological assessments. This early stage involved conducting extensive research into international test review models, including those used by the Dutch Committee on Tests and Testing (COTAN), the Buros Center for Testing in the United States, the British Psychological Society and the Norwegian system.

In 2019, ASSA was officially created, and working committees were established to focus on formulating potential review processes and guidelines specific to the South African context. These committees engaged in stakeholder meetings with industry experts and consulted with the Department of Labour and the HPCSA PBP. A working committee was formed to draft criteria for the review process, benchmarked against the European Federation of Psychologists Associations (EFPA)¹ guidelines (Evers et al., 2013) but adapted to suit the unique needs of the South African context. Assessment Standards South Africa provides clear guidelines to encourage developers to strive for excellence and allows for contextualisation relevant to the test’s application.

The review criteria include reliability, validity, bias/equivalence information, and norms are specified in the ASSA test review form (see, https://www.assa.co.za/wp-content/uploads/2020/12/ASSA-Test-Review-Form.pdf). The document outlines the ASSA process and provides comprehensive guidelines for evaluating key aspects of tests, organised into several main sections. The reliability information section offers criteria for assessing internal consistency, test-retest reliability, equivalence and inter-rater reliability, emphasising evidence appropriate for the test’s intended use. The validity information section addresses content validity, construct validity (e.g., factor analysis, group differences, convergent/discriminant validity) and criterion-related validity, stressing the importance of equivalence evidence for adapted or translated tests. The bias and equivalence information section focusses on factor structure invariance, differential item functioning and score similarities across groups. The norms section evaluates the appropriateness, representativeness and recency of norm samples for various interpretations. The review form employs a 5-point rating scale (N/A [not applicable], 0–4) with clear criteria for rating aspects of the test from inadequate to excellent. The overall quality of a test is determined by the total score across completed sections with minimum scores for all sections. The guiding principles for the review criteria prioritise clarity, transparency and objectivity, establishing minimum certification standards while encouraging test developers to aim for excellence (ASSA, 2025).

In 2020 and 2021, ASSA presented these guidelines at various conferences and initiated a pilot project with ATP members to test the efficacy of the guidelines as well as of the proposed online assessment certification process. Finally, in 2022, ASSA completed the development of the Assessment Standards Test Review System (ASTRA; see, https://astra.assa.co.za/), which was subsequently launched to formalise the ASSA psychological assessment review process.

Overview of the test submission process

With previous systems used for test review in South Africa, the criteria used to evaluate tests were not always clear. Assessment Standards South Africa’s test review process is characterised by its clarity, transparency and objectivity. The test submission process to ASSA is designed to be comprehensive and rigorous. Applicants must create a user profile on the ASTRA platform, agree to minimum criteria and conditions, provide general test information and upload the required documentation. The pre-review checklist includes materials such as administration manuals, test and answer sheets and supplementary documents. The manual must provide a rationale for the test, a description of its development and detailed information on administration, scoring and interpretation procedures. South African research is a prerequisite unless global evidence is presented with a strong justification, ensuring that the tests are relevant and applicable to the local context.

A submission fee is also part of the process. As a non-profit organisation, ASSA’s fee structure is designed to cover the costs of test reviews and ensure the sustainability of the organisation. The fees collected are used to upgrade the website and reimburse the editor, reviewers and administrator. Assessment Standards South Africa offers a subsidy for ATP South African members on their first submission per year, promoting accessibility and encouraging test submissions. If a test is found non-compliant, ASSA recommends improvement and offers a reduced fee for resubmission within 6 months. This approach encourages continuous improvement and aims to provide support so that all tests can meet the required standards.

Overview of the test review process

In the past, in South Africa, submissions for test reviews were long and arduous. Test review bottlenecks were previously associated with a lack of capacity. The ASSA process involves having two reviewers for each submission to ensure a thorough evaluation of the test’s psychometric quality. Assessment Standards South Africa has also introduced a system of a third shadow reviewer (reviewer in training) to ensure that the pool of reviewers expands continuously ensuring the sustainability of the review system. Furthermore, the process also includes feedback meetings with the editor, providing opportunities for clarification and resubmission if necessary. This iterative process ensures that the process is transparent and inclusive, and that tests meet the minimum standards before being certified.

Implications for testing in South Africa

From 2021 to 2024, ASSA has reviewed and certified a total of 24 tests that have been submitted, with 18 certified and 6 returned for further resubmission or review. Tests that have received certification may be viewed on the ASSA website (see https://www.assa.co.za/certified-tests/). This list indicates the name of the test, the test publisher/distributor and the date at which the test received certification. The test review system has advanced beyond just a proof of concept and is functioning efficiently. Through the annual engagements that ASSA conducts at local psychology conferences, there is increased awareness among test developers, test publishers and registered psychology practitioners that tests being used in South Africa must provide psychometric evidence to support their utility regardless of whether these are submitted to ASSA or not. Moreso, this process has highlighted the necessity for South African research evidence to support the use of tests in South Africa. The requirement for test quality is often misconstrued as being relevant only to psychological assessments. Assessment Standards South Africa has emphasised the necessity of this for all tests, not just those that may be classified as psychological assessments.

Through the process of test reviewing, it has become evident that test publishers are very willing to engage in processes that support ethical and responsible test use but are struggling to find large, representative samples. Oftentimes. the South African research evidence presented is based on small local sample sizes. However, the utility of these samples should not be underscored. They offer value particularly in niche contexts where tests may be employed (Laher & Cockcroft, in press).

Across the reviews and through the feedback sessions, stakeholders have also received support and guidance on the various methods used to evaluate a test’s reliability, validity, the lack of bias and fairness in line with international best practice. While ASSA’s assessment evaluation criteria align with internationally recognised systems including those as postulated in the latest draft of the EFPA guidelines (Baeza et al., 2025), questions remain about the applicability of these criteria to new-age assessments particularly technology-based assessments. Nonetheless, ASSA has reviewed tests that incorporate gamified principles among others and learnt much about using the traditional psychometric canons of reliability, validity and bias with these assessments. These lessons have informed the continuous refinement of ASSA’s processes, ensuring that they remain relevant and effective in promoting high-quality assessments.

Assessment Standards South Africa is exploring opportunities for collaboration with the HPCSA to ensure reliable, valid and unbiased tests are used in South Africa. This collaboration could involve creating more explicit guidance around using obsolete tests and mutual agreement on referring submissions to each other as needed. Other areas for discussion have also been identified including the scope of ASSA’s focus, engagement with industry and government and the evolving assessment landscape in relation to issues of equity and social justice.

Conclusion

Assessment Standards South Africa represents a significant advancement in the regulation and certification of assessment instruments, both psychological and non-psychological, in South Africa. By establishing clear standards and processes, ASSA aims to improve the quality of assessments and promote ethical testing practices. Through collaboration with professional organisations and adherence to international benchmarks, ASSA is well positioned to lead the industry towards responsible and effective assessment practices.

Acknowledgements

We would like to thank Dr Ruwayne Kock, Prof. Hennie Kriek, Dr Cas Prinsloo, Dr Nicola Taylor, Ms Christa Porter and the late Prof. Deon Meiring for their contributions to ASSA.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article. S.L. is the Editor-in-Chief of the African Journal of Psychological Assessment and D.J.F.M. is an executive editor of the African Journal of Psychological Assessment. The peer review process for this submission was handled independently, and the authors had no involvement in the editorial decision-making process for this manuscript. The authors has no other competing interests to declare.

S.L. conceptualised the paper and wrote the first draft. M.d.B., D.J.F.M. and D.A.B. assisted with revising the paper for submission.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data sharing is not applicable to this article as no new data were created or analysed in this study.

The views and opinions expressed in this article are those of the authors and are the product of professional research. The article does not necessarily reflect the official policy or position of any affiliated institution, funder, agency or that of the publisher. The authors are responsible for this article’s results, findings and content.

References