The Hooper Visual Organization Test (HVOT) (Hooper, 1958) is a neuropsychology test of visual-spatial function and visual organisation. It was developed in 1958 and consists of 30 drawings of common objects and animals that are segmented into two or more pieces, which require mental rotation to identify and name each item (Giannakou & Kosmidis, 2006; Hooper, 1958; Lezak et al., 2012). It is scored by awarding one point for a correct response. Some items allow for partially correct responses, which receive half a credit, and zero credit is given to incorrect responses (Hooper, 1958). The standardised norms provided by Hooper (1958) were used to formulate cut-off scores that reduced the number of misclassification of individuals in each normative age group (Hooper, 1958) and cut-off scores of 20 to 25 were recommended (Hooper, 1958).

The HVOT is easy to administer and sensitive to the detection of visuospatial deficits that link to a wide range of neuropathologies (Booth & Happé, 2018; Boyd, 1981; Eberson, 2014; Ferreira-Correia, Anderson, Cockcroft & Krause, 2020; Gasparini et al., 2008; Mitolo et al., 2016; Paxton et al., 2007; Sanz Cortés, Olivares Crespo & Barcia Albacar, 2011). It has been found to be valid and reliable in different populations (Campagna & Ferreira-Correia, 2021; Giannakou & Kosmidis, 2006; Greve, Lindberg, Bianchini & Adams, 2000; Lin, Su, Guo & Wuang, 2012; Lopez, Lazar & Oh, 2003), although differences in cultural item appropriateness and item ranking have been noted (Merten & Beal, 1999; Su, Lin, Wu & Wuang, 2013). Moreover, achievement on the test seems to be influenced by age (DeVries, 2005; Miller et al., 2015; Su et al., 2013), level of education and gender (Campagna & Ferreira-Correia, 2021; Elias et al., 2011; Giannakou & Kosmidis, 2006; Merten & Beal, 1999).

Despite its clinical potential, the use of the HVOT in South Africa is limited by the lack of psychometric and normative data. It is widely accepted that using foreign norms in South Africa is not an adequate practice, but it is often the only option because local norms are unavailable. This issue is further compounded by the fact that country-wide norms are also not appropriate in South Africa. This is because of the major socioeconomic inequalities and disparities of educational opportunities connected with ethnicity (Watts & Shuttleworth-Edwards, 2016), as well as cultural and linguistic diversity (Foxcroft, Paterson, Le Roux & Herbst, 2004). These in turn are linked to wide differences between standardised tests scores of different samples, especially when South Africans are compared against foreign norms (Lucas, 2013).

This study intends to mitigate the socio-cultural biases in neurocognitive assessment by exploring the psychometric value of the HVOT in a sample of South Africans who do not speak English as a first language and who attended public school. Specifically, the objectives of the study are as follows: (1) to examine the reliability, (2) to determine whether demographic variables (age, years of education and/or gender) are associated with better performance on the HVOT, (3) to evaluate item difficulty through Rasch analysis, (4) to explore the diagnostic and convergent validity of the HVOT and (5) to provide normative data for the HVOT for a homogenous South African sample.

The data were symmetrical and normally distributed (Skewness statistic = 0.162; Kurtosis statistic of –0.490; Kolmogorov Smirnov = 0.079, df (111), Sig = 0.089; and Shapiro Wilk = 0.981, df (111), Sig = 0.108). Reliability (McDonald’s omega, ω_t = 0.90; Rasch person separation index = 0.87) for the 30 items of the HVOT was excellent.

Pearson’s correlations revealed a significant (p < 0.01) moderate negative correlation between age and the total HVOT score (r = –0.368), suggesting lowered scores in older participants. There was a weaker positive correlation between gender and HVOT total score (r = 0.268) which suggests that women performed better, (n = 63, x = 19.413, SD = 5.1223) than men (n = 48, x = 16.479, SD = 5.4762). There was a moderate positive correlation between years of education and HVOT total score (r = 0.343), indicating that participants with a higher level of education (12–22 years) performed better that those with a lower level of education (2–11 years).

We tested for group differences across age, gender and education. Importantly, only two groups were created for age and years of education, as further splitting would have made the groups too small given the sample size. The independent t-tests are reported in Table 2. Statistically significant differences across the groups were observed with women scoring higher than men (p = 0.004); younger participants (aged 19–40 years) scoring higher (p = 0.004) than older participants (aged 41–70 years); and individuals with more education (12–22 years) scoring higher (p ≤ 0.001) than those with less education (2–11 years). To ensure that these observed mean score differences are meaningful and not a result of item bias, we also tested for DIF. These results are reported below as part of the Rasch analysis.

The clinical group (Table 2) had statistically significantly lower mean HVOT total score (11.30 ± 6.56) compared to the control group (18.14 ± 5.45). These results provide good evidence for the diagnostic validity of the HVOT. Whilst the effect size is large (d = 1.8), the difference in sample size between the clinical and control group should still be noted, as it likely affected the statistical power of this test, increasing the chance of a Type I error.

Table 3 presents results of the Rasch analysis. The item measure column indicates item ‘difficulty’. It shows that the items do in general increase in difficulty from the beginning to the end of the measure, although there is substantial deviation from the expected monotonic progression. Whilst items 25 (Block), 29 (Ring), 28 (Key), 30 (Broom) and 26 (Lighthouse) appear to be the most difficult items, the easiest items included, 1 (Fish), 2 (Saw), 3 (Table), 11 (Apple) and 7 (Dog). The difficulty estimates for items 7, 11 and 25 are examples of surprising results, with item 25 being relatively more difficult than expected, whereas items 7 and 11 were relatively easier than expected.

In general, however, the items of the HVOT fit the expectations of the Rasch model well. The infit mean square values are all reasonably close to the expected value of zero, although items 12 and 27 had relatively larger values, leaning towards underfit.

With regard to DIF, slight variation was observed in the item location parameters across the groups of interest, with some items being relatively more difficult for one group whilst other items were somewhat more difficult for the other group. Such variation is expected, and in general, has the effect of cancelling out.

However, a few items were flagged for DIF. Slight DIF were observed on items 5, 1 and 6 for age, gender and education respectively. Moderate to large DIF was flagged on items 3, 9, 23, and 38 for age; item 3 for gender; and items 3 and 11 for education. Whilst concerning, in this preliminary research on the HVOT, these findings should probably just be noted as such given the modest sample size, requiring further research with larger samples in future. Should the same items again be flagged in subsequent work, there might be stronger reason to investigate possible causes for the observed DIF, and if no substantive reason can be identified, one could consider amending these items, or excluding them from the measure entirely should the problem persist. At this stage, however, such actions would be premature.

Table 4 presents the frequency of full score, half scores and zero scores for each of the HVOT items. The items for which participants scored zero most frequently, included 25 (Block), 29 (Ring), 28 (Key), 30 (Broom) and 26 (Lighthouse) suggesting that these were the most difficult items. The easiest items, with the highest percentage of correct answers in the sample were 1 (Fish), 2 (Saw), 3 (Table), 11 (Apple) and 7 (Dog).

Convergent validity was explored by correlating the total scores of the HVOT with different domains of the MOCA (Table 5). There were statistically significant correlations between all the respective domains and the MOCA total score with the HVOT total score, except for the MOCA orientation total. A moderate positive correlation was noted between the HVOT and the MOCA language total (r = 0,564, p ≤ 0,001). This means that participants who achieve a high score on the HVOT will likely achieve a high language total score on the MOCA. There was a moderate, positive correlation between the HVOT and the MOCA Delayed Recall Total (r = 0.395, p ≤ 0.01). Therefore, participants who obtain a high HVOT score are likely to achieve a high score on the MOCA delayed recall subtest, and vice versa. A moderate, positive strong correlation was also noted between the HVOT and the MOCA naming total (r = 0.354, p ≤ 0.01). Consequently, high HVOT scores are likely to be accompanied high MOCA naming total scores. Similarly, a moderate positive correlation was observed between the MOCA visuospatial executive total and the HVOT total (r = 0.193, p ≤ 0.005). Lastly, a moderate positive correlation was noted between the HVOT and the MOCA total score (r = 0,548, p ≤ 0,001), indicating a tendency that high scores on the HVOT present with high total scores in the MOCA.

Preliminary norms for the HVOT are presented in Table 6, stratified by age, level of education and gender. It is important to note that the age group 19–40 years, with an education of 2–11 years for both men and women, were excluded because of small sample sizes. The groups which showed the highest performance were men and women between the ages of 19–40 years with an education of 12–22 years. Lowest performance was seen in male participants in the 41–70 years-of-age category with an education of 2–11 years.

The primary focus of the present study was to develop stratified HVOT norms for a South African sample of participants that do not speak English as a first language and who have attended public primary and secondary schools. By selecting this specific sample, our study mitigated the effects of multi or bilingualism and quality of education as sources of biases in cognitive tests (Watts & Shuttleworth-Edwards, 2016). For this, the effects of sociodemographic variables (age, number of years of formal education and gender) on the HVOT total score were investigated. McDonalds Omega was used to determine the internal consistency reliability of the HVOT. Item response theory was used to further examine item functioning on the HVOT. This study also provided evidence of diagnostic and convergent validity, which is necessary for the evaluation of the clinical utility of this test in South Africa.

In this study, the highest means were obtained by the youngest and most educated groups (female mean = 20,5/SD = 4,6 and male mean = 20,5/SD = 5.1), but these were lower than the cut-off point of 21 suggested by Hooper (1958) and by a demographically similar group from Greece (mean = 25.43/SD = 2.17) (Giannakou & Kosmidis, 2007) and from Venezuela (50^th Percentile = 25) (Campagna & Ferreira-Correia, 2021).

Our results indicate that people with more education performed better than those with less. The effects of age on the total score of the HVOT are illustrated by the common use of this variable in the norms stratification (DeVries, 2005; Hooper, 1983; Tamkin & Jacobsen, 1984). Years of education are included less frequently in the HVOT norms, despite the impact of this variable in cognitive performance, and more specifically in the visuospatial function (Roldán-Tapia, Cánovas, León & García-Garcia, 2017). Two exceptions are the HVOT norms for the Venezuelan and Greek populations (Campagna & Ferreira-Correia, 2021; Giannakou & Kosmidis, 2006).

In our study, women also performed better than men. Although a gender bias in the visuospatial functions has been suggested (Hatta et al., 2015; Parsons et al., 2004), our study challenges this notion, as other studies suggest that men outperform women (Campagna & Ferreira-Correia, 2021; Hatta et al., 2015), whereas other reports did not find any significant relationship between gender and HVOT scores (Giannakou & Kosmidis, 2006). Future studies in South Africa should explore the potential contribution of gender towards the total score of the HVOT whilst controlling for age and years of education.

Whilst the group differences described above are noteworthy, results from the DIF analysis should be noted. The items flagged for DIF across age, gender and education may have contributed somewhat, and to varying degrees, to these observed mean score differences. However, the influence of these items is likely to be minor, as relatively few items were affected in each case. These results should be investigated and confirmed in future research with larger samples to determine if the DIF results observed in this study are indeed robust.

The results from the current investigation suggest that the HVOT has good reliability with satisfactory estimates observed for both McDonalds Omega total and Rasch person reliability. These were consistent with reliability estimates reported in other work (Campagna & Ferreira-Correia, 2021). Item response theory analysis supported the construct validity of the HVOT with all items fitting the Rasch model. When inspecting the item location parameters, there was a clear progression in item difficulty with a few unexpected results, with some items being easier than anticipated whilst others were more difficult. When comparing the item frequencies reported in Table 4 to that of DeVries (2005), none of the items in this sample obtained 100% correct responses. Item one (fish), however, was considered to be the easiest, with only 1% of the participants giving incorrect responses. Surprisingly, item 11 was amongst the easiest items, whilst item 25 (block) was one of the most difficult. Hence, when clinicians administer the HVOT, the order of administration should follow empirical data that is context specific (Campagna & Ferreira-Correia, 2021). When administration takes place by presenting the items in the order of difficulty, there is a probability that one takes into account the application of a discontinuation rule (Campagna & Ferreira-Correia, 2021) which yields good discriminatory power (Wetzel & Murphy, 1991). It also decreases the time taken to administer the test as well as levels of fatigue for the patient (Campagna & Ferreira-Correia, 2021). However, given the findings of the current study, no discontinuation rule should be applied when administering the HVOT in the South African context.

In this study, we presented the HVOT norms for the South African population stratified by age, gender and years of education. All these variables had a significant correlation to the total HVOT score. To our knowledge, no other norms for the HVOT are available in this country. The mean HVOT scores of adults in Georgia, which were stratified by age (Tamkin & Jacobsen, 1984), revealed similar mean scores which were obtained for the specific South African population, as was defined previously. Also, a more recent normative study conducted on the Venezuelan population demonstrated a significant association between age, gender and level of education (Campagna & Ferreira-Correia, 2021), much like the current study. It is, however, important to consider that the generalisability of the current norms may be questionable. One of the reasons is that the use of quota sampling represented a limitation because some of the resulting subgroups were too small to be representative of a particular set of demographics (e.g. the age group of 19–40 years who had an education level of 2–11 years for both men and women).

A potential confounding association between naming abilities and performance on the HVOT has been reported (Greve et al., 2000), but it has been challenged (Paolo, Cluff & Ryan, 1996). In our investigation, a strong correlation was evident between the HVOT and the MOCA language total (r = 0.564, p = 0.000) and the MOCA naming total (r = 0.354, p = 0.000). This study therefore supports the claim that naming ability may have an impact on HVOT performance. Additional research should consider incorporating other psychometric measures apart from the MOCA in order to further validate the HVOT’s association with naming ability, and South African clinicians should consider assessing naming ability in the language of assessment when using the HVOT.

Furthermore, the significant correlation between the total MOCA and the majority of the sub-component scores and the HVOT total score would support the argument that the HVOT can act as a screening test. Studies have shown that visuo-perceptive tests like the HVOT can be multifactorial (Campagna & Ferreira-Correia, 2021; DeVries, 2005) as it indirectly recruits several cognitive functions beyond the core one. This may support the value of the HVOT as a screening tool, although keeping in mind the limitations of these tasks as diagnostic tools (Roebuck-Spencer et al., 2017).

Furthermore, the fact that answers can be accepted in different languages allows for assessment of linguistically diverse patients. However, this needs to be further explored in studies that better control for this. Although this was accepted, answering in the participants’ home language was not overtly encouraged. Therefore, the impact of these linguistic variables (naming capacity, English proficiency, and choosing to provide answers in different languages) on the the psychometric properties of the HVOT remains to be investigated.

In terms of the HVOT’s ability to discriminate between the normal control and clinical group, it was apparent that the clinical group performed significantly worse than the healthy control group. This supports literature which states that patients with HD/HDL2 often present with visuo-constructive deficits (Gómez-Tortosa, Del Barrio, Barroso & García Ruiz, 1996), and that the HVOT is known to be able to discriminate these cognitive dysfunctions in patients (Azambuja et al., 2012). Given that this study only included a small sample of HD/HDL2 clinical population, the generalisability on the results to HD/HDL2 and other pathologies is limited.

Both the clinical and control groups comprised of small sample sizes. However, literature suggests that it is better to make use of well-matched, small homogenous groups (n > 5) rather than large, heterogenous groups (Crawford & Garthwaite, 2012). A well-defined and homogenous sample was selected for this project and is representative of a large proportion of the South African population. Therefore, this study may have significant value for clinicians using the HVOT in this context, despite the small sample size.

South African clinicians working with patients with demographic characteristics similar to our sample are encouraged to use the adapted version of the HVOT and the stratified norms provided in order to reduce the biases caused by the use of non-representative norms. Future studies should expand the current control sample to include participants with different demographic characteristics (e.g. English first language speakers and younger and older adults) and better explore the construct validity of this test.

This study represents an important contribution to the literature on psychological assessment in South Africa, as it demonstrates the psychometric properties and potential of the HVOT and provides preliminary stratified normative data for South African polyglot adults who do not speak English as a first language and attended public schools. The test yielded good reliability, convergence and discriminatory validity, although the item difficulty values did not follow the expected monotonic increase. The total HVOT correlated significantly with age, years of education and gender.