The Hooper Visual Organization Test (HVOT) was designed to measure an individual’s ability to organise visual stimuli and assess visual-spatial abilities and synthesis. The current investigation sought to explore the psychometric properties of the HVOT and develop normative data for South Africans who do not speak English as a first language and who received primary and secondary public education. The research design was cross-sectional and the HVOT was administered to healthy adults (n = 111) and a clinical group (n = 17) whose ages ranged between 19 and 70 years and had an education of between 6 and 22 years. The clinical group was made up of Huntington’s Disease patients (HD/HDL2). Reliability indicators (McDonald’s omega and Rasch person reliability index) were satisfactory. The HVOT fit the Rasch model well, although item locations deviated somewhat from the expected monotonic increase in item difficulties. Statistically significant differences in total scores were observed across age, education and gender groups, forming the basis of the norms presented in this paper. A few items across these groups were flagged for potential differential item functioning. Several statistically significant associations with the Montreal Cognitive Assessment (MOCA) were observed. These were consistent with theoretical expectations and provided evidence of convergent validity. The clinical group performed worse than the control group when mean total HVOT scores were compared. Preliminary norms stratified by age, gender and years of education are presented. Future studies should include larger sample sizes and additional research on the influence of gender on the total HVOT score is needed.
The Hooper Visual Organization Test (HVOT) (Hooper, 1958) is a neuropsychology test of visual-spatial function and visual organisation. It was developed in 1958 and consists of 30 drawings of common objects and animals that are segmented into two or more pieces, which require mental rotation to identify and name each item (Giannakou & Kosmidis, 2006; Hooper, 1958; Lezak et al., 2012). It is scored by awarding one point for a correct response. Some items allow for partially correct responses, which receive half a credit, and zero credit is given to incorrect responses (Hooper, 1958). The standardised norms provided by Hooper (1958) were used to formulate cut-off scores that reduced the number of misclassification of individuals in each normative age group (Hooper, 1958) and cut-off scores of 20 to 25 were recommended (Hooper, 1958).
The HVOT is easy to administer and sensitive to the detection of visuospatial deficits that link to a wide range of neuropathologies (Booth & Happé, 2018; Boyd, 1981; Eberson, 2014; Ferreira-Correia, Anderson, Cockcroft & Krause, 2020; Gasparini et al., 2008; Mitolo et al., 2016; Paxton et al., 2007; Sanz Cortés, Olivares Crespo & Barcia Albacar, 2011). It has been found to be valid and reliable in different populations (Campagna & Ferreira-Correia, 2021; Giannakou & Kosmidis, 2006; Greve, Lindberg, Bianchini & Adams, 2000; Lin, Su, Guo & Wuang, 2012; Lopez, Lazar & Oh, 2003), although differences in cultural item appropriateness and item ranking have been noted (Merten & Beal, 1999; Su, Lin, Wu & Wuang, 2013). Moreover, achievement on the test seems to be influenced by age (DeVries, 2005; Miller et al., 2015; Su et al., 2013), level of education and gender (Campagna & Ferreira-Correia, 2021; Elias et al., 2011; Giannakou & Kosmidis, 2006; Merten & Beal, 1999).
Despite its clinical potential, the use of the HVOT in South Africa is limited by the lack of psychometric and normative data. It is widely accepted that using foreign norms in South Africa is not an adequate practice, but it is often the only option because local norms are unavailable. This issue is further compounded by the fact that country-wide norms are also not appropriate in South Africa. This is because of the major socioeconomic inequalities and disparities of educational opportunities connected with ethnicity (Watts & Shuttleworth-Edwards, 2016), as well as cultural and linguistic diversity (Foxcroft, Paterson, Le Roux & Herbst, 2004). These in turn are linked to wide differences between standardised tests scores of different samples, especially when South Africans are compared against foreign norms (Lucas, 2013).
This study intends to mitigate the socio-cultural biases in neurocognitive assessment by exploring the psychometric value of the HVOT in a sample of South Africans who do not speak English as a first language and who attended public school. Specifically, the objectives of the study are as follows: (1) to examine the reliability, (2) to determine whether demographic variables (age, years of education and/or gender) are associated with better performance on the HVOT, (3) to evaluate item difficulty through Rasch analysis, (4) to explore the diagnostic and convergent validity of the HVOT and (5) to provide normative data for the HVOT for a homogenous South African sample.
MethodsParticipants
The secondary data obtained from a study titled ‘The Neurocognitive profile of Huntington’s Disease-Like 2’ was used for this study (Ferreira-Correia, 2019). The data were collected in stages where participants for the control and clinical Huntington’s Disease / Huntington’s Disease-Like 2 (HD/HDL2) groups were recruited. Given the rarity of HD/HDL2 in the South African population within the stipulated demographics, only 18 patients participated in the study. For the recruitment of the participants for the control sample, the researcher aimed to match the HD/HDL2 participants in terms of age, years of education, and language (English not first language).
Data collection occurred simultaneously for both the control and clinical group. Both groups were formed by means of purposive homogenous sampling (Leedy & Ormrod, 2015), so as to better match the demographics of the patients. The sample is described in Table 1.
Demographic characteristics of the clinical (HD/HDL2) and control groups.
Statistics
Diagnosis group
Clinical group (HD/HDL2)
N
%
Control group
N
%
(n = 17)
(n = 111)
Age (years)
Mean
43.94
-
-
47.37
-
-
SD
9.57
-
-
9.33
-
-
Min
32
-
-
19
-
-
Max
61
-
-
65
-
-
Gender
Male
-
12
70.6
48
43.2
Female
-
5
29.4
63
56.8
Education (years)
Mean
12.65
-
-
12.28
-
-
SD
2.47
-
-
2.77
-
-
Min
9
-
-
6
-
-
Max
18
-
-
22
-
-
HVOT score
Mean
11.29
-
-
18.14
-
-
SD
6.56
-
-
5.45
-
-
25th p
4.50
-
-
14.50
-
-
50th p
10.00
-
-
17.50
-
-
75th p
17.25
-
-
22.00
-
-
Min
3
-
-
5
-
-
Max
21
-
-
29
-
-
HVOT, Hooper Visual Organization Test.
To be included in the study, participants needed to be able to speak English. Potential participants with comorbid neurological or metabolic diseases, history of traumatic brain injury with loss of consciousness, abuse of illegal drugs, and/or who did not give formal consent were not included.
Design
This investigation is non-experimental as none of the variables have been manipulated. It involves a cross-sectional design because it investigates particular variables retrospectively, without directly interfering with it (Field, 2018).
Instruments
A demographic questionnaire (Ferreira-Correia, 2019) was used to collect data on age, level of education, occupation, language experience, gender and other medical variables relevant for the original study.
The Montreal Cognitive Assessment (MOCA) (Nasreddine, 2005) was administered after the demographic questionnaire. This screening tool includes items measuring executive function and visuospatial ability, confrontational naming, short-term memory, attention and working memory, language, concentration, verbal abstraction and orientation (Nasreddine et al., 2005). Each functional area was scored separately and then points were added for a total maximum score of 30 points.
Hooper Visual Organization Test (Hooper, 1958) – The HVOT consists of 30 line drawings that depict uncomplicated objects which have been cut into pieces and placed in a puzzle-like manner. The HVOT was the sixth test in a battery of 12 neuropsychological tests given to participants. During administration, the original version of the test was used, and the following protocol was adhered to (Hooper, 1958): if participants were unable to respond in English, answers were accepted in other languages and subsequently translated into English by a research assistant who was proficient in several South African languages. The scoring rules of the manual were followed, although adjustments to accommodate for linguistic variances were made.
For example, item 3, ‘bench’ was awarded a full point; item 4, ‘flying machine’ was awarded a full point; item 5, a full point was awarded if participants named any round ball, whilst ‘football’ and ‘rugby ball’ were awarded half a point; item 7, ‘sheep’ and ‘lamb’ were awarded a full point, and ‘animal’ was awarded half a point; item 8, ‘lorry’ was awarded a full point, ‘car’ or ‘vehicle’ was given half a point; item 9, ‘mug’ was awarded a full point, ‘jug’ was scored half a point; item 11, ‘peach, tomato, pumpkin, pear and the like’ were awarded a full point, and ‘fruit’ was awarded half a point; item 14, ‘hockey stick, walking stick and stick’ were awarded a full point; item 15, ‘boat and ship’ were awarded full points; item 16, ‘kettle and teapot’ were awarded full points; item 17, ‘couch’ was awarded a full point and ‘sofa’ was awarded half a point; item 19, ‘kettle’ was awarded a full point; item 20, ‘animal’ was awarded half a point; item 21, ‘pansy’ etc. were awarded full points; item 22, ‘rat, guinea pig’ etc. were awarded full points and ‘animal’ was awarded half a point; item 23, ‘bible and dictionary’ were awarded a full point; item 24, ‘animal’ was awarded half a point; item 25, ‘cube’ was awarded a full point; item 26, ‘house of the sea’ was awarded a full point, and ‘tower, castle, watch tower, church, tower or high place’ were awarded half points; item 27, ‘boot’ was awarded a full point; and item 29, ‘diamond ring’ was awarded a full point.
Procedure
The Human Research Ethics Committee (Medical) [redacted] granted ethical clearance for this study (clearance certificate number [redacted]). The original data for the study [redacted] was collected after obtaining clearance from Human Research Ethics Committee (Medical) (clearance certificate number: [redacted]). The Helsinki Declaration and the Singapore Statement on Research Integrity (Resnik & Shamoo, 2011) were honoured. Participants received written information and a detailed briefing about the study. Volunteers signed a written consent form before they participated in the assessment. The assessment was conducted in one session of approximately 2 h.
The data was captured in Research Electronic Data Capture (REDCap ®) (Harris et al., 2009) and then analysed and reported on. For the purpose of the current project, the data obtained from the demographic questionnaire, the HVOT and the MOCA were used. The HVOT tests were scored by a registered neuropsychologist ([redacted]. Quality control of the data was conducted by implementing two additional rounds of blinded scoring conducted by an independent clinical psychologist and by the first author ([redacted]). Statistical analysis and report writing were the final step of this report.
Data analyses
Data were analysed using SPSS Statistics 26, Winsteps (version 4.8.0.0; Linacre, 2015) and the psych package (Revelle, 2021) in R (R Core Team, 2021). Distribution of the data was explored through tests of normality which were run on the HVOT total scores of the healthy participant group. The sample was described using frequency distributions, measures of central tendency and variability in order to better define demographic variables and ranges to use for the normative data. Pearson’s correlations and independent samples t-tests were used to explore group differences on the total score of the HVOT. McDonald’s coefficient omega (ωt) was computed to evaluate the internal consistency reliability of the HVOT. Item response theory (specifically, Rasch analysis) was used to examine the construct validity of the HVOT. A partial credit model was computed to investigate fit to the Rasch model. Infit mean square values close to one were expected with values > 0.60 and < 1.40 considered to fall within an acceptable range. Item fit values outside this range were thought to misfit the model, with values < 0.60 and > 1.4 indicating overfit and underfit, respectively. In addition, we examined Differential Item Functioning (DIF) across the age, gender and education groups to ensure that the observed group differences were not a result of item bias (Bond & Fox, 2015). Diagnostic validity was investigated by using an independent samples t-test to compare HVOT total scores across diagnostic groups. Convergent validity was examined by correlating the total score of the HVOT with the selected scores on the MOCA.
Ethical considerations
The Human Research Ethics Committee (Medical) University of the Witwatersrand granted ethical clearance for this study (clearance certificate number M200669). The original data for the study ‘The Neurocognitive profile of Huntington’s Disease-Like 2’ were collected after obtaining clearance from Human Research Ethics Committee (Medical) (clearance certificate number: M140872). The Helsinki Declaration and the Singapore Statement on Research Integrity (Resnik & Shamoo, 2011) were honoured. Participants received written information and a detailed briefing about the study. Volunteers signed a written consent before taking part in the assessment. The assessment was conducted in one session of approximately 2 h.
Results
The data were symmetrical and normally distributed (Skewness statistic = 0.162; Kurtosis statistic of –0.490; Kolmogorov Smirnov = 0.079, df (111), Sig = 0.089; and Shapiro Wilk = 0.981, df (111), Sig = 0.108). Reliability (McDonald’s omega, ωt = 0.90; Rasch person separation index = 0.87) for the 30 items of the HVOT was excellent.
Pearson’s correlations revealed a significant (p < 0.01) moderate negative correlation between age and the total HVOT score (r = –0.368), suggesting lowered scores in older participants. There was a weaker positive correlation between gender and HVOT total score (r = 0.268) which suggests that women performed better, (n = 63, x = 19.413, SD = 5.1223) than men (n = 48, x = 16.479, SD = 5.4762). There was a moderate positive correlation between years of education and HVOT total score (r = 0.343), indicating that participants with a higher level of education (12–22 years) performed better that those with a lower level of education (2–11 years).
We tested for group differences across age, gender and education. Importantly, only two groups were created for age and years of education, as further splitting would have made the groups too small given the sample size. The independent t-tests are reported in Table 2. Statistically significant differences across the groups were observed with women scoring higher than men (p = 0.004); younger participants (aged 19–40 years) scoring higher (p = 0.004) than older participants (aged 41–70 years); and individuals with more education (12–22 years) scoring higher (p ≤ 0.001) than those with less education (2–11 years). To ensure that these observed mean score differences are meaningful and not a result of item bias, we also tested for DIF. These results are reported below as part of the Rasch analysis.
Independent samples t-test for age, gender and years of education and the Hooper Visual Organization Test total score.
t
df
Sig. (2-tailed)
Mean difference
Standard error difference
95% Confidence interval of the difference
Lower
Upper
HVOT total (gender)
−2.901
109
0.004
−2.9335
1.0112
−4.9376
−0.9294
−2.875
97.652
0.005
−2.9335
1.0204
−4.9586
−0.9085
HVOT total (age)
2.919
109
0.004
3.3655
1.1529
1.0805
5.6506
3.159
54.005
0.003
3.3655
1.0655
1.2293
5.5018
HVOT total (years of education)
−3.814
109
0.000
−3.8798
1.0172
−5.8958
−1.8637
−3.902
86.637
0.000
−3.8798
0.9942
−5.8559
−1.9036
HVOT total (diagnosis group)
−4.693
126
0.000
−6.8500
1.4598
−9.7389
−3.9612
−4.097
19.539
0.001
−6.8500
1.6720
−10.3430
−3.3570
HVOT, Hooper Visual Organization Test.
The clinical group (Table 2) had statistically significantly lower mean HVOT total score (11.30 ± 6.56) compared to the control group (18.14 ± 5.45). These results provide good evidence for the diagnostic validity of the HVOT. Whilst the effect size is large (d = 1.8), the difference in sample size between the clinical and control group should still be noted, as it likely affected the statistical power of this test, increasing the chance of a Type I error.
Table 3 presents results of the Rasch analysis. The item measure column indicates item ‘difficulty’. It shows that the items do in general increase in difficulty from the beginning to the end of the measure, although there is substantial deviation from the expected monotonic progression. Whilst items 25 (Block), 29 (Ring), 28 (Key), 30 (Broom) and 26 (Lighthouse) appear to be the most difficult items, the easiest items included, 1 (Fish), 2 (Saw), 3 (Table), 11 (Apple) and 7 (Dog). The difficulty estimates for items 7, 11 and 25 are examples of surprising results, with item 25 being relatively more difficult than expected, whereas items 7 and 11 were relatively easier than expected.
Rasch fit and Differential Item Functioning statistics.
Item
Measure
Standard error
MNSQ
DIF
Infit
Age
Gender
Education
1
−2.40
0.51
1.06
0.15
0.67
−1.19
2
−1.81
0.30
1.06
−0.45
0.09
0.19
3
−1.81
0.30
1.16
1.52
−1.87
−1.77
4
−0.62
0.14
1.00
0.30
−0.36
−0.05
5
0.33
0.12
0.93
−0.69
0.11
0.36
6
−0.44
0.13
1.07
0.27
−0.12
−0.63
7
−1.01
0.17
1.04
0.14
−0.17
0.58
8
−0.10
0.13
1.13
0.07
0.06
−0.45
9
−0.65
0.14
0.99
−1.27
0.25
−0.39
10
−0.85
0.16
0.93
0.16
0.10
−0.32
11
−1.39
0.21
0.93
−0.46
0.67
1.13
12
−0.17
0.12
1.29
0.10
−0.38
0.43
13
0.03
0.12
1.00
0.59
0.26
−0.14
14
0.66
0.12
1.06
0.00
−0.56
0.23
15
0.78
0.12
0.82
−0.42
−0.18
0.33
16
−0.66
0.14
1.01
0.09
0.00
−0.29
17
0.42
0.14
0.87
−0.40
−0.17
0.45
18
−0.78
0.15
1.06
0.26
−0.22
0.10
19
−0.35
0.13
0.86
−0.03
0.46
−0.24
20
0.11
0.12
0.91
−0.12
0.32
0.05
21
0.72
0.12
0.95
0.38
0.40
−0.35
22
0.43
0.12
0.72
−0.17
0.54
0.17
23
−0.66
0.14
0.86
−1.65
−0.13
0.19
24
0.52
0.12
1.06
0.05
−0.06
−0.19
25
1.99
0.18
1.06
−0.08
0.71
−0.08
26
1.58
0.16
0.86
−0.10
−0.03
0.25
27
1.03
0.13
1.37
−0.20
−0.50
0.45
28
1.64
0.16
1.01
0.97
0.20
−0.42
29
1.99
0.18
1.09
0.21
−0.33
0.38
30
1.49
0.15
1.11
0.00
−0.21
−0.21
DIF, Differential Item Functioning, MNSQ, mean-square or standardized fit statistics.
In general, however, the items of the HVOT fit the expectations of the Rasch model well. The infit mean square values are all reasonably close to the expected value of zero, although items 12 and 27 had relatively larger values, leaning towards underfit.
With regard to DIF, slight variation was observed in the item location parameters across the groups of interest, with some items being relatively more difficult for one group whilst other items were somewhat more difficult for the other group. Such variation is expected, and in general, has the effect of cancelling out.
However, a few items were flagged for DIF. Slight DIF were observed on items 5, 1 and 6 for age, gender and education respectively. Moderate to large DIF was flagged on items 3, 9, 23, and 38 for age; item 3 for gender; and items 3 and 11 for education. Whilst concerning, in this preliminary research on the HVOT, these findings should probably just be noted as such given the modest sample size, requiring further research with larger samples in future. Should the same items again be flagged in subsequent work, there might be stronger reason to investigate possible causes for the observed DIF, and if no substantive reason can be identified, one could consider amending these items, or excluding them from the measure entirely should the problem persist. At this stage, however, such actions would be premature.
Table 4 presents the frequency of full score, half scores and zero scores for each of the HVOT items. The items for which participants scored zero most frequently, included 25 (Block), 29 (Ring), 28 (Key), 30 (Broom) and 26 (Lighthouse) suggesting that these were the most difficult items. The easiest items, with the highest percentage of correct answers in the sample were 1 (Fish), 2 (Saw), 3 (Table), 11 (Apple) and 7 (Dog).
Frequencies of full credit, half credit and no credit for each Hooper Visual Organization Test item.
Item
1 credit
0.5 credit
0 credit
Frequent errors
n
%
n
%
n
%
Answer
n
%
1. Fish
102
99.03
-
0.00
1
0.97
-
-
-
2. Saw
101
98.06
-
0.00
2
1.94
-
-
-
3. Table
100
97.09
-
0.00
3
2.91
-
-
-
4. Airplane
86
83.50
-
0.00
17
16.50
-
-
-
5. Baseball
51
49.51
9
8.74
43
41.75
Kite
8
7.77
6. Hammer
79
76.70
-
0.00
24
23.30
Axe
12
11.65
7. Dog
91
88.35
3
2.91
9
8.74
-
-
0.00
8. Truck
54
52.43
16
15.53
33
32.04
-
-
0.00
9. Cup
84
81.55
5
4.85
14
13.59
Kettle
5
4.85
10. Hand
85
82.52
5
4.85
13
12.62
-
-
0.00
11. Apple
95
92.23
3
2.91
5
4.85
-
-
0.00
12. Basket
69
66.99
-
0.00
34
33.01
Net
10
9.71
13. Scissors
66
64.08
-
0.00
37
35.92
Hanger
7
6.80
14. Cane
44
42.72
-
0.00
59
57.28
Knife
15
14.56
0.00
-
0.00
-
0.00
Umbrella
10
9.71
15. Sailboat
39
37.86
-
0.00
64
62.14
Dress
7
6.80
0.00
-
0.00
-
0.00
Washing
5
4.85
16. Teakettle
83
80.58
-
0.00
20
19.42
-
-
0.00
17. Chair
35
33.98
39
37.86
29
28.16
-
-
0.00
18. Candle
87
84.47
-
0.00
16
15.53
Flower†
9
8.74
19. Teapot
72
69.90
-
0.00
31
30.10
-
-
0.00
20. Cat
62
60.19
3
2.91
38
36.89
Rabbit
10
9.71
21. Flower
42
40.78
-
0.00
61
59.22
Clouds
9
8.74
0.00
-
0.00
-
0.00
Mountain
5
4.85
0.00
-
0.00
-
0.00
Tree‡
32
31.07
22. Mouse
52
50.49
3
2.91
48
46.60
Pipe§
11
10.68
23. Book
87
84.47
-
0.00
16
15.53
-
-
0.00
24. Rabbit
44
42.72
7
6.80
52
50.49
Dog
7
6.80
0.00
-
0.00
-
0.00
Cat
7
6.80
25. Block
10
9.71
-
0.00
93
90.29
Box
11
10.68
0.00
-
0.00
-
0.00
House
31
30.10
26. Lighthouse
18
17.48
16
15.53
69
66.99
House¶
11
10.68
27. Shoe
29
28.16
-
0.00
74
71.84
Iron
19
18.45
0.00
-
0.00
-
0.00
Toilet seat††
19
18.45
28. Key
19
18.45
-
0.00
84
81.55
Cutting tool
5
4.85
0.00
-
0.00
-
0.00
Knife
23
22.33
0.00
-
0.00
-
0.00
Razor‡‡
9
8.74
29. Ring
11
10.68
-
0.00
92
89.32
Lock§§
13
12.62
30. Broom
21
20.39
-
0.00
82
79.61
Pumpkin
21
20.39
n = 103 (8 participants were excluded because of missing data).
, includes flowerpot
, includes tree with bird
, includes pipe for smoking; smoke pipe; smoking pipe
, includes rondavel; house with chimney
, includes toilet; toilet room; potty; seat for toilet; base for toilet
, includes razor blade
, includes padlock
Convergent validity was explored by correlating the total scores of the HVOT with different domains of the MOCA (Table 5). There were statistically significant correlations between all the respective domains and the MOCA total score with the HVOT total score, except for the MOCA orientation total. A moderate positive correlation was noted between the HVOT and the MOCA language total (r = 0,564, p ≤ 0,001). This means that participants who achieve a high score on the HVOT will likely achieve a high language total score on the MOCA. There was a moderate, positive correlation between the HVOT and the MOCA Delayed Recall Total (r = 0.395, p ≤ 0.01). Therefore, participants who obtain a high HVOT score are likely to achieve a high score on the MOCA delayed recall subtest, and vice versa. A moderate, positive strong correlation was also noted between the HVOT and the MOCA naming total (r = 0.354, p ≤ 0.01). Consequently, high HVOT scores are likely to be accompanied high MOCA naming total scores. Similarly, a moderate positive correlation was observed between the MOCA visuospatial executive total and the HVOT total (r = 0.193, p ≤ 0.005). Lastly, a moderate positive correlation was noted between the HVOT and the MOCA total score (r = 0,548, p ≤ 0,001), indicating a tendency that high scores on the HVOT present with high total scores in the MOCA.
Pearson correlation coefficient showing the relationship between the Hooper Visual Organization Test total score and different subtests of the Montreal Cognitive Assessment.
Scores
Pearson correlation
Sig. (2-tailed)
MOCA Visuospatial executive total (trails, cube, clock) & HVOT total
0.193*
0.042
MOCA Naming total & HVOT total
0.354**
< 0.001
MOCA attention total (digits, tapping, subtraction) & HVOT total
0.291**
0.002
MOCA language total (naming, repetition, fluency) & HVOT total
0.564**
< 0.001
MOCA abstraction total & HVOT total
0.296**
0.002
MOCA delayed recall total & HVOT total
0.395**
< 0.001
MOCA orientation total & HVOT total
0.044
0.650
MOCA total & HVOT total
0.548**
< 0.001
HVOT, Hooper Visual Organization Test; MOCA, Montreal Cognitive Assessment.
, Correlation is significant at the 0.05 level (2-tailed);
, Correlation is significant at the 0.01 level (2-tailed).
Preliminary norms for the HVOT are presented in Table 6, stratified by age, level of education and gender. It is important to note that the age group 19–40 years, with an education of 2–11 years for both men and women, were excluded because of small sample sizes. The groups which showed the highest performance were men and women between the ages of 19–40 years with an education of 12–22 years. Lowest performance was seen in male participants in the 41–70 years-of-age category with an education of 2–11 years.
Hooper Visual Organization Test normative performance of a South African sample stratified by age, education and gender (percentiles).
Age range (years)
Education (years)
Gender
N
Mean
SD
Median
Min
Max
5
10
25
50
75
90
95
19–40
12–22
Male
11
20.545
4.6393
20
13.5
28.5
13.5
14.1
17
20
24
28.2
-
41–70
2–11
Male
17
12.412
3.4425
12.5
5
17.5
5
6.6
10.5
12.5
14.75
17.5
-
41–70
12–22
Male
18
17.194
5.2332
15.75
9.5
29
9.5
9.5
14.375
15.75
22.125
24.05
-
19–40
12–22
Female
15
20.533
5.1100
18
11.5
28
11.5
13.3
17.5
18
26
27.4
-
41–70
2–11
Female
21
17.667
4.3970
18
9
25.5
9.2
11.2
14.5
18
21
22.9
25.25
41–70
12–22
Female
27
20.148
5.4664
19
10
29
10.6
13.5
16.5
19
25.5
28.2
29
Note: Sample of South Africans who had primary and secondary public education and do not speak English as a first language.
Discussion
The primary focus of the present study was to develop stratified HVOT norms for a South African sample of participants that do not speak English as a first language and who have attended public primary and secondary schools. By selecting this specific sample, our study mitigated the effects of multi or bilingualism and quality of education as sources of biases in cognitive tests (Watts & Shuttleworth-Edwards, 2016). For this, the effects of sociodemographic variables (age, number of years of formal education and gender) on the HVOT total score were investigated. McDonalds Omega was used to determine the internal consistency reliability of the HVOT. Item response theory was used to further examine item functioning on the HVOT. This study also provided evidence of diagnostic and convergent validity, which is necessary for the evaluation of the clinical utility of this test in South Africa.
In this study, the highest means were obtained by the youngest and most educated groups (female mean = 20,5/SD = 4,6 and male mean = 20,5/SD = 5.1), but these were lower than the cut-off point of 21 suggested by Hooper (1958) and by a demographically similar group from Greece (mean = 25.43/SD = 2.17) (Giannakou & Kosmidis, 2007) and from Venezuela (50th Percentile = 25) (Campagna & Ferreira-Correia, 2021).
Our results indicate that people with more education performed better than those with less. The effects of age on the total score of the HVOT are illustrated by the common use of this variable in the norms stratification (DeVries, 2005; Hooper, 1983; Tamkin & Jacobsen, 1984). Years of education are included less frequently in the HVOT norms, despite the impact of this variable in cognitive performance, and more specifically in the visuospatial function (Roldán-Tapia, Cánovas, León & García-Garcia, 2017). Two exceptions are the HVOT norms for the Venezuelan and Greek populations (Campagna & Ferreira-Correia, 2021; Giannakou & Kosmidis, 2006).
In our study, women also performed better than men. Although a gender bias in the visuospatial functions has been suggested (Hatta et al., 2015; Parsons et al., 2004), our study challenges this notion, as other studies suggest that men outperform women (Campagna & Ferreira-Correia, 2021; Hatta et al., 2015), whereas other reports did not find any significant relationship between gender and HVOT scores (Giannakou & Kosmidis, 2006). Future studies in South Africa should explore the potential contribution of gender towards the total score of the HVOT whilst controlling for age and years of education.
Whilst the group differences described above are noteworthy, results from the DIF analysis should be noted. The items flagged for DIF across age, gender and education may have contributed somewhat, and to varying degrees, to these observed mean score differences. However, the influence of these items is likely to be minor, as relatively few items were affected in each case. These results should be investigated and confirmed in future research with larger samples to determine if the DIF results observed in this study are indeed robust.
The results from the current investigation suggest that the HVOT has good reliability with satisfactory estimates observed for both McDonalds Omega total and Rasch person reliability. These were consistent with reliability estimates reported in other work (Campagna & Ferreira-Correia, 2021). Item response theory analysis supported the construct validity of the HVOT with all items fitting the Rasch model. When inspecting the item location parameters, there was a clear progression in item difficulty with a few unexpected results, with some items being easier than anticipated whilst others were more difficult. When comparing the item frequencies reported in Table 4 to that of DeVries (2005), none of the items in this sample obtained 100% correct responses. Item one (fish), however, was considered to be the easiest, with only 1% of the participants giving incorrect responses. Surprisingly, item 11 was amongst the easiest items, whilst item 25 (block) was one of the most difficult. Hence, when clinicians administer the HVOT, the order of administration should follow empirical data that is context specific (Campagna & Ferreira-Correia, 2021). When administration takes place by presenting the items in the order of difficulty, there is a probability that one takes into account the application of a discontinuation rule (Campagna & Ferreira-Correia, 2021) which yields good discriminatory power (Wetzel & Murphy, 1991). It also decreases the time taken to administer the test as well as levels of fatigue for the patient (Campagna & Ferreira-Correia, 2021). However, given the findings of the current study, no discontinuation rule should be applied when administering the HVOT in the South African context.
In this study, we presented the HVOT norms for the South African population stratified by age, gender and years of education. All these variables had a significant correlation to the total HVOT score. To our knowledge, no other norms for the HVOT are available in this country. The mean HVOT scores of adults in Georgia, which were stratified by age (Tamkin & Jacobsen, 1984), revealed similar mean scores which were obtained for the specific South African population, as was defined previously. Also, a more recent normative study conducted on the Venezuelan population demonstrated a significant association between age, gender and level of education (Campagna & Ferreira-Correia, 2021), much like the current study. It is, however, important to consider that the generalisability of the current norms may be questionable. One of the reasons is that the use of quota sampling represented a limitation because some of the resulting subgroups were too small to be representative of a particular set of demographics (e.g. the age group of 19–40 years who had an education level of 2–11 years for both men and women).
A potential confounding association between naming abilities and performance on the HVOT has been reported (Greve et al., 2000), but it has been challenged (Paolo, Cluff & Ryan, 1996). In our investigation, a strong correlation was evident between the HVOT and the MOCA language total (r = 0.564, p = 0.000) and the MOCA naming total (r = 0.354, p = 0.000). This study therefore supports the claim that naming ability may have an impact on HVOT performance. Additional research should consider incorporating other psychometric measures apart from the MOCA in order to further validate the HVOT’s association with naming ability, and South African clinicians should consider assessing naming ability in the language of assessment when using the HVOT.
Furthermore, the significant correlation between the total MOCA and the majority of the sub-component scores and the HVOT total score would support the argument that the HVOT can act as a screening test. Studies have shown that visuo-perceptive tests like the HVOT can be multifactorial (Campagna & Ferreira-Correia, 2021; DeVries, 2005) as it indirectly recruits several cognitive functions beyond the core one. This may support the value of the HVOT as a screening tool, although keeping in mind the limitations of these tasks as diagnostic tools (Roebuck-Spencer et al., 2017).
Furthermore, the fact that answers can be accepted in different languages allows for assessment of linguistically diverse patients. However, this needs to be further explored in studies that better control for this. Although this was accepted, answering in the participants’ home language was not overtly encouraged. Therefore, the impact of these linguistic variables (naming capacity, English proficiency, and choosing to provide answers in different languages) on the the psychometric properties of the HVOT remains to be investigated.
In terms of the HVOT’s ability to discriminate between the normal control and clinical group, it was apparent that the clinical group performed significantly worse than the healthy control group. This supports literature which states that patients with HD/HDL2 often present with visuo-constructive deficits (Gómez-Tortosa, Del Barrio, Barroso & García Ruiz, 1996), and that the HVOT is known to be able to discriminate these cognitive dysfunctions in patients (Azambuja et al., 2012). Given that this study only included a small sample of HD/HDL2 clinical population, the generalisability on the results to HD/HDL2 and other pathologies is limited.
Both the clinical and control groups comprised of small sample sizes. However, literature suggests that it is better to make use of well-matched, small homogenous groups (n > 5) rather than large, heterogenous groups (Crawford & Garthwaite, 2012). A well-defined and homogenous sample was selected for this project and is representative of a large proportion of the South African population. Therefore, this study may have significant value for clinicians using the HVOT in this context, despite the small sample size.
Implications and recommendations
South African clinicians working with patients with demographic characteristics similar to our sample are encouraged to use the adapted version of the HVOT and the stratified norms provided in order to reduce the biases caused by the use of non-representative norms. Future studies should expand the current control sample to include participants with different demographic characteristics (e.g. English first language speakers and younger and older adults) and better explore the construct validity of this test.
Conclusion
This study represents an important contribution to the literature on psychological assessment in South Africa, as it demonstrates the psychometric properties and potential of the HVOT and provides preliminary stratified normative data for South African polyglot adults who do not speak English as a first language and attended public schools. The test yielded good reliability, convergence and discriminatory validity, although the item difficulty values did not follow the expected monotonic increase. The total HVOT correlated significantly with age, years of education and gender.
Acknowledgements
We are grateful to all the participants who volunteered their time to be part of this study. The article is based on the first author’s thesis, submitted to the University of the Witwatersrand for the Degree of Master of Arts in Social and Psychological Research. The data reported in this article were collected for the PhD thesis of the second author (Ferreira-Correia, 2019).
Competing interests
The authors declare that they have no conflict of interests, financial or personal, that may have inappropriately influenced them in writing this article.
Authors’ contributions
S.M.-K. was responsible for the study concept and design, quality control of the data, data analysis and interpretation, writing of manuscript. A.F.-C. was responsible for the study concept and design, organisation, acquisition and capturing of data, supervision of the study, and critical revision of the manuscript for important intellectual content. C.J.J.v.Z. was responsible for the psychometric data analysis, critical revision of the manuscript for important intellectual content.
Funding information
The assessment of the HDL2 cases was partially financed by the Medical Research Council’s Self-Initiated Research Grant, South Africa entitled ‘The clinical and genetic profile of Huntington disease-like 2 (HDL2) in South Africa’.
Data availability
The anonymised data set that supports the findings of this study are available on request from the corresponding author.
Disclaimer
The views expressed in the present article are of the authors alone and not an official position of the institutions they are affiliated with.
ReferencesArdila, A., Ostrosky-Solis, F., Rosselli, M. & Gomez, C. (2000). Age-related cognitive decline during normal aging: The complex effect of education. , 15(6), 495–513. https://doi.org/10.1093/arclin/15.6.495Azambuja, M.J., Radanovic, M., Haddad, M.S., Adda, C.C., Barbosa, E.R. & Mansur, L.L. (2012). Language impairment in Huntington’s disease. , 70(6), 410–415. https://doi.org/10.1590/S0004-282X2012000600006Booth, R.D.L. & Happé, F.G.E. (2018). Evidence of reduced global processing in autism spectrum disorder. , 48(4), 1397–1408. https://doi.org/10.1007/s10803-016-2724-6Bond, T.G. & Fox, C.M. (2015). (3rd ed.). Mahwah, NJ: L. Erlbaum.Boyd, J.L. (1981). A validity study of the Hooper Visual Organization Test. , 49(1), 15–19. https://doi.org/10.1037/0022-006X.49.1.15Campagna, L. & Ferreira-Correia, A. (2021). . Retrieved from http://mc.manuscriptcentral.com/hapnCrawford, J.R. & Garthwaite, P.H. (2012). Single-case research in neuropsychology: A comparison of five forms of t-test for comparing a case to controls. , 48(8), 1009–1016. https://doi.org/10.1016/j.cortex.2011.06.021DeVries, M.R. (2005). . Western Michigan University. Retrieved from https://scholarworks.wmich.edu/dissertations/1027Eberson, S.C. (2000). . Masters Thesis, California University. California, USA: California State University San Marcos. https://scholarworks.calstate.edu/concern/theses/tm70mv67fElias, M.F., Dore, G.A., Goodell, A.L., Davey, A., Zilioli, M.K.C., Brennan, S. & Robbins, M.A. (2011). Normative data for elderly adults: The Maine-Syracuse study. , 37(2), 142–178. https://doi.org/10.1080/0361073X.2011.554511Ferreira-Correia, A. (2019). . PhD thesis. Johannesburg: University of the Witwatersrand.Ferreira Correia, A., Anderson, D.G., Cockcroft, K. & Krause, A. (2020). The neuropsychological deficits and dissociations in Huntington Disease-Like 2: A series of case control studies. 136, 107238. https://doi.org/10.1016/j.neuropsychologia.2019.107238Field, A. (2018). (5th ed.). UK, London: SAGE.Foxcroft, C., Paterson, H., Le Roux, N. & Herbst, D. (2004). . Pretoria: Human Science Research Council.Gasparini, M., Hufty, A.M., Masciarelli, G., Ottaviani, D., Angeloni, U., Lenzi, G.L. & Bruno, G. (2008). Contribution of right hemisphere to visual imagery: A visual working memory impairment? , 14(5), 902–911. https://doi.org/10.1017/S1355617708080995Giannakou, M. & Kosmidis, M.H. (2006). Cultural appropriateness of the Hooper Visual Organization Test? Greek normative data. , 28(6), 1023–1029. https://doi.org/10.1080/13803390591004374Gómez-Tortosa, E., Del Barrio, A., Barroso, T. & García Ruiz, P.J. (1996). Visual processing disorders in patients with Huntington’s disease and asymptomatic carriers. , 243(3), 286–292. https://doi.org/10.1007/BF00868528Greve, K.W., Lindberg, R.F., Bianchini, K.J. & Adams, D. (2000). Construct validity and predictive value of the Hooper Visual Organization Test in stroke rehabilitation. , 7(4), 215–222.Harris, P.A., Taylor, R., Thielke, R., Payne, J., Gonzalez, N. & Conde, J.G. (2009). Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. , 42(2), 377–381. https://doi.org/10.1016/j.jbi.2008.08.010Hatta, T., Iwahara, A., Hatta, T., Ito, E., Hatta, J., Hotta, C., … Hamajima, N. (2015). Developmental trajectories of verbal and visuospatial abilities in healthy older adults: Comparison of the hemisphere asymmetry reduction in older adults model and the right hemi-ageing model. , 20(1), 69–81. https://doi.org/10.1080/1357650X.2014.917656Hooper, H.E. (1958). . California, USA: Western Psychological Services.Leedy, P.D. & Ormrod, J.E. (2015). (11th ed.). New York, USA: Pearson.Lezak, M.D., Howieson, D.B., Bigler, E.D. & Tranel, D. (2012). . (5th ed.). USA, New York: Oxford University Press.Linacre, J.M. (2015). . Beaverton, OR: Winsteps.com.Lin, Y.-H., Su, C.-Y., Guo, W.-Y. & Wuang, Y.-P. (2012). Psychometric validation and normative data of a second Chinese version of the Hooper Visual Organization Test in children. , 33(6), 1919–1927. https://doi.org/10.1016/j.ridd.2012.05.016Lopez, M.N., Lazar, M.D. & Oh, S. (2003). Psychometric properties of the Hooper Visual Organization Test. , 10(1), 66–70. https://doi.org/10.1177/1073191102250183Lucas, M.D. (2013). Neuropsychological assessment in South Africa. In S.Laher & K.Cockcroft (Eds.), (pp. 186–200). Johannesburg: Wits University Press.Merten, T. & Beal, C. (1999). An analysis of the Hooper Visual Organization Test with neurological patients. , 13(4), 521–529. https://doi.org/10.1076/1385-4046(199911)13:04;1-Y;FT521Miller, I.N., Himali, J.J., Beiser, A.S., Murabito, J.M., Seshadri, S., Wolf, P.A. & Au, R. (2015). Normative data for the cognitively intact oldest-old: The Framingham Heart study. , 41(4), 386–409. https://doi.org/10.1080/0361073X.2015.1053755Mitolo, M., Hamilton, J.M., Landy, K.M., Hansen, L.A., Galasko, D., Pazzaglia, F. & Salmon, D.P. (2016). Visual perceptual organization ability in autopsy-verified dementia with Lewy bodies and Alzheimer’s disease. , 22(6), 609–619. https://doi.org/10.1017/S1355617716000436Nasreddine, Z.S., Phillips, N.A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., Cummings, J.L. & Chertkow, H. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. , 53(4), 695–699. https://doi.org/10.1111/j.1532-5415.2005.53221.xParsons, T.D., Larson, P., Kratz, K., Thiebaux, M., Bluestein, B., Buckwalter, J.G. & Rizzo, A.A. (2004). Sex differences in mental rotation and spatial rotation in a virtual environment. , 42(4), 555–562.Paolo, A.M., Cluff, R.B. & Ryan, J.J. (1996). Influence of perceptual organization and naming on the Hooper Visual Organization Test. , 9(4), 254–257.Paxton, J.L., Peavy, G.M., Jenkins, C., Rice, V.A., Heindel, W.C. & Salmon, D.P. (2007). Deterioration of visual-perceptual organization ability in Alzheimer’s sisease. , 43(7), 967–975. https://doi.org/10.1016/S0010-9452(08)70694-4Resnik, D.B. & Shamoo, A.E. (2011). The Singapore Statement on Research Integrity. , 18(2), 71–75. doi: 10.1080/08989621.2011.557296Revelle, W. (2021). . Evanston, IL: Northwestern University. R package version 2.1.3. Retrieved from https://CRAN.R-project.org/package=psychR Core Team (2020). . Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/Roebuck-Spencer, T.M., Glen, T., Puente, A.E., Denney, R.L., Ruff, R.M., Hostetter, G. & Bianchini, K.J. (2017). Cognitive screening tests versus comprehensive neuropsychological test batteries: a national academy of neuropsychology education paper. , 32(4), 491–498. https://doi.org/10.1093/arclin/acx021Roldán-Tapia, M.D., Cánovas, R., León, I. & García-Garcia, J. (2017). Cognitive vulnerability in aging may be modulated by education and reserve in healthy people. , 9, 340. https://doi.org/10.3389/fnagi.2017.00340Sanz Cortés, A., Olivares Crespo, M.E. & Barcia Albacar, J.A. (2011). Aspectos Neuropsicológicos en Pacientes Diagnosticados de Tumores Cerebrales. , 22(2), 139–155. https://doi.org/10.5093/cl2011v22n2a4Su, C.-Y., Lin, Y.-H., Wu, Y.-Y. & Wuang, Y.-P. (2013). Development of the Chinese version of the Hooper Visual Organization Test: Normative data. , 36(1), 56–67. https://doi.org/10.1097/MRR.0b013e3283588b95Tamkin, A.S. & Jacobsen, R. (1984). Age-related norms for the Hooper Visual Organization Test. , 40(6), 1459–1463. https://doi.org/10.1002/1097-4679(198411)40:6<1459::AID-JCLP2270400633>3.0.CO;2-3Watts, A.D. & Shuttleworth-Edwards, A.B. (2016). Neuropsychology in South Africa: Confronting the challenges of specialist practice in a culturally diverse developing country. , 30(8), 1305–1324.Wetzel, L. & Murphy, S.G. (1991). Validity of the use of a discontinue rule and evaluation of discriminability of Hooper visual organization test. , 5(2), 119–122. https://doi.org/10.1037/0894-4105.5.2.119
How to cite this article: Mahomed-Kola, S., Ferreira-Correia, A. & Van Zyl, C.J.J. (2022). Preliminary normative data for the Hooper Visual Organization Test for a South African sample. African Journal of Psychological Assessment, 4(0), a64. https://doi.org/10.4102/ajopa.v4i0.64