Neuropsychological testing is widely used for specialised placements within the military. Within the South African National Defence Force (SANDF), there is concern about the representation of the normative information currently available for these tests. The letter cancellation test, a paper-and-pencil-based test used as a quick measure of attention, is subject to unstandardised administration and scoring procedures as well as broad cut-off scores. The aim of this study was to develop detailed administration and scoring procedures for the single and double letter cancellation test and to provide preliminary normative data on these versions of the test in the SANDF. A non-probability sampling strategy resulted in a sample of 292 participants. Normative data are provided for the total sample and classified into three performance categories: omissions, errors and time. Between-group comparisons indicated gender and age-related differences (but no differences for rank) in terms of time, and normative data are therefore also provided for related subgroups.
Neuropsychological assessment is integral to clinical work (Lucas,
Military personnel take up many different jobs, such as pilots, weapon handling, medical staff and deployment. It is therefore essential that there are tests available that can help evaluate attention and concentration in order to ensure that the individuals are competent enough to carry out their specific duties. Currently, related tests are used in the military for specialised career placements. These tests are also used as part of the soldier’s rehabilitation processes. Even slight impairments in attention and concentration which can be a result of traumatic brain injury can have substantial repercussions for a soldier’s effectiveness while on duty or in combat during the recovery period (Hatta, Yoshizaki, Ito, Mase, & Kabasawa,
[
In a classic quote by psychologist William James, attention is defined as processing ‘one out of what seem several simultaneously possible objects or trains of thought … It implies withdrawal from some things in order to deal effectively with others’ (James,
Stankov (
As attention consists of a variety of processes, a comprehensive test of attention would consequently measure a range of these processes. Based on a review of existing literature, Coetzer and Balchin (
Cancellation tests are usually paper-and-pencil tests where an individual needs to identify and cancel target items (Azouvi et al.,
The use of cancellation tests has been extensively documented in neuropsychological literature as measures of:
visual selectivity and sustained attention (Lezak et al.,
processing speed, perceptual speed and visuomotor ability (McCrea & Robinson,
‘visual selectivity at fast speed with a repetitive motor response’ (Lezak et al.,
A range of cancellation tests are discussed in literature including line bisection tests, symbol cancellation tests and letter cancellation tests. Each test differs in terms of the stimuli used and the method of administration and scoring. Letter cancellation tests, the focus of this study, make use of a certain letter(s) as
Studies on standardisations of the letter cancellation test have been conducted in different contexts. Amongst others, these include the original development for the 1946 birth cohort study in a British context (Richards, Kuhn, Hardy, & Wadsworth,
In addition, no South African standardisations were found. The letter cancellation test has been used in research studies in South Africa (e.g. Jossub, Cassimjee, & Cramer,
In particular, no normative data are available which provide for the context-specific demographics and skills profile of the South African military environment. The letter cancellation test is a paper-and-pencil test that may prove beneficial as a quick measure for the attention (Pradhan & Nagendra,
constructing standardised administration and scoring procedures for testing
investigating the influence of demographic variables with a view to establishing subgroup normative data for military personnel
establishing preliminary normative data for a sample of military personnel.
The target population comprised military personnel in the SANDF. Non-random (voluntary) sampling was used, resulting in an initial selection of 300 participants. The sample comprised people who were multilingual. Demographic variables of interest were age (the majority of the military personnel are 18–49 years old), gender (approximately 30% of the population are female and 70% male) and rank (15% are officers and 85% non-commissioned officers) (Defence Web,
Sample frequencies: Age, gender and rank (
Demographic variables | Categories | Frequency | |
---|---|---|---|
Age (years) | 20–29 | 100 | 34.2 |
30–39 | 101 | 34.6 | |
40–49 | 72 | 24.7 | |
50–59 | 18 | 6.2 | |
60+ | 1 | 0.3 | |
Gender | Female | 92 | 32.5 |
Male | 198 | 67.8 | |
Missing | 2 | - | |
Rank | Officers | 53 | 18.2 |
Non-commissioned officers | 238 | 81.5 |
The aim of this study was to develop standardised administration and scoring procedures for the letter cancellation test before establishing normative data on the test. Two trials of the letter cancellation test were constructed for the data collection of this study, namely the single (H) letter cancellation test and the double (CE) letter cancellation test. This was done to establish normative data for simple and double mental tracking. Currently, the existing H letter cancellation test used to assess single mental tracking, which is presented in the work of Lezak et al. (
In order to ensure uniform administration of the letter cancellation test, the instructions were documented in text and the test administrators were required to read it out verbatim so that the testing instructions remained consistent. Clear and detailed instructions were provided on how to complete the test:
Firstly, participants were instructed to scan the test from left to right, and then to go down one row at a time following the same scanning process, and to cancel targets by striking out the specified letter using a pencil.
The second instruction was that their performance on the test will be timed and they were required to work as quickly as they could. They were also informed that there was no specific time limit imposed on how long they should take to complete the test.
Lastly, participants were informed that they would be completing two trials of the test.
Additionally, a scoring profile was created so that the scoring remained consistent, thus enhancing the integrity of the study. This document was constructed to record participants’ time and performance in each part of the test. Test administrators were instructed to record the time taken to complete the task (in seconds), the number of errors made (i.e. non-target items erroneously identified), the number of omitted letters (i.e. target items not identified) and any self-correcting attempts for each part in order to establish what is significant and what is not. A second scoring sheet was included for recording qualitative notable observations made during testing.
This study, therefore, provides test scores for each of the six parts of the H and CE letter cancellation tests in terms of time, error and a total score, a significant improvement on earlier scoring procedures. The proposed detailed scoring aims to provide clinicians with more comprehensive information on the letter cancellation test, and to further aid assessment and diagnostic practices.
All SANDF members have their health status examined annually. Appointments are made on a random basis implying that at any given period, representation in terms of the specified stratification variables (age, gender, rank) could be expected amongst those being assessed. Participants were recruited on a voluntary basis during an arbitrary selected period of assessments. They were primarily from the Gauteng assessment centre with some participants selected from the Western Cape centre. (Note the former centre often also caters for members from other provinces.) All possible efforts were made to ensure that the testing environment was comfortable and reasonably quiet. A screening questionnaire was completed by all participants. Socio-demographic information was obtained and participants had to answer questions regarding their suitability for the study. Psychologists (clinical and counselling) and registered counsellors employed in the SANDF administered the test on an individual basis. The administrators attended a training session and also met with the researcher before each session to prepare for the testing. The tests were administered in English. This is the main medium of communication in the SANDF, and as such, proficiency in the language is a requirement and could be assumed in this study.
Ethical clearance was obtained from the University of South Africa (UNISA) Ethics Committee, reference number: SG (D Psych)/R/104/10/5, for a study involving human participants. In the case of the SANDF, the chain of command implied clearance by various structures, departments and units; (Defence Intelligence), reference number: DI/ DDS/R/202/3/7 and (Military Health Service), reference number: AMHF/R/104/10/05. In the case of the latter, the chain of command implied clearance by various structures, departments and units. Permission was also granted for collecting and using the data for a master’s dissertation and for publishing the results in a journal. Informed consent was obtained from all participants, and confidentiality was maintained by securing the data (a locked cupboard and password protection) and ensuring that no personal information was published. Arrangements were made for appropriate referral should the test results indicate the need for further intervention in individual cases.
Descriptive statistics (i.e. means and standard deviations) were calculated for the two trials of the test (H and CE letter cancellation test), for each of the six parts, and for each score, that is, time, omissions and errors made. Comparative analyses were conducted to determine if selected demographic variables had a significant impact on test performance (and thus warranted separate tables for comparison). Analyses were only performed in cases where the cell size was at least
The means and standard deviations for each of the six parts of the H letter cancellation test and the CE letter cancellation test are provided in
Descriptive statistics for the H and the CE letter cancellation tests (
Scoring categories | H letter cancellation test | CE letter cancellation test | ||||||
---|---|---|---|---|---|---|---|---|
Mean | SD | Min | Max | Mean | SD | Min | Max | |
Omissions 1 | 0.48 | 0.93 | 0 | 6 | 2.79 | 2.79 | 0 | 14 |
Omissions 2 | 0.35 | 0.92 | 0 | 9 | 1.78 | 2.58 | 0 | 20 |
Omissions 3 | 0.42 | 1.10 | 0 | 9 | 2.26 | 2.67 | 0 | 16 |
Omissions 4 | 0.49 | 1.23 | 0 | 11 | 1.80 | 2.15 | 0 | 13 |
Omissions 5 | 0.50 | 1.30 | 0 | 13 | 2.39 | 2.63 | 0 | 17 |
Omissions 6 | 0.36 | 0.84 | 0 | 6 | 1.74 | 2.11 | 0 | 20 |
Omissions total | 2.59 | 4.40 | 0 | 30 | 12.73 | 11.47 | 0 | 73 |
Errors 1 | 0.00 | 0.59 | 0 | 1 | 0.06 | 0.85 | 0 | 4 |
Errors 2 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.00 | 0 | 0 |
Errors 3 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.06 | 0 | 1 |
Errors 4 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.00 | 0 | 0 |
Errors 5 | 0.01 | 0.08 | 0 | 1 | 0.00 | 0.00 | 0 | 0 |
Errors 6 | 0.00 | 0.00 | 0 | 0 | 0.00 | 0.00 | 0 | 0 |
Errors total | 0.08 | 0.33 | 0 | 3 | 0.13 | 0.42 | 0 | 4 |
Time 1 | 31.99 | 7.86 | 18 | 59 | 51.21 | 11.76 | 25 | 104 |
Time 2 | 33.12 | 8.79 | 16 | 90 | 51.96 | 12.10 | 21 | 99 |
Time 3 | 32.14 | 8.21 | 11 | 75 | 53.12 | 12.28 | 28 | 107 |
Time 4 | 32.45 | 7.88 | 12 | 67 | 52.54 | 12.87 | 24 | 120 |
Time 5 | 32.19 | 8.27 | 10 | 77 | 53.65 | 12.80 | 27 | 110 |
Time 6 | 33.54 | 8.78 | 14 | 79 | 53.47 | 13.65 | 19 | 122 |
Time total | 196.57 | 48.31 | 95 | 484 | 316.03 | 73.34 | 59 | 628 |
SD, standard deviation; H, single letter cancellation test; CE, double letter cancellation test.
Only in the case of time taken to complete the tests did the distributions resemble normality (see Pillay,
Independent samples
Independent samples
H time | Gender | Mean | SD | Sig. (2-tailed) | |
---|---|---|---|---|---|
H time 1 | M | 33.41 | 7.76 | 0.000 | |
F | 29.11 | 7.23 | |||
H time 2 | M | 34.13 | 7.97 | 0.005 | |
F | 31.07 | 10.07 | |||
H time 3 | M | 33.16 | 8.01 | 0.003 | |
F | 30.09 | 8.31 | |||
H time 4 | M | 33.63 | 7.79 | 0.000 | |
F | 30.02 | 7.57 | |||
H time 5 | M | 33.39 | 8.28 | 0.000 | |
F | 29.77 | 7.72 | |||
H time 6 | M | 34.88 | 9.18 | 0.000 | |
F | 30.78 | 7.17 | |||
H time total | M | 204.30 | 49.93 | 0.000 | |
F | 180.83 | 40.42 |
Note: Time in seconds.
M, male; F, female; SD, standard deviation; H, single letter cancellation test.
Independent samples
CE time | Gender | Mean | SD | Sig. (two-tailed) | |
---|---|---|---|---|---|
CE time 1 | M | 53.74 | 11.50 | 0.000 | |
F | 45.85 | 10.44 | |||
CE time 2 | M | 53.66 | 12.02 | 0.000 | |
F | 48.41 | 11.56 | |||
CE time 3 | M | 55.23 | 12.15 | 0.000 | |
F | 48.70 | 11.41 | |||
CE time 4 | M | 54.48 | 13.21 | 0.000 | |
F | 48.45 | 11.04 | |||
CE time 5 | M | 55.49 | 12.80 | 0.000 | |
F | 49.88 | 12.01 | |||
CE time 6 | M | 55.25 | 13.97 | 0.001 | |
F | 49.78 | 12.22 | |||
CE time total | M | 327.92 | 70.20 | 0.000 | |
F | 291.21 | 74.04 |
Note: Time in seconds.
M, male; F, female; SD, standard deviation; CE, double letter cancellation test.
ANOVA: Comparison of time scores for age groups on the H letter cancellation test.
H time | Sum of squares | Mean square | Sig. | ||
---|---|---|---|---|---|
7.646 | 0.000 | ||||
Between groups | 50102.135 | 3 | 16700.712 | ||
Within groups | 629067.629 | 288 | 2184.263 | ||
Total | 679169.764 | 291 | - | ||
6.567 | 0.000 | ||||
Between groups | 1151.209 | 3 | 383.736 | ||
Within groups | 16827.760 | 288 | 58.430 | ||
Total | 17978.969 | 291 | - | ||
4.173 | 0.006 | ||||
Between groups | 935.895 | 3 | 311.965 | ||
Within groups | 21532.146 | 288 | 74.764 | ||
Total | 22468.041 | 291 | - | ||
10.417 | 0.000 | ||||
Between groups | 1922.131 | 3 | 640.710 | ||
Within groups | 17713.828 | 288 | 61.506 | ||
Total | 19635.959 | 291 | - | ||
4.971 | 0.002 | ||||
Between groups | 888.942 | 3 | 296.314 | ||
Within groups | 17167.181 | 288 | 59.608 | ||
Total | 18056.123 | 291 | - | ||
6.913 | 0.000 | ||||
Between groups | 1337.579 | 3 | 445.860 | ||
Within groups | 18575.682 | 288 | 64.499 | ||
Total | 19913.260 | 291 | - | ||
4.043 | 0.008 | ||||
Between groups | 905.640 | 3 | 301.880 | ||
Within groups | 21504.946 | 288 | 74.670 | ||
Total | 22410.586 | 291 | - |
Note: Time in seconds.
H, single letter cancellation test.
ANOVA: Comparison of time scores for age groups on the CE letter cancellation test.
CE time | Sum of squares | Mean square | Sig. | ||
---|---|---|---|---|---|
6.422 | 0.000 | ||||
Between groups | 98131.239 | 3 | 32710.413 | ||
Within groups | 1467004.419 | 288 | 5093.765 | ||
Total | 1565135.658 | 291 | - | ||
7.820 | 0.000 | ||||
Between groups | 3028.551 | 3 | 1009.517 | ||
Within groups | 37180.285 | 288 | 129.098 | ||
Total | 40208.836 | 291 | - | ||
4.227 | 0.006 | ||||
Between groups | 1797.693 | 3 | 599.231 | ||
Within groups | 40831.814 | 288 | 141.777 | ||
Total | 42629.507 | 291 | - | ||
4.418 | 0.005 | ||||
Between groups | 1930.358 | 3 | 643.453 | ||
Within groups | 41945.684 | 288 | 145.645 | ||
Total | 43876.041 | 291 | - | ||
- | - | ||||
Between groups | - | 3 | - | ||
Within groups | - | 288 | - | ||
Total | - | 291 | - | ||
5.115 | 0.002 | ||||
Between groups | 2410.488 | 3 | 803.496 | ||
Within groups | 45242.180 | 288 | 157.091 | ||
Total | 47652.668 | 291 | - | ||
3.910 | 0.009 | ||||
Between groups | 2120.279 | 3 | 706.760 | ||
Within groups | 52060.379 | 288 | 180.765 | ||
Total | 54180.658 | 291 | - |
Note: Time in seconds.
CE, double letter cancellation test.
Descriptive statistics for total time for gender by age.
Age | Female | Male | ||||
---|---|---|---|---|---|---|
Mean | SD | Mean | SD | |||
20–29 | 47 | 179.45 | 46.08 | 54 | 194.24 | 53.56 |
30–39 | 35 | 177.17 | 30.84 | 67 | 194.09 | 41.27 |
40–49 | 15 | 190.80 | 38.32 | 55 | 220.75 | 51.62 |
50–59 | 1 | 153.00 | . | 17 | 224.82 | 45.36 |
60+ | - | - | - | 1 | 154.00 | . |
20–29 | 47 | 297.83 | 89.43 | 54 | 315.33 | 64.86 |
30–39 | 35 | 278.49 | 41.84 | 67 | 313.81 | 68.18 |
40–49 | 15 | 313.53 | 82.56 | 55 | 347.95 | 70.12 |
50–59 | 1 | 269.00 | . | 17 | 353.59 | 60.04 |
60+ | - | - | - | 1 | 247.00 | . |
SD, standard deviation; H, single letter cancellation test; CE, double letter cancellation test.
At present, comparative data for the CE letter cancellation test are limited to an overall score (for two parts) and the statement that ‘normal performance limits have been defined as 0–2 omissions in 120 seconds’ (Diller, Ben-Yishay, & Gerstman,
Although Diller et al. (
Significant differences were also found between those below and above 40 years of age with the latter taking more time to complete the tasks. Age-related decline in speed for the letter cancellation test has been reported previously (Pradhan & Nagendra,
Skewness could be attributed to the target population being a pre-selected group. According to Kennedy and Moore (eds.
A major contribution of this study is the development of standardised administration and scoring procedures for a test of attention. Additionally, the military context implies a need for appropriate normative data on this construct. However, larger sample sizes are required for adequate representation in terms of some of the demographic variables (i.e. individuals older than 50 years, females older than 40 years and left-handed individuals). This will also enable further exploration of the distribution of the performance. Standard scores based on the present data set cannot be interpreted in terms of the properties of a normal distribution. These recommendations would allow for a comprehensive standardisation and evaluation of the psychometric properties of the test in the military context. The study furthermore involved a highly specific subgroup of the general population and replication studies including additional subpopulations should be considered.
The letter cancellation test is widely used despite being subject to unstandardised administration and scoring procedures and broad cut-off scores. This study provides a review of the letter cancellation test and puts forward improved administration procedures, detailed scoring methods and relevant normative data for adequate sample sizes. This was done to provide clinicians in the SANDF with meaningful scores for interpretation and to guide future developments in the wider South African context.
The authors would like to thank the SANDF for allowing them to conduct the research and supporting the publication of the results.
The authors declare that no competing interests exist and this is an original research.
This empirical work was conducted by C.P.H. with assistance from C.G. and guidance from B.S. and R.v.E. The authors co-wrote the manuscript.
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Data sharing is not applicable because of the sensitive nature thereof.
The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.