Neuropsychological assessment is integral to clinical work (Lucas, 2013) and also forms part of test batteries used in organisations such as mining, manufacturing, construction and the military. Psychological testing within the military has become invaluable in the assessment and preparation of its personnel. Nwafor and Adesuwa (2014) described the use of psychological testing within the military context as a process that takes place on a continuum, starting from recruitment where an individual is assessed, to job utilisations for promotions and placements, to special missions and the diagnosis and treatment of disorders, and this continues until their retirement. Within the South African military context, its personnel perform a wide array of functions and occupational duties each of which has its own specific requirements and criteria. Attention was highlighted as a central neurocognitive skill that is necessary for highly specialised occupational duties as well as simple everyday functions in the military. Kennedy and Zillmer (eds. 2012) stated that soldiers are required to ‘maintain high levels of consistent attention and concentration in order to perform effectively and safely’ (p. 199). Even when preparing for the start of a day, the command of ‘attention’ is often given by the commander in order to make all soldiers focus on their duties for the day. It is therefore a standard practice to include a measure of attention as part of an assessment battery. One of the major concerns in the field of psychological testing for this context is, however, ensuring that the normative data are representative in terms of military personnel.

Military personnel take up many different jobs, such as pilots, weapon handling, medical staff and deployment. It is therefore essential that there are tests available that can help evaluate attention and concentration in order to ensure that the individuals are competent enough to carry out their specific duties. Currently, related tests are used in the military for specialised career placements. These tests are also used as part of the soldier’s rehabilitation processes. Even slight impairments in attention and concentration which can be a result of traumatic brain injury can have substantial repercussions for a soldier’s effectiveness while on duty or in combat during the recovery period (Hatta, Yoshizaki, Ito, Mase, & Kabasawa, 2012; Kennedy & Zillmer, 2012). Attentional disorders (e.g. attentional deficit and hyperactivity disorder, perseveration and distractibility, confusional state and visual neglect), if undetected or not treated, would also impact on effective functioning in this context. For example, risk factors associated with attentional deficit and hyperactivity disorder include:

In a classic quote by psychologist William James, attention is defined as processing ‘one out of what seem several simultaneously possible objects or trains of thought … It implies withdrawal from some things in order to deal effectively with others’ (James, 1890, pp. 403–404). A person’s capacity for paying attention to daily activities is crucial for the successful completion of everyday tasks (Lezak, Howieson, & Loring, 2004). In order for individuals to function effectively, they need the ability to focus on the task at hand while simultaneously ignoring other distracting factors. This ability requires them to filter, select, focus, shift and track information (Groth-Marnat, 2009).

Stankov (1988) identified six components of attention:

As attention consists of a variety of processes, a comprehensive test of attention would consequently measure a range of these processes. Based on a review of existing literature, Coetzer and Balchin (2014), Lezak, Howieson, Bigler and Tranel (2012), Mirsky, Anthony, Duncan, Ahearn and Kellam (1991) and Stump (2002) recommended letter cancellation tests as comprehensive measures of attention. Letter cancellation tests can also be considered as screening tests. Hatta et al. (2012) stated that cancellation tests are simple, yet effective measures of attention as they are cost-effective and applicable over a wide spectrum of age groups.

Cancellation tests are usually paper-and-pencil tests where an individual needs to identify and cancel target items (Azouvi et al., 2006). Most cancellation tests consist of target stimuli that are distributed amongst distractor stimuli. The target stimulus is the identified symbol or letter that the individual needs to identify and cancel, while the distractor stimuli aim to divert the individual’s attention from the target stimulus. Performance is scored by recording the number of omissions, errors and time taken to complete the test (Lezak et al., 2004).

The use of cancellation tests has been extensively documented in neuropsychological literature as measures of:

A range of cancellation tests are discussed in literature including line bisection tests, symbol cancellation tests and letter cancellation tests. Each test differs in terms of the stimuli used and the method of administration and scoring. Letter cancellation tests, the focus of this study, make use of a certain letter(s) as target stimuli that are distributed in columns and rows amongst other letters that serve as the distractor stimuli. Examinees are then required to cancel the target letter(s) distributed amongst the distractor letters. There are various types of letter cancellation tests, for instance, the single letter cancellation test, double letter cancellation test (Lezak et al., 2004) and six letter cancellation test (Pradhan, 2013). This study focussed on the single and double letter cancellation tests.

Studies on standardisations of the letter cancellation test have been conducted in different contexts. Amongst others, these include the original development for the 1946 birth cohort study in a British context (Richards, Kuhn, Hardy, & Wadsworth, 1999), the use of the letter cancellation test on American samples (Uttl & Pilkenton-Taylor, 2001; Warren, Moorre, & Vogtle, 2008) and normative data for Indian school going children (Pradhan & Nagendra, 2008). These studies all differ in terms of the administration and scoring instructions utilised. The differing administration and scoring instructions pose a challenge to the reliable use of the letter cancellation test as administration with differing instructions alters the quality of the responses by the participants, thus compromising comparability of test results (Groth-Marnat, 2009). Each of these studies also developed differing sets of normative data.

In addition, no South African standardisations were found. The letter cancellation test has been used in research studies in South Africa (e.g. Jossub, Cassimjee, & Cramer, 2017); however, to date, there have been no studies focussing on the suitability of the test for South African populations. Practitioners and academics in the South African context indicate that Lezak et al.’s (2012) international guideline that ‘normal performance limits have been defined as 0-2 omissions in 120 seconds’ (p. 381) is used. This, however, provides a vague description of the scores, was developed on an international platform and does not allow for consideration of the impact of South African socio-demographic variables on test performance. According to Nell (2000) and Shuttleworth-Edwards (2016) neuropsychological tests without relevant normative data place clinicians at risk of misdiagnosing their patients. Anderson (2001) argued that, ‘the injudicious use of imported normative data could result in an unacceptably high diagnostic rate of neuropsychological impairment in otherwise healthy South Africans’ (p. 33).

In particular, no normative data are available which provide for the context-specific demographics and skills profile of the South African military environment. The letter cancellation test is a paper-and-pencil test that may prove beneficial as a quick measure for the attention (Pradhan & Nagendra, 2008) of military personnel. Given that crucial decisions are made using test results, appropriate normative data are essential to ensure fairness. Therefore, this study set out to standardise the letter cancellation test for military personnel in the South African National Defence Force (SANDF), by:

At present, comparative data for the CE letter cancellation test are limited to an overall score (for two parts) and the statement that ‘normal performance limits have been defined as 0–2 omissions in 120 seconds’ (Diller, Ben-Yishay, & Gerstman, 1974; Lezak et al., 2012, p. 381). After standardising the administration and scoring procedures for the test, this study provides detailed data on three performance categories (i.e. omissions, errors and time) on the total score as well as the six parts of the H and CE letter cancellation tests respectively. For the target population in this study, normal performance limits on the H letter cancellation test are defined as 196.57 seconds, with 2.59 omissions (for six parts). For the CE letter cancellation test, the normal performance limits are defined as 316.03 seconds, with 12.73 omissions (for six parts).

Although Diller et al. (1974) found that there were no significant differences in performance based on gender and age, the present study did find age and gender to impact on an individual’s time taken to complete the tests. In addition to the comparative data for the total sample provided in this manuscript, stratification in terms of age and gender was necessary. Significant differences between males and females in terms of the time taken to complete the tasks are consistent with the findings of Pradhan and Nagendra (2008), Upadhayay and Guragain (2014) and Uttl and Pilkenton-Taylor (2001). Upadhayay and Guragain (2014) also found that women performed faster than men in paper-and-pencil tests. These findings could be partly explained by the fact that different parts of men’s and women’s brains are activated during different tasks, thus demonstrating that the genders utilise different parts of their brains to solve problems (Brizendine, 2009).

Significant differences were also found between those below and above 40 years of age with the latter taking more time to complete the tasks. Age-related decline in speed for the letter cancellation test has been reported previously (Pradhan & Nagendra, 2008; Uttl & Pilkenton-Taylor, 2001). This may be accounted for by age-related slowing and attentional deficits (Erel & Levy, 2016; Fortinash & Worret, 2014; Kramer & Madden, 2011). Deficits have been noted in the ability to selectively attend to certain tasks (Brink & McDowd, 1999; Glisky, 2007), for example, when requiring an individual to focus their attention on one stimulus among several other sets of information. Madden et al. (2007) found that older adults performed slower and less accurately than younger adults in visual search tests. Military personnel are recruited at a young age, based on their functioning at that point in time. Continuous evaluation of fitness for duty would therefore imply the need to assess any decline, especially in attention associated with normal ageing in addition to those associated with injury.

Skewness could be attributed to the target population being a pre-selected group. According to Kennedy and Moore (eds. 2010), some samples of the military population outperform the general population on neuropsychological tests, as only healthy and generally well-functioning individuals are considered to be fit for duty. Nwafor and Adesuwa (2014) further supported this by adding that the specialised skills required by soldiers for their operational and functional duties require them to function higher than the general population. The raw scores can be converted to standard scores by means of Z-score conversions using the typical performance presented in Tables 2 and 7. A Z-score represents the distance from the mean expressed in standard deviation units (i.e. Z = (the raw score – the mean)/the standard deviation). It is important to note that the distribution of Z-scores has the same form as the raw scores on which they are based. In this instance, the Z-scores will therefore be right skewed and not normally distributed. Although these scores do not have the statistical advantages of normally distributed scores, conversation will nevertheless allow for comparison within and between individuals in this population. In the case of age and gender stratification for total time, comparisons will be limited to each specified group (e.g. females, 20–29 years) (Gadd & Phipps, 2012).

A major contribution of this study is the development of standardised administration and scoring procedures for a test of attention. Additionally, the military context implies a need for appropriate normative data on this construct. However, larger sample sizes are required for adequate representation in terms of some of the demographic variables (i.e. individuals older than 50 years, females older than 40 years and left-handed individuals). This will also enable further exploration of the distribution of the performance. Standard scores based on the present data set cannot be interpreted in terms of the properties of a normal distribution. These recommendations would allow for a comprehensive standardisation and evaluation of the psychometric properties of the test in the military context. The study furthermore involved a highly specific subgroup of the general population and replication studies including additional subpopulations should be considered.

The letter cancellation test is widely used despite being subject to unstandardised administration and scoring procedures and broad cut-off scores. This study provides a review of the letter cancellation test and puts forward improved administration procedures, detailed scoring methods and relevant normative data for adequate sample sizes. This was done to provide clinicians in the SANDF with meaningful scores for interpretation and to guide future developments in the wider South African context.