Introduction
Executive functioning (EF) is an umbrella term for the capacity to create, sustain and shift mental sets (Suchy, 2009, 2016). Broadly, it refers to a set of top-down cognitive processes responsible for the management of coordinated thought and action (Gray-Burrows et al., 2019). There appears to be some consensus amongst researchers that there are essentially three core domains of EF (Diamond, 2013; Gray-Burrows et al., 2019; Miyake & Friedman, 2012). These include: inhibition (i.e. referring to cognitive and behavioural restraint and the determination of attentional focus); updating or working memory (i.e. momentarily holding information in memory for later processing) and set shifting (i.e. cognitive flexibility required to switch between mental tasks and operations). In combination, they facilitate several critical capabilities such as reasoning, generating goals and plans along with the ability to sustain attention and motivation to see them through (Aron, 2008). It also includes the mental flexibility required to adjust goals and plans in the event of changing circumstances. This family of behaviours are conscious and effortful and are in contrast to intuitive, instinctive, routine, automatic or otherwise overlearned behaviours (Diamond, 2013).
Whilst there may be broad consensus on the core functions, there is still no single definition or universally adopted conceptualisation of executive function (for reviews see Goldstein, Naglieri, Princiotta, & Otero, 2014; McCloskey & Perkins, 2013). Further, EF is considered a multidimensional construct rather than a single unitary trait. (McCloskey, Perkins, & Van Divner, 2009). Thus, with more than 30 definitions of EF (Goldstein et al., 2014) and as many constructs hypothesised to be contained under this umbrella (McCloskey & Perkins, 2013), it should be clear that EF refers to an array of complex, multidimensional cognitive processes and abilities (Otero & Barker, 2014).
Executive functions are predominately associated with the prefrontal cortex and associated areas (Jacobs, Anderson, & Anderson, 2008; Otero & Barker, 2014). Its developmental progression is prolonged, starting in infancy and continuing to adulthood (De Luca & Leventer, 2008). Early research on executive functions and the parts of the brain they are associated with, involve the well-known story of Phineas Gage, a man who suffered severe damage to his ventromedial prefrontal cortex, which had particularly interesting effects on his executive functions (Barkley, 2012). In subsequent years, interest in EF has continued to increase. This is not surprising, as the relevance of EF can hardly be overstated.
Executive functions play a role in just about every domain of life (Diamond, 2013). For example, they have been investigated in the context of school readiness (Morrison, Ponitz, & McClelland, 2010), school success (Borella, Carretti, & Pelgrina, 2010), job success (Bailey, 2007), romantic relationships (Eakin et al., 2004), health behaviours (Crescioni et al., 2011; Miller, Barnes, & Beaver, 2011), criminal and other potentially threatening behaviours (Broidy et al., 2003; Denson, Pederson, Friese, Hahm, & Roberts, 2011) and even quality of life studies (Brown & Landgraf, 2010; Davis, Marra, Najafzadeh, & Liu-Ambrose, 2010). Not to mention its importance to mental health. For example, EF has been implicated in schizophrenia, obsessive-compulsive disorder, depression, addictions, attention deficit hyperactivity and conduct disorder, to name but a few mental health problems where it has been implicated (Goldstein & Naglieri, 2014).
Importantly, evidence suggests that conditions of disadvantage in early life are associated with adverse cognitive development from childhood through adolescence (Berthelsen, Hayes, White, & Williams, 2018; Hackman & Farah, 2009; Hackman, Farah, & Meaney, 2010; Hackman, Gallop, Evans, & Farah, 2015; McEwen & Gianaros, 2010; Sheridan, Sarsour, Jutte, D’Esposito, & Boyce, 2012). This is particularly relevant to South Africa when considering the disadvantaged circumstances in which many children are raised that render them particularly vulnerable to deficits in EF.
As mentioned earlier, executive functions cover many constructs and behaviours. This has given rise to a number of different approaches to its measurement (Egger, De Mey, & Janssen, 2007; Spinella, 2005). Broadly, these can be categorised into subjective and objective measures of executive functions (Smithmyer, 2013). For example, a well-known objective assessment is the Wisconsin Card Sorting Test. It assesses inhibition and mental flexibility as it requires an individual to maintain a task set, to be flexible in response to feedback and to avoid perseveration by inhibiting prior incorrect responses (Salthouse, Atkinson, & Berish, 2003). Another common objective measure is the Stroop Test. This measure requires inhibiting an overlearned response in order to engage with an incongruent stimulus (MacLeod, 1991). Verbal fluency tests are another important class of objective measures. These require participants to generate several items related to some category, whilst observing and evading replication and using different retrieval strategies (Strauss, Sherman, & Spreen, 2006).
In contrast, subjective measures allow individuals to report on various aspects of EF, which provides an indication of their competence in complex, daily problem-solving activities (Toplak, West, & Stanovich, 2012), commonly referred to as self-rated executive function (SREF) measures. A well-known example includes the Behaviour Rating Inventory of Executive Functioning (BRIEF). This instrument assesses EF behaviours in children and adolescents at home and school environments. There are two versions of this measure. One requires parents and teachers to complete separate forms and the other is self-report (BRIEF-SR; Guy, Isquith, & Gioia, 2004; Toplak et al., 2012). Other self-report measures of EF include the Barkley Deficits in Executive Function Scale – Children and Adolescents (Barkley, 2012), the Delis Rating of Executive Functions (D-REF; Delis, 2012) and the Comprehensive Executive Function Inventory (CEFI; Naglieri & Goldstein, 2013).
However, there are limitations to all measures of EF (Smithmyer, 2013). For example, some do not map well to real world settings, whilst others measure only a single aspect of EF, and some – indeed most – were developed for use in clinical populations. Owing to such limitations, Spinella (2005) undertook development of the Executive Functioning Inventory (EFI), a self-report measure that seeks to index a broad spectrum of executive functions within a healthy population (Egger et al., 2007). In contrast to objective assessments of EF, self-report measures have the added advantage of being cost-effective and easy to administer.
The EFI contains 27 items and 5 subscales, namely Motivational Drive (MD), Impulse Control (IC), Empathy (EM), Organisation (ORG) and Strategic Planning (SP). Using parallel analysis, Spinella (2005) found a five-factor model as best representing the data, which is consistent with the theoretical model. Collectively these factors accounted for 49.7% of total variance (Spinella, 2005). This five-factor structure has also found support in subsequent research (Janssen, De Mey, & Egger, 2009; Smithmyer, 2013). In a second-order factor analysis, Spinella (2005) also found three higher-order factors and argued that this model is consistent with the way executive functions have been associated with the dorsolateral (SP, ORG), orbitofrontal (IC, EM ) and the anterior cingulated (MD scale) regions of the brain (Cummings, 1993). As a reviewer correctly pointed out, such models reflect a time when executive function theories still mirrored functional divisions of the frontal lobes. A view no longer accepted today (Chung, Weyandt, & Swentosky, 2014; Otero & Barker, 2014). Indeed, this model has not found support in other studies examining higher-order models of the EFI. For example, Janssen et al. (2009) only found support for two higher-order factors.
Whilst there appears to be some support for the reliability and construct validity of the EFI, to the authors’ knowledge, the EFI has not been examined for use within the South African context. The purpose of this study is to investigate the internal psychometric properties of the EFI amongst university students for use in this setting. Specifically, its reliability, factor structure and item functioning given its susceptibility to variation when used in different populations. It is therefore important to examine these properties of the EFI in this context.
Discussion
The aim of this study was to examine the psychometric evidence for the EFI amongst South African students. Specifically, its internal consistency reliability, factor structure and item response functioning. Previous studies have found reasonable support for the reliability and factor structure of the EFI in different populations, however, no research has been conducted in South Africa.
The reliability results for the subscales of the EFI were mostly satisfactory, with McDonald’s omega coefficients ranging between 0.59 and 0.76, although MD (ω = 0.59) and IC (ω = 0.64) were somewhat weaker than expected. The Cronbach’s alpha estimates in this study are largely similar to those by Janssen et al. (2009) who reported estimates ranging between 0.63 and 0.69. However, both these studies found one scale each to be notably weaker; IC (0.41) was weaker in the study by Janssen et al. (2009), whilst MD (0.59) was weaker in the present study. Both these studies were conducted on university samples with very similar age and gender representation, although the present study is larger and culturally much more diverse. Spinella’s (2005) initial alpha estimates were somewhat stronger than the present results, whereas in contrast, a later study by Smithmyer (2013) reported slightly weaker results in general compared with the present study. Interestingly, Smithmyer (2013) found SP (0.49) and ORG (0.59) to have the weakest reliability amongst the five subscales, neither of which were flagged for weak reliability in previous work. Overall, there appears to be some fluctuation across reliability estimates in the literature, although results are reasonable in general. All studies – with the exception of Spinella (2005) – were conducted amongst university students, so it remains unclear to what degree these fluctuations are attributable to random sample variation. With respect to the previous samples, however, the present study was unique with regard to its diverse cultural representation. Nonetheless, whilst some weak results were observed for most scales of the EFI at least once in different studies, it is arguably a good thing that no one scale was consistently flagged as problematic. The fact that most estimates were Cronbach’s alpha coefficients – which typically underestimates true reliability – suggests that the EFI’s true reliability is under-reported in the literature and supports the notion that the EFI’s reliability is mostly acceptable.
Turning to the confirmatory factor analysis, the results for both the five-factor and three-factor higher-order models were somewhat ambiguous when considering their goodness-of-fit values. Whilst the absolute fit indices were satisfactory in general, the incremental fit values were weaker. However, when comparing the five-factor and three-factor higher-order models, the former had better fit compared with the latter. As such, the factor loadings of only the five-factor model were presented for consideration. Most items had satisfactory loadings on their expected factors, although there were eight items with relatively weak loadings, which influenced model fit. These results provide support for Spinella’s (2005) theoretical model, with five separate constructs contributing to the assessment of executive function, although model fit was modest. These results are also consistent with Smithmyer (2013), who found support for a five-factor solution. In their evaluation of the Dutch translated version of the EFI, Janssen et al. (2009) were also able to replicate the EFI structure proposed by Spinella (2005), although three items (3, 9, 10) of the SP scale had primary loadings on a separate factor in a principal components analysis. In contrast, these items had satisfactory loadings in the present study, whilst item 13 had a relatively weak loading on this factor.
Compared with the five-factor model, the three-factor higher-order model found weak support in this study. This model corresponds to three major regions of the prefrontal cortex (Cummings, 1993), for which Spinella (2005) reported some support. Like this study, Janssen et al. (2009) also found little support for a three-factor higher order model.
Item response theory analysis showed that the items of the EFI generally function appropriately in their respective subscales. This view is supported by both Rasch rating scale and graded response models. Only the two reversed scored items, 4 and 12, on the MD and EM subscales, respectively, were found to function quite poorly. It is recommended that researchers using the EFI in South Africa carefully examine the impact of these items in their own work. It would also be important to see if these items emerge as problematic in future studies conducted in this context to determine the robustness of the present results.
Some limitations of the present study should be noted. Although the data were collected in a diverse sample of students within an urban setting, its conclusions are necessarily limited to this population. As such, results cannot be generalised to the South African population broadly. Further, as Spinella (2005) indicated, the EFI is potentially sensitive to cross-cultural factors as well as differences in age, gender and education levels. This is important when considering the EFI – or any measure of EF – for use in the South African context. As mentioned before, conditions of disadvantage are known to affect the development of executive functions, and this should be borne in mind given the socio-economic disparities across racial groups that remains present in South African society. Future research is required to explore the degree to which scores on the EFI are invariant across relevant demographic strata.