Abstract
The purpose of this study was to investigate the psychometric properties of the short form Burnout Assessment Tool (BAT-12). As a result of the pandemic, job stress has been compounded and the use of conceptually grounded and accurate measures is needed to identify burnout risks within specific organisations and the overall workforce. The study sample comprised 660 employees from various occupational settings who filled out an online survey. Latent variable methods with ordinal categorical data were implemented to model the data and to test the hypotheses for the study. Results showed that the proposed second-order factor model of the BAT-12 showed a good fit to the data and was invariant across gender and ethnicity. In addition, burnout – as operationalised with the BAT-12 – played the hypothesised mediating role in the Job Demands-Resources model. The BAT-12 also showed convergent validity with the Maslach Burnout Inventory. The authors conclude that BAT-12 is a robust instrument with adequate psychometric properties to measure burnout risk and present a freely available online application for employees to estimate their risk of burnout.
Keywords: burnout; burnout assessment tool; work engagement; Job Demands-Resources model; measurement invariance.
Introduction
Even though there has been an overemphasis on the psychometric properties of burnout scales that has impeded needed theory development (Bakker & De Vries, 2021), the South African context demands, ethically and by law, that instruments present robust evidence of unbiasedness and fair measurement because of the potential of misuse of psychological scales (Barnard, 2021). Furthermore, efficiencies in business have become increasingly important, and the implementation of surveys by researchers and practitioners is not exempt from this, as survey fatigue has been identified as a concern (e.g. De Koning et al., 2021). Gatekeepers to participating employees in organisations therefore consider it important to use accurate short scales where possible. In this way, information that can be extracted from participants is maximised, whilst respecting participants’ and the organisation’s time.
As a result of the coronavirus disease 2019 (COVID-19) pandemic and the associated national lockdowns, organisations did not only struggle to cope (Katare, Marshall, & Valdivia, 2021), but the situation accelerated – by necessity – remote and digital work transformation strategies. This, in turn, has also rekindled a focus on the well-being of employees and public health in general (Juchnowicz & Kinowska, 2021; Kniffin et al., 2021). Consequently, it should come as no surprise that the term ‘burnout’ has been used frequently in the media (e.g. Bernard, 2021) and its dynamics remain a focus of academic research (e.g. Chirico et al., 2021). Burnout has been included in the 11th revision of the International Classification of Diseases (ICD-11) – effective from 2022 – by the World Health Organization (2019) and is classified as an ‘occupational phenomenon’ defined as ‘a syndrome conceptualised as resulting from chronic workplace stress that has not been successfully managed’. The consequences of burnout have become apparent over almost half a century of research: decreased performance (Roczniewska & Bakker, 2021; Taris, 2006), impaired job satisfaction and affective commitment (Park, Nam, & Yang, 2011; Salvagioni et al., 2017), increased turnover intention, negative perceptions of quality and safety (Garcia et al., 2019; Salyers et al., 2017) and more physical and psychological distress symptoms (Salvagioni et al., 2017).
Not only has research in South Africa shown that burnout risk is associated with increased reporting of receiving treatment for conditions such as depression, diabetes, irritable bowel syndrome and hypertension by employees (De Beer, Pienaar, & Rothmann, 2016) but also that the medical aid provider expenditure by private insurers on employees categorised into a high burnout risk group is approximately double the amount compared with a low burnout risk group (De Beer, Pienaar, & Rothmann, 2013). Therefore, burnout does not only affect employees – its ripple effect on the surrounding ecosystem of the organisation and society cannot be discounted. In fact, the cost of burnout to economies has been estimated at between $125 and $190 billion in the United States of America (Garton, 2017). This was well before the pandemic and it is not far-fetched to posit that this estimate might currently be significantly higher. Consequently, high burnout levels are not only an individual risk but also have implications for organisations and the society at large (public health). This means that burnout should be measured (identifiable) and managed to lessen its harmful impact (Salvagioni et al., 2017). Therefore, an accurate short version of a burnout measure becomes increasingly important.
However, the measurement of burnout has been plagued by inconsistencies and criticisms over time. Specifically, the most popular measure of burnout, the Maslach Burnout Inventory (MBI; Maslach & Jackson, 1981; Maslach, Leiter, & Jackson, 2017), was initially never designed or intended as a diagnostic instrument or screening device for any disease, but primarily as a research tool after interviews with individuals in human services work. This has led to critique that as burnout was defined by the MBI there is a conflation of the terminology and instrument that impedes innovation (Schaufeli, Desart, & De Witte, 2020). in addition, peer-reviewed research has shown that the factor structure of MBI-assessed burnout has been partly inconsistent with its generally accepted presentation as a syndrome as modelling has shown not only its proposed three-factor structure but also two-factor, four-factor, five-factor, second-order and bifactor solutions (see De Beer et al., 2020). To some this may not necessarily be problematic, but the reality is that if burnout is considered a syndrome, a total risk score indicated by the individual components should also be feasible. For example, the MBI does not allow for a total score to be established – instructing that its components (emotional exhaustion, cynicism or professional efficacy) should be considered separately (Maslach & Leiter, 2021). However, an overall score, based on the components, is ideal as one then has evidence for a syndrome with a cluster of components that is presented in line with the WHO description of the phenomenon. This then indicates the potential of a unidimensional, second-order (higher-order or hierarchical) and potential bifactor solutions as the possible options available to model burnout as a total score. Moreover, in the absence of evidence-based diagnostic criteria for burnout, erroneous cut-off scores and prevalence estimates of burnout have been presented (Brisson & Bianchi, 2017). Maslach and Leiter (2021) have however decried ‘misuses’ of the MBI to diagnose any disease or present estimate prevalence stating that they ‘… never designed the MBI as a tool to diagnose an individual health problem’ (p. 4).
Furthermore, research evidence supports the notion that the purported third component of burnout, professional efficacy, should not be considered a core aspect of burnout (De Beer & Bianchi, 2019; Kim & Ji, 2009). Researchers have also argued that the positively framed items for the professional efficacy component are problematic as it is measured with positive items, implying wording effects (e.g. Lheureux, Truchot, Borteyrou, & Rascle, 2017). Indeed, research has shown that changing the valence of professional efficacy to professional inefficacy with negatively framed items yielded more accurate results (Schaufeli & Salanova, 2007). Other research has proposed that the efficacy component may act as either an outcome or a precursor of burnout (Schaufeli & Taris, 2005).
Clearly, maintaining the status quo would likely only perpetuate the present situation. This is without even considering the debates in the literature regarding the overlap of burnout with depression (see Bianchi et al., 2021) or reducing the definition of burnout to only exhaustion (see Canu et al., 2021). However, burnout is both about inability and unwillingness (Schaufeli, 2021). Consequently, the Burnout Assessment Tool (BAT) was developed based on the conceptual framework of Schaufeli and Taris (2005), which considers both the aforementioned aspects, to address some of the problems of the MBI by using both an inductive and a deductive approach. For the inductive approach, as burnout has been recognised in the Netherlands as an occupational disease for over two decades, and in Flanders an occupation-related disease, there are various health professionals and occupational physicians who have worked with employees categorised as burned-out. Specifically, 49 Dutch and Flemish professionals were interviewed who are involved at various stages of the burnout process, asked to ‘describe a patient with prototypical burnout symptoms and to focus on specific symptoms, causes, and the way burnout unfolds …’ and ‘… describe burnout in their own words, and to prioritise the burnout symptoms they mentioned in terms of their relevance for diagnosing burnout’ (Schaufeli, Desart, & De Witte, 2020, p. 3). Then, in terms of the deductive development process of the BAT, more than 357 items (representing 66 dimensions) were analysed using factor analytic methods (see Schaufeli et al., 2020, for a complete overview). Based on these approaches, the BAT defines burnout as ‘a work-related state of exhaustion that occurs amongst employees, which is characterised by extreme tiredness, reduced ability to regulate cognitive and emotional processes and mental distancing’ (Schaufeli et al., 2020). Noticeably, professional (in)efficacy is not present as one of the components of the BAT, but there is an addition of two components with exhaustion and mental distance, that is, cognitive impairment (reduced ability to regulate cognitions) and emotional impairment (reduced ability to regulate emotions) (Schaufeli, De Witte, & Desart, 2020). Recent results showed that BAT-23 functions well as a second-order factor in factor analyses of data collected in Italy, Romania, Ecuador, Poland and Korea. In addition, the instrument showed measurement invariance across European countries and Japan (De Beer et al., 2020).
The job demands-resources approach to burnout
Arguably, over the last two decades significant advancements in the field of occupational health psychology have occurred. One of the first was the development of the Job Demands-Resources (JD-R) model of burnout at the turn of the millennium (see Demerouti, Bakker, Nachreiner, & Schaufeli, 2001) – explaining how exhaustion and disengagement may develop as result of working conditions, that is, job demands and job resources. The next was the publication of the Utrecht Work Engagement Scale (UWES) that measures work engagement, which is described as a positive work-related state characterised by vigour, dedication and absorption (Schaufeli & Bakker, 2004; Schaufeli, Salanova, González-Romá, & Bakker, 2002). Work engagement has also been positioned as the positive antipode of burnout (Bakker & Oerlemans, 2011; Schaufeli & Bakker, 2004). Subsequently, the JD-R model was adapted to include work engagement (Schaufeli & Bakker, 2004) and it formally describes dual processes: (1) the health impairment process in which burnout is the result of inordinate job demands (and a lack of job resources) and that burnout in turn leads to undesired outcomes and (2) the motivational process in which work engagement is the result of job resources and this, in turn, leads to desired organisational outcomes (Bakker & Demerouti, 2007, 2017; Bakker, Demerouti & Sanz-Vergel, 2014). As a result of a causal chain of three variables being implied in each process, the possibility of indirect effects, that is, burnout and work engagement acting as potential mediators, will also be revisited as part of this validation.
Subsequently, the following hypotheses are presented for this study:
H1: Burnout, assessed with the BAT-12, can be operationalised as a second-order factor, which is an overall latent score indicated by four latent components.
H2: Burnout, assessed with the BAT-12, shows convergent validity with burnout as assessed with the MBI.
H3: Burnout, assessed with the BAT-12, shows acceptable measurement invariance based on:
gender ethnicity.
H4: Burnout is a mediator in the relationship between job demands (work overload) and turnover intention in the health impairment process of the specified JD-R model.
H5: Burnout is a mediator in the relationship between job resources and turnover intention in the JD-R model.
Therefore, the general objective of this study was to investigate the construct validity of the BAT-12, to test measurement invariance and to gauge BAT-assessed burnout’s performance within mediation model based on the dual process of JD-R theory.
Methods
Study design and participants
The data for this study formed part of the BAT project and were collected at one point in time, indicating a cross-sectional design. Cross-sectional designs are suitable for studies that seek to establish the psychometric properties and correlational relationships between variables. The data were collected using a purposive sampling strategy, that is, participants had to be South African employees at least 18 years of age.
Participants
Participants were recruited via social media and could voluntarily participate according to their own volition. The sample comprised 660 employees working in South Africa. The minority of the participants were men (n = 277; 42%) and the average age of the participants was 38 years, with a standard deviation of 10.60 years. Regarding ethnicity, most of the sample participants were African employees (39%), followed by white employees (29.70%), coloured1 employees (12.90%) and Indian employees (4.42%).
Measuring instruments
Burnout was measured with the short form BAT-12 (Schaufeli, De Witte, & Desart, 2020). The scale comprises 12 items measuring the four components of BAT-defined burnout with three items for each of the components (exhaustion, mental distance, cognitive impairment and emotional impairment). The items of the BAT-12 are provided in Table 1. Maslach Burnout Inventory-assessed burnout was measured using the 16-item version of the Maslach Burnout Inventory-General Survey (MBI-GS): emotional exhaustion, cynicism and professional efficacy (Schaufeli, Leiter, Maslach, & Jackson, 1996). The job demands and job resources used in this study were measured with scales from the job demands-resources scale (JDRS) that was validated by Rothmann, Mostert and Strydom (2006). Specifically, the following dimensions were used and rated on a four-point Likert scale, ranging from Never to Always: work overload (six items; e.g. ‘I have too much work to do’), autonomy (three items; e.g. ‘Do you have influence in planning your work activities?’), colleague support (three items; e.g. ‘Can you count on your colleagues when you come across difficulties in your work?’), supervisor support (three items; e.g. ‘Can you count on your direct supervisor when you come across difficulties in your work?’) and role clarity (four items; e.g. ‘Do you know exactly what other people expect of you in your work?’). Work engagement was measured with the three-item ultra-short version of the UWES-3 (e.g. ‘At work, I feel bursting with energy’) (Schaufeli, Shimazu, Hakanen, Salanova, & De Witte, 2019). Lastly, turnover intention was measured with a three-item scale (e.g. ‘I am actively looking for other jobs’) (Sjöberg & Sverke, 2000).
| TABLE 1: Factor loadings from the confirmatory factor analysis model. |
| TABLE 2: Descriptive statistics, omega reliability, average variance extracted and correlation matrix with shared variances. |
Data analysis
The software program Mplus 8.6 (Muthén & Muthén, 2021) was used to model the data. It is important to note that the items were considered to be ordered categorical in nature and not purely continuous. Therefore, the mean- and variance-adjusted weighted least squares (WLSMV) estimation method was used, as this estimator is also robust against non-normality of data (Li, 2016). Specifically, confirmatory factor analysis (CFA) was implemented by specifying a second-order model – in line with the assumption that the BAT should also be able to model burnout to be a syndrome indicated by its four first-order components. In terms of fit statistics, the comparative fit index (CFI) and Tucker–Lewis index (TLI) were considered and these values need to be above 0.90 (Van de Schoot, Lugtig, & Hox, 2012). In addition, the root mean squared error of approximation (RMSEA) and standardised root mean residual (SRMR) were also considered, and these values should ideally be below 0.08. However, recent research has shown that SRMR performs better compared with RMSEA when data are estimated as ordered categorical in nature (see Shi, Maydeu-Olivares, & Rosseel, 2020). Factor loadings were considered acceptable at approximately 0.50 and effect sizes for correlation coefficients were small (0.10+), medium (0.30+) and large (0.50+). For support of discriminant validity correlation coefficients had to be below the guideline of 0.85 in all correlational relationships between the variables (Brown, 2015).
To test the equivalence of the BAT-12 across gender and ethnicity, measurement invariance analyses were implemented with WLSMV and theta parameterisation (Millsap & Yun-Tein, 2004). As the data were specified as categorical (considering category thresholds and not only intercepts) and the BAT-12 was modelled as a second-order factor, the analyses were also somewhat more complex when compared with normal measurement invariance with maximum likelihood and continuous data. A series of models had to be tested for both gender and ethnicity in line with the approach taken in De Beer et al. (2020) when the BAT-23 was tested for invariance in a sample of six European countries and Japan (see Table 3 for the model descriptions). As there is no agreement in the literature as to whether loading or threshold invariance should be tested first, this step was combined (see De Beer et al., 2020, for a complete overview). Moreover, as guidelines for delta (Δ) changes in CFI and RMSEA for second-order models with categorical data have not been formally established, we used a change in CFI of no larger than 0.008 and RMSEA of 0.060 for the first-order models to not be significantly worse-fitting (cf. De Beer et al., 2020). But we used the conventional criteria of changes no larger than 0.010 for CFI and 0.015 for RMSEA between the second-order models as these included intercept parameters (Rudnev, Lytkina, Davidov, Schmidt, & Zick, 2018). If these aforementioned criteria were met between the models, the BAT-12 could be considered invariant across gender and ethnicity in the sample, allowing for fair comparison between groups if required.
| TABLE 3: Results of the second-order measurement invariance testing. |
To test the criterion validity of the BAT-12, a classical dual process model based on JD-R theory was specified as a mediation model – see Figure 1. In this mediation model, the focus was on the significance, size and direction of the standardised beta coefficients. The bootstrapping option was also enabled to resample 50 000 times from the data to obtain 95% confidence intervals (CIs) for the indirect effects in the model. For a meaningful indirect effect to exist, the guideline is that the 95% CI for that parameter should not include the value zero, that is, the parameter should not change sign from negative to positive or vice versa.
 |
FIGURE 1: The Job Demands-Resources model for the research study. |
|
Ethical considerations
Ethical clearance to conduct the study was obtained from the Economic and Management Sciences Research Ethics Committee of the North-West University (NWU-00558-17-A4). The participants followed a process of informed consent that explained the purpose of the study and that all data would be handled in a confidential manner. Every person had to agree to participate in the study before they could continue with answering any of the questions in the survey. As the project was advertised online, the possibility of repercussions for any person who did not wish to participate in the study is almost impossible as these participants cannot be identified.
Results
Modelling the Burnout Assessment Tool-12 as a second-order model
The CFA modelling of the BAT as a second-order factor indicated by four first-order factors (exhaustion, mental distance, cognitive impairment and emotional impairment) resulted in a good fit to the data: χ2 = 541.33; df = 50; CFI = 0.95; TLI = 0.93; RMSEA = 0.12; and SRMR = 0.06. All the fit statistics except the RMSEA were satisfactory, but as mentioned it has been shown that RMSEA is biased when ordered categorical data are used in estimation procedures. We therefore deferred to the SRMR which has shown to be more accurate under these conditions (Shi et al., 2020). This second-order model was compared to a strictly unidimensional (one-factor) model that was clearly shown to be inferior: χ2 = 1330.31; df = 54; CFI = 0.86; TLI = 0.82; RMSEA = 0.19; and SRMR = 0.08. Table 1 presents factor loading values, standard errors and the associated statistical significance values for the second-order model.
As shown in Table 1, all factor loadings were significant; p < 0.001 for all items. The values of the loadings were all the given guideline of 0.50 (Hair, Black, Babin, & Anderson, 2010) and a majority above 0.70, except for mental distance item 2 which had a loading of 0.47. However, given that this was only a 0.03 difference from the guideline, we decided to keep the item, as it was well above the conventional 0.30 criterion, and a factor with two items would not be identified and disqualify the measure from cross-country comparison in future studies.
As shown in Table 2, all components of the BAT-12 showed acceptable omega reliability estimates (ω > 0.70). Furthermore, the AVE as indicator of convergent validity was also satisfied, except for mental distance which was just below the 0.50 cut-off. However, one must be pragmatic in considering cut-off values, and mental distance still showed discriminant validity in all of its correlations with the other BAT variables. Indeed, discriminant validity was evident for all correlational relationships as the AVEs for all factors were greater than the shared variances (squared correlations) between them – indicating that the components of the BAT-12 can be distinguished from one another in line with a ‘syndrome’ (i.e. a set of underlying symptoms that refer to an underlying common condition). Furthermore, all correlations between the BAT components showed a large effect size (r ≥ 0.63).
Therefore, based on the given evidence, H1 was supported because BAT-12-assessed burnout can be modelled as a second-order model indicated by four first-order factors, namely, exhaustion, mental distance, cognitive impairment and emotional impairment.
Convergent validity of burnout measured by the Burnout Assessment Tool-12 and the Maslach Burnout Inventory
To test H2, a CFA model was specified with the BAT as a second-order burnout factor (indicated by exhaustion, mental distance, cognitive impairment and emotional impairment) and the Maslach Burnout Inventory (MBI) as a two-factor model: burnout (indicated by its exhaustion and mental distance items as the core) and professional efficacy as a separate factor. This model showed an acceptable fit to the data: χ2 = 2668.09; df = 343; CFI = 0.92; TLI = 0.91; RMSEA = 0.101; and SRMR = 0.06. The correlation between BAT-assessed burnout and the MBI-assessed burnout was 0.92, which indicates strong evidence for convergent validity (83.72% shared variance) – supporting H2. The correlations with professional efficacy showed medium effect sizes with both the MBI (r = –0.49) and the BAT (r = –0.45) assessed burnout.
Second-order measurement invariance of the Burnout Assessment Tool-12 based on gender and ethnicity
The results of the measurement invariance models showed that the BAT-12 was invariant across gender and ethnicity – see Table 3. Specifically, the changes (Δ) in CFA, RMSEA and SRMR all met the criteria followed as described. Hypotheses 3a and 3b were therefore supported; the BAT-12 measures fairly across these groups and levels of burnout can be directly compared if required.
Criterion-related validity of the Burnout Assessment Tool-12 in the Job Demands-Resources model
The model specified for the criterion-related validity of burnout as assessed by the BAT-12 in the context of JD-R theory also showed a good fit to the data: χ2 = 2262.03; df = 617; CFI = 0.93; TLI = 0.92; RMSEA = 0.06; and SRMR = 0.06. The omega reliabilities and the correlations for the model are provided in Table 4. As can be seen, all factors were reliable and the correlations were all in the expected directions.
| TABLE 4: Correlation matrix for the Job Demands–Resources model. |
As can be seen from Figure 1, the path coefficients (structural relationships) in the model were as expected. Work overload showed a positive path to burnout (β = 0.49), and burnout was positively related to turnover intention (β = 0.31). Job resources had a negative path to burnout (β = –0.58) and a positive path to work engagement (β = 0.63). Work engagement, in turn, had a negative path to turnover intention (β = –0.14). Lastly, work overload did not have a significant path to turnover intention (p = 0.15), but job resources showed a negative direct path to turnover intention (β = –0.28).
Bootstrapping revealed that there was a meaningful indirect effect from work overload to turnover intention through burnout (β = 0.15, 95% CI [0.05, 0.26]). Similarly, job resources had a negative indirect effect on turnover intention through burnout (β = –0.18, 95% CI [–0.30, –0.07]). These results supported H4 and H5. As an additional analysis, work engagement was also tested as a mediator in the relationship between job resources and turnover intention. The 95% CI crossed through zero by the closest possible margin (β = –0.09, 95% CI [–0.18, 0.001]). But given this is a thousandth of a decimal threshold and considering the 90% CIs, there is tentative evidence for an effect (β = –0.09, 90% CI [–0.17, –0.01]) in line with the literature.
Discussion
The purpose of this study was to investigate the validity and measurement invariance of the short form BAT-12 in the context of JD-R theory. The results of CFA showed that BAT-12 can be modelled as a second-order factor indicated by four facets: exhaustion, mental distance, cognitive impairment and emotional impairment. These findings are consistent with theoretical expectations for the instrument based on Schaufeli, Desart, & De Witte (2020). Furthermore, there were no discriminant validity concerns between the components of the BAT-12. All in all, the BAT-12 showed robust construct validity, supporting H1.
The BAT-12 also showed convergent validity with the MBI – modelled with its core items of emotional exhaustion and cynicism – indicating a similar concept being measured, supporting H2. Although this overlap is substantial, it is important to consider that the BAT was developed not only inductively but also deductively and explicitly includes the component of executive functioning that may be impaired: cognitive impairment (see Deligkaris, Panagopoulou, Montgomery, & Masoura, 2014; Demerouti, Bakker, Peeters, & Breevaart, 2021) and emotional impairment that is not present within the MBI.
Furthermore, H3a and H3b were also supported as the BAT-12 was found to be invariant for both gender and ethnicity. This result is in line with the measurement invariance tests conducted for the BAT-23 within the South African context that showed strong measurement invariance for gender and ethnicity (De Beer et al., 2022) and other invariance tests that have shown the cross-cultural validity of the BAT (e.g. De Beer et al., 2020). Consequently, the BAT-12 can be used to compare scores fairly between groups or persons if such comparisons are needed.
Finally, considering H4–H5, the proposed JD-R model showed a good fit to the data and the indirect effects were generally as expected. There were indirect effects from job demands (work overload) and job resources through BAT-12 burnout to turnover intention. Specifically, job demands had a positive effect and job resources had a negative effect – indicating the importance of optimal resources. Therefore, strong evidence for the health impairment process was present. Contrastingly, even though the direct effects in the proposed regression chain were significant for the motivational process, the indirect effect from job resources to turnover intention through the ultra-short work engagement construct was only marginally meaningful. Considering the literature, the 90% CIs and the very small violation of the guideline (one thousandths of a decimal) this is considered an artefact in this sample and future studies will likely find different. In general, the results are in line with previous studies on the JD-R model in South Africa (e.g. De Beer, Rothmann, & Pienaar, 2012).
In summary, BAT-12 was shown to have robust psychometric properties and the instrument can be used in a valid way to measure employees’ burnout levels.
Limitations and recommendations for future research
This study is not without limitations. Firstly, this study used a cross-sectional design – hence it was not possible to investigate the test–retest reliability of the BAT-12, even though adequate omega reliability coefficients were presented. Future studies should therefore consider a longitudinal design to investigate test–retest reliability and establish causal ordering in the nomological network. Secondly, the sample was non-probability and therefore cannot be completely representative of the South African working population. Therefore, generalisation is cautioned, even though the results are in line with the available literature in other contexts. Lastly, this study did not include a measure of depression. The debate about the overlap of burnout and depression is current in the literature (e.g. Bianchi et al., 2021; Meier & Kim, 2021) and the BAT should also be investigated in this context. Future studies should therefore consider including measures of depression and using techniques such as latent profile analyses and bifactor exploratory structural equation modelling analyses to attempt to disentangle the overlap of the BAT with depression scales (see Morin, Arens, & Marsh, 2016). Another avenue is to identify a group with serious burnout problems and use receiver operating characteristic (ROC) analysis to establish cut-off values that can be used for screening to identify (potential) burnout cases. For other future research direction considerations, we referred to Demerouti et al. (2021).
Conclusion
The results of this study indicate that the BAT-12 is a robust tool to measure the burnout risk of employees within an organisational context. More specifically, the BAT-12 can be used to measure individual levels of burnout, as well as group-level burnout within a company as part of a psychosocial risk analysis. The BAT-assessed burnout also performs well within a JD-R framework to explain the process of health impairment in employees. However, it must be emphasised that at present the BAT does not categorise someone as burned out or not burned out and only assesses burnout risk (level), which if problematic should refer the employee to the necessary employee assistance programme or relevant health professional for a clinical interview. Therefore, prevalence estimates are discouraged.
An online application for South African employees to screen their personal burnout risk level against the current norms of the BAT project data set is freely accessible at https://theburnout.app/?mod=bat12sa. We are optimistic that this validation study and the online application will assist South African organisations and their employees to prevent burnout and facilitate occupational well-being.
Acknowledgements
The authors would like to acknowledge Maksim Rudnev’s work on second-order measurement invariance in general, as well as his assistance with the adaptation of the online application of the BAT-23 to be able to accommodate the 12-item version of the BAT for South Africa.
Competing interests
The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.
Authors’ contributions
L.D.B. was responsible for conceptualisation, methodology, statistical analyses and writing of the original draft. W.B.S and A.B.B. were involved in the writing, review and editing of the article. All the authors have read and approved the final version of the manuscript to be published.
Funding information
The data for this study formed part of the BAT project in South Africa that was supported by the National Research Foundation (South Africa) with reference number CSRP170523232041 (Grant No: 112106). The views and opinions expressed are those of the researchers and do not reflect the opinions or views of the National Research Foundation.
Data availability
The data set used and analysed during this study can be available from the corresponding author upon reasonable requests.
Disclaimer
The views and opinions expressed in this article are those of the authors and do not reflect the opinions or views of the National Research Foundation or any other agency of the authors.
References
Bakker, A.B., & Demerouti, E. (2007). The job demands-resources model: State of the art. Journal of Managerial Psychology, 22(3), 309–328. https://doi.org/10.1108/02683940710733115
Bakker, A.B., & Demerouti, E. (2017). Job demands–resources theory: Taking stock and looking forward. Journal of Occupational Health Psychology, 22(3), 273–285. https://doi.org/10.1037/ocp0000056
Bakker, A.B., Demerouti, E., & Sanz-Vergel, A.I. (2014). Burnout and work engagement: The JD–R approach. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 389–411. https://doi.org/10.1146/annurev-orgpsych-031413-091235
Bakker, A.B., & De Vries, J.D. (2021). Job demands–resources theory and self-regulation: New explanations and remedies for job burnout. Anxiety, Stress, & Coping, 34(1), 1–21. https://doi.org/10.1080/10615806.2020.1797695
Bakker, A.B., & Oerlemans, W. (2011). Subjective well-being in organizations. In K.S. Cameron & G.M. Spreitzer (Eds.), The Oxford handbook of positive organizational scholarship (pp. 178–189). New York, NY, USA: Oxford University Press.
Barnard, A. (2021). Psychological assessment: Predictors of human behaviour. In M. Coetzee, E. Botha & L. De Beer (Eds.), Personnel psychology: An applied perspective (3rd ed., pp. 159–200). Cape Town: Oxford University Press.
Bernard, A. (2021, October). COVID-19 pandemic leading to higher levels of employee burnout. TechRepublic. Retrieved from https://www.techrepublic.com/article/covid-19-pandemic-leading-to-higher-levels-of-employee-burnout/
Bianchi, R., Verkuilen, J., Schonfeld, I.S., Hakanen, J.J., Jansson-Fröjmark, M., Manzano-García, G., … Meier, L.L. (2021). Is burnout a depressive condition? A 14-sample meta-analytic and bifactor analytic study. Clinical Psychological Science, 9(14), 579–597. https://doi.org/10.1177/2167702620979597
Brisson, R., & Bianchi, R. (2017). Stranger things: On the upside down world of burnout research. Academic Psychiatry, 41(2), 200–201. https://doi.org/10.1007/s40596-016-0619-7
Brown, T.A. (2015). Confirmatory factor analysis for applied research (2nd ed.). New York, NY: Guilford Press.
Canu, I.G., Marca, S.C., Dell’Oro, F., Balázs, Á., Bergamaschi, E., Besse, C., … Wahlen, A. (2021). Harmonized definition of occupational burnout: A systematic review, semantic analysis, and Delphi consensus in 29 countries. Scandinavian Journal of Work, Environment & Health, 47(2), 95–107. https://doi.org/10.5271/sjweh.3935
Chirico, F., Ferrari, G., Nucera, G., Szarpak, L., Crescenzo, P., & Ilesanmi, O. (2021). Prevalence of anxiety, depression, burnout syndrome, and mental health disorders among healthcare workers during the COVID-19 pandemic: A rapid umbrella review of systematic reviews. Journal of Health and Social Sciences, 6(2), 209–220. https://doi.org/10.19204/2021/prvl7
De Beer, L.T. (2021). Is there utility in specifying professional efficacy as an outcome of burnout in the employee health impairment process. International Journal of Environmental Research and Public Health, 18(12), 6255. https://doi.org/10.3390/ijerph18126255
De Beer, L.T., & Bianchi, R. (2019). Confirmatory factor analysis of the Maslach Burnout Inventory: A Bayesian structural equation modeling approach. European Journal of Psychological Assessment, 35(2), 217–224. https://doi.org/10.1027/1015-5759/a000392
De Beer, L.T., Pienaar, J., & Rothmann, S., Jr. (2013). Linking employee burnout with medical aid provider expenditure. South African Medical Journal, 103(2), 89–93. https://doi.org/10.7196/samj.6060
De Beer, L.T., Pienaar, J., & Rothmann, S., Jr. (2016). Job burnout, work engagement and self-reported treatment for health conditions in South Africa. Stress and Health, 32(1), 36–46. https://doi.org/10.1002/smi.2576
De Beer, L., Rothmann, S., Jr., & Pienaar, J. (2012). A confirmatory investigation of a job demands-resources model using a categorical estimator. Psychological Reports, 111(2), 528–544. https://doi.org/10.2466/01.03.10.PR0.111.5.528-544
De Beer, L.T., Schaufeli, W.B., & De Witte, H. (2022). The psychometric properties and measurement invariance of the Burnout Assessment Tool (BAT-23) in South Africa (Unpublished Manuscript). North-West University, Potchefstroom Campus.
De Beer, L.T., Schaufeli, W.B., De Witte, H., Hakanen, J.J., Shimazu, A., Glaser, J., … Rudnev, M. (2020). Measurement invariance of the Burnout Assessment Tool (BAT) across seven cross-national representative samples. International Journal of Environmental Research and Public Health, 17(15), 5604. https://doi.org/10.3390/ijerph17155604
De Koning, R., Egiz, A., Kotecha, J., Ciuculete, A.C., Ooi, S.Z.Y., Bankole, N.D.A., … & Kanmounye, U.S. (2021). Survey fatigue during the COVID-19 pandemic: an analysis of neurosurgery survey response rates. Frontiers in Surgery, 8, 690680. https://doi.org/10.3389/fsurg.2021.690680
Deligkaris, P., Panagopoulou, E., Montgomery, A.J., & Masoura, E. (2014). Job burnout and cognitive functioning: A systematic review. Work and Stress, 28, 107–123. https://doi.org/10.1080/02678373.2014.909545
Demerouti, E., Bakker, A.B., Nachreiner, F., & Schaufeli, W.B. (2001). The job demands-resources model of burnout. Journal of Applied Psychology, 86(3), 499–512. https://doi.org/10.1037/0021-9010.86.3.499
Demerouti, E., Bakker, A.B., Peeters, M.C., & Breevaart, K. (2021). New directions in burnout research. European Journal of Work and Organizational Psychology, xx, 1–6. https://doi.org/10.1080/1359432X.2021.1979962
Garcia, C.D.L., Abreu, L.C.D., Ramos, J.L.S., Castro, C.F.D.D., Smiderle, F.R.N., … Bezerra, I.M.P. (2019). Influence of burnout on patient safety: Systematic review and meta-analysis. Medicina, 55(9), 553. https://doi.org/10.3390/medicina55090553
Garton, E. (2017, April). Employee burnout is a problem with the company, not the person. Harvard Business Review. Retrieved from https://hbr.org/2017/04/employee-burnout-is-a-problem-with-the-company-not-the-person
Hair, J.F., Black, W.C., Babin, B.J., & Anderson, R.E. (2010). Multivariate data analysis: A global perspective (7th ed.). Upper Saddle River, NJ, London: Pearson Education.
Juchnowicz, M., & Kinowska, H. (2021). Employee well-being and digital work during the COVID-19 pandemic. Information, 12(8), 293. https://doi.org/10.3390/info12080293
Katare, B., Marshall, M.I., & Valdivia, C.B. (2021). Bend or break? Small business survival and strategies during the COVID-19 shock. International Journal of Disaster Risk Reduction, 61, 102332. https://doi.org/10.1016/j.ijdrr.2021.102332
Kim, H., & Ji, J. (2009). Factor structure and longitudinal invariance of the Maslach Burnout Inventory. Research on Social Work Practice, 19(3), 325–339. https://doi.org/10.1177/1049731508318550
Kniffin, K.M., Narayanan, J., Anseel, F., Antonakis, J., Ashford, S.P., Bakker, A.B., … Van Vugt, M. (2021). COVID-19 and the workplace: Implications, issues, and insights for future research and action. American Psychologist, 76(1), 63–77. https://psycnet.apa.org/doi/10.1037/amp0000716
Lheureux, F., Truchot, D., Borteyrou, X., & Rascle, N. (2017). The Maslach Burnout Inventory–Human Services Survey (MBI-HSS): Factor structure, wording effect and psychometric qualities of known problematic items. Le Travail Humain, 80(2), 161–186. https://doi.org/10.3917/th.802.0161
Li, C.H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387. https://doi.org/10.1037/met0000093
Maslach, C., & Jackson, S.E. (1981). The measurement of experienced burnout. Journal of Organizational Behavior, 2(2), 99–113. https://doi.org/10.1002/job.4030020205
Maslach, C., & Leiter, M.P. (2021, March 19). How to measure burnout accurately and ethically. Harvard Business Review. Retrieved from https://hbr.org/2021/03/how-to-measure-burnout-accurately-and-ethically
Maslach, C., Leiter, M.P., & Jackson, S.E. (2017). Maslach Burnout Inventory manual (4th ed.). Palo Alto, CA, USA: Mind Garden.
Meier, S.T., & Kim, S. (2021). Meta-regression analyses of relationships between burnout and depression with sampling and measurement methodological moderators. Journal of Occupational Health Psychology. Retrieved from https://psycnet.apa.org/doi/10.1037/ocp0000273
Millsap, R.E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479–515. https://doi.org/10.1207/S15327906MBR3903_4
Morin, A.J., Arens, A.K., & Marsh, H.W. (2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal, 23(1), 116–139. https://doi.org/10.1080/10705511.2014.961800
Muthén, L.K., & Muthén, B.O. (2021). Mplus user’s guide (8 ed.). Los Angeles, CA: Muthén & Muthén.
Park, H., Nam, S., & Yang, E. (2011). Relationships of burnout with job attitudes and turnover intention among Koreans: A meta-analysis. Korean Journal of Industrial and Organizational Psychology, 24(3), 457–491. https://doi.org/10.24230/kjiop.v24i3.457-491
Roczniewska, M., & Bakker, A.B. (2021). Burnout and self-regulation failure: A diary study of self-undermining and job crafting among nurses. Journal of Advanced Nursing, 77(8), 3424–3435. https://doi.org/10.1111/jan.14872
Rothmann, S., Mostert, K., & Strydom, M. (2006). A psychometric evaluation of the job demands-resources scale in South Africa. SA Journal of Industrial Psychology, 32(4), 76–86. Retrieved from https://hdl.handle.net/10520/EJC89107
Rudnev, M., Lytkina, E., Davidov, E., Schmidt, P., & Zick, A. (2018). Testing measurement invariance for a second-order factor: A cross-national test of the Alienation Scale. Methods, Data, Analyses: A Journal for Quantitative Methods and Survey Methodology, 12(1), 47–76. https://doi.org/10.12758/mda.2017.11
Salvagioni, D.A.J., Melanda, F.N., Mesas, A.E., González, A.D., Gabani, F.L., & De Andrade, S.M. (2017). Physical, psychological and occupational consequences of job burnout: A systematic review of prospective studies. PLoS One, 12(10), e0185781. https://doi.org/10.1371/journal.pone.0185781
Salyers, M.P., Bonfils, K.A., Luther, L., Firmin, R.L., White, D.A., Adams, E.L., & Rollins, A.L. (2017). The relationship between professional burnout and quality and safety in healthcare: A meta-analysis. Journal of General Internal Medicine, 32(4), 475–482. https://doi.org/10.1007/s11606-016-3886-9
Schaufeli, W. (2021). The burnout enigma solved? Scandinavian Journal of Work, Environment & Health, 47(3), 169–170. https://doi.org/10.5271/sjweh.3950
Schaufeli, W.B., & Bakker, A.B. (2004). Job demands, job resources, and their relationship with burnout and engagement: A multi-sample study. Journal of Organizational Behavior, 25(3), 293–315. https://doi.org/10.1002/job.248
Schaufeli, W.B., Desart, S., & De Witte, H. (2020). Burnout Assessment Tool (BAT) – Development, validity, and reliability. International Journal of Environmental Research and Public Health, 17(24), 9495. https://doi.org/10.3390/ijerph17249495
Schaufeli, W.B., De Witte, H., & Desart, S. (2020). Manual Burnout Assessment Tool (BAT) – Version 2.0. KU Leuven: Unpublished Internal Report.
Schaufeli, W.B., Leiter, M.P., Maslach, C., & Jackson, S.E. (1996). Maslach Burnout Inventory-General Survey. In C. Maslach, S.E. Jackson, & M.P. Leiter (Eds.), The Maslach Burnout Inventory-Test Manual (3rd ed.). Palo Alto, CA, USA: Consulting Psychologists Press.
Schaufeli, W.B., & Salanova, M. (2007). Efficacy or inefficacy, that’s the question: Burnout and work engagement, and their relationships with efficacy beliefs. Anxiety, Stress & Coping, 20(2), 177–196. https://doi.org/10.1080/10615800701217878
Schaufeli, W.B., Salanova, M., González-Romá, V., & Bakker, A.B. (2002). The measurement of engagement and burnout: A two sample confirmatory factor analytic approach. Journal of Happiness Studies, 3, 71–92. https://doi.org/10.1023/A:1015630930326
Schaufeli, W.B., Shimazu, A., Hakanen, J., Salanova, M., & De Witte, H. (2019). An ultra-short measure for work engagement: The UWES-3 validation across five countries. European Journal of Psychological Assessment, 35(4), 577–591. https://doi.org/10.1027/1015-5759/a000430
Schaufeli, W.B., & Taris, T.W. (2005). The conceptualization and measurement of burnout: Common ground and worlds apart. Work & Stress, 19(3), 256–262. https://doi.org/10.1080/02678370500385913
Sjöberg, A., & Sverke, M. (2000). The interactive effect of job involvement and organizational commitment on job turnover revisited: A note on the mediating role of turnover intention. Scandinavian Journal of Psychology, 41(3), 247–252. https://doi.org/10.1111/1467-9450.00194
Shi, D., Maydeu-Olivares, A., & Rosseel, Y. (2020). Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Structural Equation Modeling: A Multidisciplinary Journal, 27(1), 1–15. https://doi.org/10.1080/10705511.2019.1611434
Taris, T.W. (2006). Is there a relationship between burnout and objective performance? A critical review of 16 studies. Work & Stress, 20(4), 316–334. https://doi.org/10.1080/02678370601065893
Van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9(4), 486–492. https://doi.org/10.1080/17405629.2012.686740
World Health Organization. (2019, May). Burn-out an “occupational phenomenon”: International Classification of Diseases. Retrieved from https://www.who.int/news/item/28-05-2019-burn-out-an-occupational-phenomenon-international-classification-of-diseases
Footnote
1. All descriptions in this section are used in line with the terminology of the Employment Equity Act, 55 of 1998 for designated and non-designated groups. ‘Coloured’ is an official term in South Africa and indicates citizens of mixed ethnic origins. No offense is intended.
|
Email this article 
