Depression is a global concern as with an estimated 300 million individuals worldwide experiencing depression. In South Africa, the prevalence rate of depression is estimated at 9.7% of the population. With the increase in mobile internet usage in South Africa, an online depression screening tool could provide opportunities for the screening of depression symptoms aiding access to mental health interventions. This project identified an open access tool for screening depression, the Center for Epidemiologic Studies Depression Scale – Revised (CESD-R), and adapted it for online use by the adult South African population. This study followed on from the adaptation phase on the CESD-R and aimed to determine the content validity of the adapted CESD-R for online use in South Africa using the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) methodology. The study followed a two-phased design. Study one utilised a qualitative approach, where 50 experts commented on the content validity of the tool. The results were used to further adapt the tool which resulted in a 20-item depression screening tool. Study two followed a quantitative design in order to establish the content validity in terms of determining the Content Validity Ratios, Item-Content Validity Index as well as the Kappa Statistic of the 20 items. Based on these statistics, 19 of the 20 items were retained. Overall, the adapted online depression screening tool displays good content validity and holds potential as a screening tool where access to mental health may be limited.
According to the World Health Organization’s (WHO,
The treatment of depression in South Africa is often met with many obstacles, such as stigmas, lack of mental health facilities, lack of depression screening, and under-resourced hospital settings. In 2017, mental health facilities in South Africa were limited with 4.33 beds per 100 000 population in general hospitals, and 16.56 beds per 100 000 in mental health hospitals (WHO,
In an attempt to overcome the barriers to mental health access and care, researchers propose the use of digital technologies such as the Internet, portable electronic devices and mobile applications (Aguilera,
This study focussed on adapting a depression screening tool for online usage. As part of the study, a systematic review was conducted to identify any existing, psychometrically sound, online depression screening tools for the general public of South Africa. Results indicated that the Beck Depression Inventory-II (BDI-II), the Center for Epidemiology Studies Depression Scale (CESD) and the Patient Health Questionnaire (PHQ-9) were the most commonly utilised online depression screening tools, but only one depression screening tool was specifically designed for use by the general public. There were no screening tools specifically designed for the diverse groups in South Africa (Hassem & Laher,
Based on the results of the systematic review, it was decided to adapt the CESD. The revised version of the CESD (CESD-R) is the most recent version of the CESD which reflects the depression symptoms in accordance with the Diagnostic and Statistical Manual of Mental Disorders - Fourth Edition (DSM-IV) (Eaton, Smith, Ybarra, Muntaner, & Tien,
The adaptation of the CESD-R was grounded in the biopsychosocial-spiritual (BPSS) model. The model recognises biological, psychological, social and spiritual as distinct dimensions which cannot be separated from the whole as the components are intertwined allowing for multifactorial understandings of mental illness aetiology (Van Rensburg, Poggenpoel, Myburgh, & Szabo,
While spirituality and culture are relevant across the diverse South African population, screening tools for depression have not been adapted to account for these unique cultural and spiritual presentations of depression. The South Africa population is made up of individuals from various ethnic and religious groups as well as being a multilingual country with 11 official languages. Defining or translating the term ‘depression’ in African cultures is complex because it is not recognised within traditional African practices (Ellis,
The distress idiom called ‘thinking too much’ is also often associated with depression or as a symptom of depression (Kaiser et al.,
When reporting depression symptoms, studies have shown that individuals in African cultures tend to report more somatic than cognitive symptoms (Andersen et al.,
In addition to cultural factors, English language proficiency needs to be considered. According to the General Household Survey (Statistics South Africa,
The unique context of the online psychological assessment screening environment, where the typical face-to-face interaction is removed, necessitates for a number of ethical issues to be considered. Draft ethical guidelines recommended by Hassem and Laher (
The aim of this study was to determine the content validity of the adapted CESD-R for online depression screening for diverse groups in South Africa. Content validity refers to the extent to which items on a scale accurately measure a construct based on the conceptual definition of the construct (Grahn & Gard,
All participants had the right to decline participation. Participation was anonymous and completely confidential as no identifying information was requested. All participants had the right to stop participation at any point and also had the right not to answer any of the questions. There were no potential risks or benefits to participation in the study.
Two studies were carried out in order to determine the content validation of the online depression screening tool. Study one followed a qualitative research design, whereas study two followed a quantitative research design. In the following sections two, the studies are discussed in detail.
The initial step of study one involved the adaptation of the CESD-R items. This step followed the first three stages of the test development guidelines proposed by Foxcroft (
Item adaptation for study one and study two.
CESD-R items | Study one items | Study two items |
---|---|---|
My appetite was poor | I am not eating as much as I normally eat | My eating has changed (eating less than normal or more than normal) |
I could not shake off the blues | I cannot get rid of this sad feeling | I could not get rid of this sad feeling |
I felt depressed | I feel sad | I have been feeling sad or down |
I felt sad | I have been feeling happy | |
My sleep was restless | I do not sleep well | My sleep has changed (having trouble sleeping or sleeping more than usual) |
I had a lot of trouble getting to sleep | I have a lot of trouble going to sleep | |
I slept much more than usual | I sleep much more than usual | |
I could not get going | I cannot do things that I always do | I could not do things that I always do |
Nothing made me happy | Nothing makes me happy | Nothing has made me happy |
I felt like a bad person | I feel like a bad person | I felt that most things are my fault |
I lost interest in my usual activities | I have lost interest in my usual activities | I have lost interest in my usual activities |
I felt like I was moving too slowly | I feel like I am moving too slowly | I felt like I am moving slowly |
I felt fidgety | I have the need to play with my fingers or move around for no reason | I could not make a decision about simple things |
I was tired all the time | I feel tired all the time | I have been feeling tired |
I did not like myself | I do not like myself | I have not liked myself |
I lost a lot of weight without trying to | I lost a lot of weight without trying to | My weight has changed without me trying (lost weight or gained weight) |
I could not focus on the important things. |
I cannot focus on the important things. |
I could not focus on important things. |
I wished I were dead. Item removed | ||
I wanted to hurt myself. Item removed | ||
- | I feel bewitched almost all of the time | Item removed |
- | I am experiencing more body aches and pains lately | I have been experiencing more body aches and pains (e.g. Headaches, neck pain or back pain) |
- | I am thinking too much | I have been thinking too much |
I have been feeling alone | ||
- | - | I have not felt like myself |
CESD-R, Center for Epidemiologic Studies Depression Scale-Revised.
Demographic characteristics of study one and study two.
Variable | Study one | Study two |
---|---|---|
Frequency | Frequency | |
Psychologist | 14 | 16 |
Psychiatrist | 15 | 2 |
Health science/Psychology student | 13 | 3 |
Other | 7 | - |
Less than 10 years | 14 | 1 |
10–20 years | 12 | 10 |
21–30 years | 4 | 4 |
31–40 years | 3 | 2 |
Male | 7 | 4 |
Female | 43 | 17 |
Black | 6 | 3 |
Coloured |
3 | 2 |
Indian | 15 | 3 |
White | 25 | 11 |
Other | 1 | 1 |
Islam | 13 | 2 |
Christianity | 24 | 11 |
Hinduism | 2 | 2 |
Judaism | 1 | 1 |
No religious affiliation | 8 | 5 |
English | 36 | 16 |
Afrikaans | 8 | 2 |
African Language | 6 | 2 |
Other | - | 1 |
Never | 23 | 8 |
Weekly | 17 | 6 |
Once a month | 6 | 1 |
Once in 6 months | 1 | 3 |
Once a year | 2 | 3 |
Yes | 20 | 15 |
No | 29 | 6 |
, The coloured population group in South Africa refers to individuals of mixed-race ancestry.
Individuals were invited to participate in the study based on their expertise, the field of depression screening and depression. A non-probability, purposive sample of 50 mental healthcare personnel participated in the study (Patton,
Questions provided to participants and the frequencies of responses in study one.
Question | Frequency | |
---|---|---|
Yes | No | |
1. Are the instructions provided appropriate? | 47 | 3 |
2. Would you recommend any changes to the instructions provided? | 18 | 32 |
3. Does the tool appear to measure depression? | 46 | 4 |
4. Will the statements be easily understood by the general public? | 49 | 1 |
5. Based on your experience with the culturally diverse population of South Africa, are there any items which need to be changed or added to the tool? | 17 | 31 |
6. The diagnostic and statistical manual-5 is often criticised for being based on Western assumptions of the self which tend to be individualistic, would you recommend changing the ‘I’ in the following statements to ‘my family and friends have noticed’ | ||
6a. I am not eating as much as I normally eat | 18 | 27 |
6b. I lost a lot of weight without trying | 17 | 26 |
6c. I have trouble keeping my mind on what I am doing | 10 | 33 |
6d. I cannot do things that I always do | 13 | 29 |
6e. I am thinking too much recently | 5 | 38 |
6f. I have the need to play with my fingers or move around for no reason | 19 | 24 |
7. Is the scoring provided appropriate for the tool? | 41 | 7 |
8. Is the feedback provided appropriate for the tool? | 35 | 8 |
Just over half of the sample (
Participants were required to complete a brief demographic questionnaire requesting information on occupation, number of years practising, gender, population group, religious affiliation, home language, frequency in diagnosing depression as well as previous experience using a depression screening tool. Once the demographic questions were completed, participants were presented with a page detailing the content validation instructions. Participants were required to only read the instructions, items, scoring as well as feedback of the adapted CESD-R, and not to complete the tool. This was followed by 10 (yes or no response format) questions which were informed by the COSMIN evaluation of content validity of a PROM. Participants were given the choice to provide additional comments in relation to the 10 questions. Given the unique nature of the tool where instant feedback would be provided to the individual, two specific questions assessing the appropriateness of the scoring and feedback of the tool were included.
Once items were finalised, an email was circulated to various South African healthcare professionals listed above (see
Data were downloaded from the online tool and coded for analysis. Demographic variables and yes or no response options were analysed using frequencies on Statistical Package for the Social Sciences (SPSS) Version 25 (IBM Corp,
Results are discussed in terms of the three broad themes recommended in the COSMIN guidelines that are relevance and comprehensiveness, comprehensibility and scoring and feedback.
Because of the open-ended nature of the questions, various themes regarding the relevance of the tool emerged, namely, depression according to the DSM criteria, appropriateness of the items, time period, response format as well as the cultural applicability of the items.
The majority of the participants indicated that the tool does appear to measure depression (
As evident in
‘Some patients eat less and lose weight whilst others eat more and gain weight (this is not dependent on depression type). Asking if there has been a change in weight can be very helpful in distinguishing depression severity.’ (Participant 9, Female, Researcher)
Four participants felt that an item targeting indecisiveness should be added as it is a common cognitive symptom of depressed patients. In addition, suicide ideation items were recommended to be included by five participants as they are symptom criteria for depression, according to the DSM.
This theme is closely related to the theme of item appropriateness as the item, ‘I feel bewitched almost all of the time’, was the most commonly cited item that was inappropriate. This was because the term ‘bewitched’ is not culturally appropriate and the term was not clearly understood. Participants suggested that the item questions the cultural aetiology of depression in a tool which needs to relate to the experience of depression. In addition, participants suggested that the term ‘bewitched’ can have both negative and positive connotations. Participants suggested that if a culturally fair item is to be added, it should have follow-up questions.
With regards to the time frame of ‘two weeks’, three participants noted that the time frame needed to be reconsidered as the symptoms experienced could be a result of life changes or a traumatic event, as indicated by Participant 1:
‘Two weeks is a problematic time period. A person could have undergone a life change or trauma and may not have Major Depressive Disorder (MDD) but would meet the criteria based on the last two weeks. This could result in an implied misdiagnosis.’ (Participant 1, Female, Psychologist)
In addition to the change in the time frame of the symptoms experienced, the attribution of symptoms to life changes was also found in the narratives of four other participants. These participants highlighted that symptoms could be attributed to various medical conditions (chronic conditions or stomach bugs) as well as the lifestyle the individual leads (symptoms of sleep attributed to exhaustion). Therefore, these suggestions showed the need for a statement ruling out medical conditions, lifestyle, as well as life changes causing the symptoms experienced.
Although no specific questions targeted the response format, six participants made comments which related to it. One participant specifically suggested that it might not be understood by the general public, while four participants suggested that the response format did not match well with the items, recommending that either the items be rephrased or the response format be changed for specific items. One participant felt that placing a time frame in the response format was inappropriate.
Lastly, two participants suggested that the responses can yield many false positives or negatives or be easily faked. The use of reverse scoring as well as a control question was suggested in order to prevent faking.
Comprehensibility of the tool is discussed in terms of appropriateness of the instructions, effectiveness of the language used, length of the tool and item order.
As evidenced in
A majority of the participants (
All participants who commented on the length of the tool felt that it was appropriate for an individual who is depressed. This sentiment is represented by the following comment by participant 9: ‘The short length of the questionnaire is a strength of the tool as someone with moderate to severe depression will struggle to complete tasks’.
Only two participants mentioned that the order of the items needs to be reconsidered. Participants suggested that items measuring similar symptoms do not appear together and to prioritise symptoms specific to the South African context in the ordering of items.
Although scoring and feedback does not form part of the COSMIN criteria for content validity, it is a vital point to discuss as the end-users of the tool will ultimately receive immediate feedback in a non-typical, face-to-face interaction.
Only five participants commented on the scoring of the tool, with three participants suggesting that the scoring was not transparent as they did not know the score that was attributed to each response and the cut-offs were not mentioned upfront. Comments made regarding the feedback given to participants all emphasised the need to highlight the urgency of seeking help when individuals receive a high score on the tool. In addition, participants stated that the feedback was calmly stated so that it should not further perpetuate depression symptoms. Participants indicated that individuals who do not meet the criteria for depression should also be offered contact details if they feel the need to seek help. Lastly, participants felt that ‘consulting with your family practitioner’ should be the first point of contact that needs to be included.
A non-probability, purposive sample of 21 mental healthcare personnel participated in the study (Patton,
Only eight participants had not previously diagnosed depression, with the majority of participants (
The first step of this study involved item refinement based on the results of study one. Weight, eating patterns and sleep items were represented by one item each which accounted for either side of the scale (weight gain or loss, increased or decreased appetite and less or more sleep) (see
Participants were required to complete the same brief demographic questionnaire as per study one. Once the demographic questions were completed, participants were presented with a page detailing the content validation instructions. A 3-point Likert scale was used to assess the relevance of each of the 20 items included in the screening tool (1 = item is not essential; 2 = item is useful but not necessary and 3 = item is essential for diagnosing depression). Participants were also asked to rate the response format used as being ‘relevant’, ‘somewhat relevant’ or ‘not relevant’. Lastly, participants were asked for any additional feedback or input on the tool. The questionnaire took, on average, 10 min to complete. Data collection took place electronically between October and December 2019 through an online survey tool.
Data were downloaded from the online tool and coded for analysis. Demographic variables were analysed using frequencies. There are various statistical calculations for content validity, which either provide a consensus estimate or a consistency estimate. Most commonly used in the field of psychology is the Content Validation Ratio (CVR) proposed by Lawshe (
Polit et al. (
In order to compute the I-CVI and Kappa values, response options 2 (‘item is useful but not necessary’) and 3 (‘item is essential for diagnosing depression’) were combined and represented as a relevant item and the response option 1 (‘item is not essential’) represented a non-relevant item.
When looking at items rated as essential by the experts, only three items (3, 8 and 9) were rated by all the 21 experts as being essential. Fifteen items received an essential rating by the majority of the experts (
Content validation statistics of study two.
Item | Ne (Essential) | Nr (relevant) | CVR | I-CVI | Kappa |
---|---|---|---|---|---|
3. I have been feeling sad or down | 21 | 21 | |||
8. I could not get rid of this sad feeling | 21 | 21 | |||
9. I have lost interest in my usual activities | 21 | 21 | |||
14. My sleep has changed (having trouble sleeping or sleeping more than usual) | 21 | 21 | |||
19. Nothing has made me happy | 19 | 21 | |||
4. I had trouble keeping my mind on what I was doing | 18 | 21 | |||
17. I could not focus on important things | 17 | 21 | |||
5. My weight has changed without me trying (lost weight or increased weight) | 15 | 21 | |||
20. I have been feeling alone | 15 | 21 | |||
12. I have not liked myself | 15 | 19 | |||
16. I have been feeling tired | 14 | 21 | 0.333 | ||
18. My eating has changed (eating less than normal or more than normal) | 14 | 21 | 0.333 | ||
21. I have not felt like myself | 14 | 21 | 0.333 | ||
7. I could not make a decision about simple things | 14 | 20 | 0.333 | ||
15. I could not do things that I have always performed | 13 | 19 | 0.238 | ||
10. I felt that most things are my fault | 11 | 20 | 0.048 | ||
2. I have been thinking too much | 10 | 19 | −0.048 | ||
1. I have been experiencing more body aches and pains (e.g. headaches, neck pain or back pain) | 9 | 20 | −0.143 | ||
11. I have been feeling happy | 8 | 13 | −0.238 | 0.62 | |
6. I felt like I have been moving too slowly | 5 | 18 | −0.524 |
Note: Content validation ratio (CVR) = [Ne-(N/2)]/(N/2).
I-CVI, Item-Content Validation Index; S-CVI, Scale-Content Validation Index; I-CVI = Nr/Number of experts.
S-CVI average = 0.95.
Kappa = I-CVI (1)- Pc /1-Pc
S-CVI = [I-CVIitem1+ ICVIitem2+ ICVIitem3… ICVIitem19]/19
Ne, Number of experts rating an item as essential; Nr, Number of experts rating an item as relevant.
Values in bold meet the criteria for inclusion in the online depression screening tool.
The CVR calculations had a very wide range from −0.524 to 1. Both I-CVI and Kappa ranged from 0.62 to 1. The lowest CVR value was obtained on item 6, while the lowest I-CVI and Kappa values were obtained for item 11. It is evident that all 20 items included in the tool were considered as good items with the exception of item 11. Item 11 did not meet two out of the three content validity criteria (CVR and I-CVI). The Kappa value was 0.62, which indicated the item was good.
Using the guidelines in the literature (Ayre & Scally,
The average S-CVI score for the tool was 0.94. The majority of participants (
Two studies were used to determine the content validity of an adapted online depression screening tool. For study one, 50 mental health experts unanimously agreed that the tool was valid in terms of the instructions, items, scoring and feedback provided. However, various suggestions were made to improve the quality of the tool. For study two, the recommendations suggested during study one were undertaken and the tool was revised.
In terms of the COSMIN criteria for content validity, the adaption of the depression screening tool for online usage by diverse groups in South Africa shows good relevance.
A follow-up content validation was conducted with 21 experts in the field and three independent researchers. Nineteen of the experts in study two (final tool) felt that all the items on the tool were relevant for screening of depression, with the exception of item 11 (‘I have been feeling happy’). Only 13 experts felt item 11 was relevant. This could be attributed to the item being reversed scored. Therefore, it was decided that this item would be excluded from the final tool. The average S-CVI further highlights the content validity of the overall tool by achieving an average S-CVI score of higher than 80% (Davis,
Four South African idioms of distress and depression were added to the tool, namely, ‘I have been experiencing more body aches and pains (e.g. headaches, neck pain or back pain)’, ‘I have been thinking too much’, ‘I have been feeling alone’ and ‘I have not felt like myself’. Three of these items did not meet the CVR criteria; however, both the I-CVI and Kappa criteria were met. These items received a low CVR as a result of few experts rating the items as essential in depression screening;, however, many experts indicated that the time was useful in depression screening, hence the items were retained.
The nine items which were rephrased from the CESD-R and which represented the DSM 5 depression criteria received absolute I-CVI and Kappa scores, and met the CVR criteria. Items 7 and 15 did not appear on the original CESD-R, but were recommended by experts in study one to be included in the screening tool. These items did not meet the CVR criterion, but met the I-CVI and Kappa criteria. It should be noted that these two items assessed the symptom criteria for depression as highlighted in the DSM 5 (focus and indecisiveness).
This study demonstrated that a 19-item adapted online depression screening tool displays relevance in terms of the construct being measured and the appropriateness of the target population, and context for which it is intended. There are limitations, in that content validation is a subjective view of experts and therefore could result in bias when items are rated; however, the two studies with various experts and both qualitative and quantitative indicators reduced this risk. It is acknowledged that the pool of experts is small and does not represent the various language groups present in South Africa. Going forward, it would be necessary to establish the content validity in terms of comprehensiveness and comprehensibility ratings as described by COSMIN. These ratings are dependent on the target population views and pilot testing of the tool and not the development stage of the tool. In addition, the construct and criterion validity as well as reliability of the tool would need to be assessed.
This study highlighted that the adapted online depression screening tool designed for diverse groups in South Africa shows good relevance in terms of content validity. In addition, the tool was phrased using simple language free from psychological jargon, which is hoped would encourage better understanding by second language English speakers. Screening individuals for depression allows for early detection of their depression risk and has much to offer for the under-resourced South African mental healthcare landscape. The online screening of depression allows for early detection and self-help options for depression. Further, it empowers individuals to discuss their symptoms from a better knowledge base with a doctor or other healthcare professional allowing for intervention much sooner than would have been the case if the individual had no support or no means of checking their symptoms and of accessing information on depression. The tool, therefore, has the potential to be incorporated as a screening tool for depression across standard platforms in university counselling centres, primary health care intake forms or even on web platforms such as the South African Depression and Anxiety Support Group (SADAG).
The author declares that she has no financial or personal relationships that may have inappropriately influenced her in writing this article.
T.H. is the sole author and was responsible for the conceptualisation, data collection and analysis as well as write up for the article.
The study was approved by the Human Research Ethics Committee – Medical (HRECM) of the University of the Witwatersrand, reference number: M180402.
This work is based on the research supported wholly or in part by the National Research Foundation of South Africa (grant number: 112948).
Data sharing is not possible, due to the nature of the data and related ethical principles.
The views and opinions expressed in this article are those of the author and do not necessarily reflect the official policy or position of any affiliated agency of the author, and the publisher.
The honours degree in South Africa is a postgraduate qualification that follows on from an undergraduate degree and precedes a Master’s degree
Depression in South Africa is diagnosed by clinical, counselling and educational psychologists, general practitioners as well as psychiatrists.