According to MacRae and Furnham (2020), Conscientiousness is a higher-order personality trait in the Five Factor Model. It comprises industriousness, self-control, responsibility, order, traditionalism and virtue. This trait has been found to have a moderate correlation with job success and other job metrics (Barrick et al, 2001; MacRae & Furnham, 2016)

Adjustment is described by MacRae and Furnham (2020) as emotional resilience to stress and positive affect and is the inverse of trait Neuroticism in the Five Factor Model. Higher levels of Adjustment were found to be associated with better teamwork and higher performance, while lower levels were associated with low job satisfaction and subjective well-being (Judge & Locke, 1992; MacRae & Furnham, 2020).

Curiosity is synonymous with the openness trait of the Five Factor Model. The trait is characterised as being open to new ideas and experiences, as well as being creative, reflective and innovative (MacRae & Furnham, 2020). Curiosity and openness were associated with job satisfaction, trainability and learning outcomes (Barrick et al, 2001; Judge et al, 1999; Linden et al, 2010; MacRae & Furnham, 2020).

Risk Approach is defined as how an individual handles challenging, difficult or threatening situations (MacRae & Furnham, 2016). It is the mitigation of negative, threat-based emotions that cause a strong drive to avoid that situation, restricting the potential range of responses to avoidance (MacRae & Furnham, 2020).

Ambiguity Acceptance is a measure of how an individual perceives and processes unfamiliarity and that which is not clear (MacRae & Furnham, 2020). Herman et al. (2010) suggest that tolerance for ambiguity involves unfamiliarity, change, challenging perspectives and valuing diversity. High-fliers and senior leadership are thought to require a tolerance for and adaption to ambiguity because of the need to make sense of and incorporate multiple streams of mixed information to make effective decisions (Keenan & McBain, 1979; MacRae & Furnham, 2020; McCall, 1997).

Finally, MacRae and Furnham (2020) describe Competitiveness as a dimension that drives self-improvement and the desire for success. In a study of sales performance, Wang and Netemeyer (2002) found competitiveness to be a significant predictor of performance.

Rasch analysis provides two fit statistics for persons and items: infit and outfit statistics. Winsteps (Linacre, 2020a) provides a couple of infit and outfit statistics, namely mean-square (MNSQ), which is an average value of residuals, and z-standardised (ZSTD), which is a t-statistic (Boone et al, 2014; Bond et al., 2020).

A reasonable fit statistic range for rating scales, such as Likert-type scales, contains MNSQ values between 0.6 and 1.4 (Wright & Linacre, 1994). On the other hand, MNSQ values less than 0.6 with a ZSTD greater than 2.0 are indicative of an overfitting item and can be interpreted as being more than 40% (1.0 – 0.6 = 0.4) less varied than the Rasch model expects (Bond et al., 2020). A MNSQ value greater than 1.4 with ZSTD greater than 2.0 is indicative of an underfitting item and can be interpreted as being 40% more varied than the Rasch model expects (Bond et al., 2020).

According to Linacre (2020a), outfit is outlier sensitive, and high outfit values tend to be the result of random responses from lower performers. Infit, on the other hand, is information weighted and therefore less influenced by outliers. High infit values are an indication of the items mis-performing and are a greater threat to validity. Linacre (2020a) then suggests that outfit be examined before infit. However, Bond et al. (2020) indicate that deviant infit statistics are more concerning than deviant outfit statistics. Therefore, while outfit will be reported, infit statistics will be the focus, as this statistic is a greater concern to the validity of the scale.

The reliability of an instrument is its degree of consistency at measuring what it purports to measure (Roodt & De Kock, 2018). A typical measure of a questionnaire’s reliability is internal consistency reliability, traditionally evaluated by Cronbach’s (1951) alpha coefficient.

In Rasch measurement, internal consistency reliability is reported as two metrics: person reliability and person separation (Bond et al., 2020). Rasch reliability statistics indicate the reproducibility of the person ordering or placement (Wright & Masters, 1982). That is, if the same group of respondents took an equivocal test, would they be placed in a similar order based on their measures? Boone et al. (2014) indicate that the person reliability statistics produced by Winsteps are interpreted similarly to traditional reliability indices. Therefore, using conventional guidelines, a person reliability index of 0.7 or higher is considered acceptable (DeVillis, 2017; Kaplan & Saccuzzo, 2017; MacRae & Furnham, 2016; Nunnally, 1978; Pallant, 2020; Yang & Green, 2011). Person separation, on the other hand, is the spread of the respondents on that measure (Bond et al., 2020). A higher separation statistic indicates a larger spread of person measures, with and index of 1.50 regarded as acceptable and 2.00 and 3.00 as good and excellent, respectively (Wright & Masters, 1982).

Unidimensionality and local independence are two interrelated conditions required for Rasch measurement (Fan & Bond, 2019; Heffernan et al., 2019). Unidimensionality refers to the measurement of a single construct (or latent trait or dimension). For example, the trait Extraversion can be considered a single construct. A scale measuring extraversion alone is therefore unidimensional. A scale that measures extraversion and anxiety, then, is multidimensional (measuring more than one dimension). In Rasch measurement, the requirement, then, is that each latent trait be measured one at a time (Fan & Bond, 2019) and is therefore unidimensional. Perfect unidimensionality, however, is not a realistic expectation. Instruments are then required to have a close approximation to unidimensionality. The unidimensionality of an instrument is estimated through a principal component analysis of the residuals (PCAR) (Fan & Bond, 2019), available in software such as Winsteps (Linacre, 2020b). From the PCAR, contrasts with eigenvalues at or greater than 2 indicate the possibility of the scale possessing more than one dimension, whereas contrasts with eigenvalues less than 2 are regarded as insignificant (Bond et al., 2020). Furthermore, items underfitting the Rasch model provide additional concerns to the threat of unidimensionality (Fan & Bond, 2019).

Local independence is the condition in which an individual’s responses to an item is not affected by their response to any other items (Fan & Bond, 2019). For example, an item regarding reading several times a week would affect, or be affected by, an item regarding reading once a week. In such a case, the items are dependent rather than independent and therefore violate the condition of local independence (Fan & Bond, 2019). However, like unidimensionality, it is unrealistic to expect perfect independence. An estimate of the correlation between item residuals is therefore required to determine whether there are items that are significantly dependent on each other (Fan & Bond, 2019; Linacre, 2020a). According to Linacre, positive correlations of 0.7 are the beginning of concern for dependency. Furthermore, items overfitting the Rasch model provide additional concerns to the threat of local independence (Fan & Bond, 2019).

Differential item functioning is an evaluation of how congruently the items of a measure define a construct between certain groups (Boone et al., 2014). In Winsteps (Linacre, 2020b), average measures of the relevant groups (e.g. male and female groups) are presented in logits and are compared against each other. Boone et al. (2014) indicate that DIF may be present in comparisons with a significant p-value (p < 0.05) of the Rasch–Welch statistic. Linacre (2020a) substantiates that in addition to statistical significance between groups, an effect size ≥ 0.64 is considered moderate to large, whereas between 0.43 and 0.64 is considered slight to moderate. Below 0.43 is considered negligible and insufficient to flag items as having DIF present. A DIF effect size in a Rasch analysis is provided in Winsteps as ‘DIF contrast’ (Linacre, 2020a).

Table 1 provides a breakdown of the sample. The sample consisted of 1257 respondents who completed the HPTi through the South African subsidiary of Thomas International and had agreed to participate in further research. Slightly more than half of the sample were female (n = 684, 54.4%) and the rest male (n = 573, 45.6%).

Regarding ethnic background, slightly less than half reported as white (n = 577, 45.9%). Less than a third reported as black (n = 380, 30.2%), followed by mixed race (n = 184, 14.6%), then Asian and Indian (n = 106, 8.4%). Ten (0.8%) respondents reported other.

When indicating their home language, 558 (44.4%) reported English and 370 (29.4%) indicated Afrikaans as their home language. IsiXhosa was the next most frequently reported home language with 76 (6.0%) respondents, followed by isiZulu (n = 59, 5.7%), Setswana (n = 49, 3.9%), Sesotho (n = 48, 3.8%), Sepedi (n = 47, 3.7%), Tshivenda (n = 20, 1.6%), Xitsonga (n = 16, 1.6%), siSwati (n = 9, 0.7%), French (n = 3, 0.2%) and isiNdebele (n = 2, 0.2%).

Nearly half of the respondents reported Gauteng (n = 586, 46.6%) as their residential province. The Western Cape (n = 313, 24.9%) was the next most frequently reported residential province, followed by the Free State (n = 123, 9.8%), KwaZulu-Natal (n = 112, 8.9%), the Eastern Cape (n = 80, 6.4%), Mpumalanga (n = 15, 1.2%), Limpopo (n = 13, 1.0%), North West (n = 10, 0.8%) and the Northern Cape (n = 5, 0.4%).

Table 2 contains the median, mode, oldest, youngest and range of birth years of the sample. The youngest respondent was born in 1999 and the oldest in 1945. The most commonly occurring year of birth was 1985.

Secondary data were obtained from Thomas International Ltd – the intellectual property right holder of the HPTi. The dataset includes the raw data, with scores from strongly disagree (1) to strongly agree (7) of 1257 individuals. The participants completed the HPTi for various purposes, including third-party recruitment and research conducted by Thomas International Ltd. Only data of respondents who completed the HPTi through the South African division of the organisation and had indicated their voluntary participation in further research were obtained. Negatively phrased items were reverse scored.

The primary data analyses were conducted in Winsteps (Linacre, 2020b) using the Rasch rating scale model. The descriptive statistics were constructed in Microsoft Excel (Microsoft Corporation, Redmond, Washington, United States) and Winsteps (Linacre, 2020b). Each of the six traits were analysed on person fit; descriptive statistics; item reliability and separation; item fit; unidimensionality and local dependence; and DIF.

The secondary data obtained from Thomas International Ltd (thomas.co) contained the anonymised responses of individuals who completed the HPTi and indicated their voluntary participation in further research. Respondents were presented with the opportunity to indicate their voluntary participation in further research after completing the HPTi. Ethical approval was obtained from the Psychology Research and Ethics Committee at the University of Pretoria (reference number: HUM037/0720).

The reliability indices ranged from adequate to inadequate (see Table 3). Curiosity obtained the highest reliability indices with a person reliability of 0.73 and separation of 1.62. Risk Approach (0.64, 1.33) had the lowest reliability indices, followed closely by Ambiguity Acceptance (0.65, 1.35), Conscientiousness (0.65, 1.37), then Competitiveness (0.69, 1.48) and Adjustment (0.69, 1.49).

Outfit and infit statistics were evaluated for the items of the HPTi traits, with precedence given to infit. The infit and outfit statistics of the items can be viewed in Table 4.

Conscientiousness contained two misfitting items: CN01 and CN10 underfit the model (IN.MNSQ = 1.42 and 2.02, IN.ZSTD = 5.32 and 9.90, respectively). Adjustment had one misfitting item, where AJ06 underfit the model (IN.MNSQ = 1.42, IN.ZSTD = 7.19). Risk Approach contained one underfitting item: RA13 (IN.MNSQ = 1.44, IN.ZSTD = 9.37). Ambiguity Acceptance had two misfitting items: AA01 (IN.MNSQ = 1.48, IN.ZSTD = 9.90) and AA13 (IN.MNSQ = 1.39, IN.ZSTD = 8.88) underfit the model.

The unidimensionality of each scale was examined through PCAR. The item loadings of the first contrast can be seen in Table 4 as ‘1c Load.’. The results revealed that Curiosity had the highest first contrast eigenvalue (λ = 2.24), followed by Ambiguity Acceptance (λ = 1.87), Adjustment (λ = 1.75), Conscientiousness (λ = 1.70), Risk Approach (λ = 1.70) and Competitiveness (λ = 1.61).

The largest standardised residual correlations were analysed to evaluate the local independence of the items of the scale. No item pairs were found to be above the correlation of 0.70, indicating that none of the items of the scales are in violation of local independence.

This study set out to evaluate the psychometric properties of the HPTi, a personality assessment. The psychometric properties were evaluated through Rasch analysis, namely person reliability and separation, fit to the Rasch model and the Rasch version of DIF.

The reliability indices of five of the six HPTi scales would not be considered reliable against the widely accepted minimum standards. The five scales are Adjustment (0.69), Competitiveness (0.69), Conscientiousness (0.65), Ambiguity Acceptance (0.65) and Risk Approach (0.64), of which Adjustment and Competitiveness bordered on the minimum value required to be regarded as being reliable. When evaluating other personality-based psychological assessments in the South African context, de Bruin et al.’s (2022) evaluation of the Basic Traits Inventory revealed reliability indices (Cronbach’s alpha) ranging from 0.87 (Openness) to 0.94 (Conscientiousness) in the adult sample. Similarly, the Myers–Briggs Type Indicator® (MBTI®; Myers et al., 1998) obtained high alphas ranging from 0.88 (both Sensing–Intuition and Thinking–Feeling) to 0.91 (both Extraversion–Introversion and Judging–Perceiving. The Rasch person reliability of the Judging–Perceiving dichotomy was 0.83, with the rest being 0.84 (Van Zyl & Taylor, 2012). The South African Personality Inventory (SAPI; Fetvadjiev et al., 2015) achieved mean alphas from 0.71 (Social Relation – Negative) to 0.81 (Social Relation – Positive), although subscales had alphas as low as 0.61 (Deceitfulness). When re-evaluating the SAPI, Morton et al. (2018) obtained alphas ranging from 0.61 (Neuroticism) to 0.88 (Social Relation – Positive). Hill et al. (2021) evaluated the psychometric properties of the Tshivenda and Southern Sotho versions of the SAPI. The Tshivenda version obtained mean alphas ranging from 0.61 (Extraversion) to 0.72 (Social Relation – Positive). The Southern Sotho version obtained alphas ranging from 0.50 (Extraversion) to 0.77 (Social Relation – Negative). Boshoff and Laher (2015) reviewed the utility of the NEO-PI-3 (McCrae et al., 2005) in the South African context. They found reliability coefficients ranging from 0.61 (Agreeableness) to 0.79 (Extraversion) for the domains of the NEO-PI-3. Thus, the reliability findings of the HPTi subscales are neither irregular nor the worst in the South African context but can certainly be improved upon. Person reliability, according to Linacre (2020a), is largely dependent on the dispersion of the characteristics of the sample – in other words, a sample with varying degrees of the trait being measured – the length of the instrument, the number of response options per item and the targeting of the sample and items. Some traits’ reliability indices could potentially be impacted by the inadequate targeting, evident in the large differences between the average person scores in traits Conscientiousness, Adjustment, Curiosity and Risk Approach and the constrained item measure average of zero. On the matter of targeting, Boone et al. (2014) recommend the revision of the difficulty of the items accordingly, making them either more or less difficult to endorse. On the other hand, the scales that appear well targeted – Ambiguity Acceptance and Competitiveness – may have their respective reliability indices impacted by the inability to adequately separate the higher scorers on the trait from those who have lower person measures of that trait, resulting in a lower person separation index and therefore reliability index. Given the results, it may be difficult to defend the reliability of the HPTi subscales in the South African context, with Curiosity being the most defensible.

Item fit statistics evaluated how well the items of each HPTi trait conformed to the Rasch model. Fit mean-squared statistics greater than 1.40 indicate that the item is 40% less predictable (more varied) than the model. The same statistic under 0.60 indicates that the item is 40% more predictable (less varied) than the model expects (Bond et al., 2020). Six of the 78 HPTi items (7.7%) underfit the Rasch model: Conscientiousness and Ambiguity Acceptance with two (15.4%) items each, Adjustment and Risk Approach with one (7.7%) item each, while Curiosity and Competitiveness had no items underfitting the model. Curiosity, however, had one item that bordered on the underfitting criterion of 1.40, item CU13. In contrast, the evaluation of the MBTI® Form M in the South African context had no items across all four dichotomous dimensions overfitting or underfitting the model at the criteria employed in this study (Van Zyl & Taylor, 2012). According to Tennant and Conaghan (2007), however, fit to the Rasch model can be influenced by item bias such as DIF.

The DIF is an evaluation of how congruently the items of a measure define a construct between certain groups (Boone, et al., 2014). It is especially important in cross-cultural settings (Tennant et al., 2004).

Across all scales, while statistically significant DIF was found between gender groups, the findings were not practically significant, as measured by DIF contrast (Linacre, 2020a). This suggests that the HPTi scales do not contain items with definitions that are interpreted differently between men and women. Therefore, none of the items of the HPTi could be considered biased towards either men or women. In contrast, Van Zyl and Taylor (2012) found 12 (13%) of the 93 items with a DIF contrast above 0.43 (slight to moderate effect; Linacre, 2020a) when evaluating DIF between gender, two (2%) of which were above 0.64 (moderate to large effect; Linacre, 2020a).

Apart from the Adjustment scale, DIF was discovered in several items in the ethnic groups and first-language groups comparisons. The severity varied across the scales. Between ethnic or racial groups, the largest and second largest exhibitions of DIF occurred in item CM02 in trait Competitiveness between the black group and white group (–0.85) and the Asian and Indian group and mixed-race group (-0.73). Most instances of DIF involved the black African group. Similarly, most instances of DIF between language groups involved the African languages group. The DIF between black and white ethnic groups was also found in Van Zyl and Taylor (2012) and between African languages and both English and Afrikaans in Grobler and De Beer’s (2015) evaluation of the Basic Traits Inventory. The findings of the item-level bias are not dissimilar to historic findings of personality questionnaires in the South African context, in which bias between African and European ethnic groups and language groups is usually found and recommended for further investigation (Abrahams & Mauer, 1999; Spence, 1982; Taylor & Boeyens, 1991). It may therefore be required that the relevant HPTi items be re-examined in the South African context to reduce the item-level bias.