Psychometric properties of the Perceived Stress Scale (PSS): measurement invariance between athletes and non-athletes and construct validity

Introduction

Since the development of the Perceived Stress Scale (PSS, Cohen, Kamarack & Mermelstein, 1983) it has been widely used in various research such as the degree of global stress of a given situation (Leon et al., 2007; McAlonan et al., 2007), or effectiveness of an intervention on psychological stress (Holzel et al., 2010; Seskevich & Pieper, 2007; Taylor-Piliae et al., 2006), or the associations of perceived stress and psychiatric/physical disorder (Culhane et al., 2001; Garg et al., 2001). In addition, many studies used PSS to examine its relationship with quality of life (Golden-Kreutz et al., 2004; Golden-Kreutz et al., 2005), job satisfaction (Norvell et al., 1993), immune functioning (Burns et al., 2002; Maes & Van Bockstaele, 1999), depression (Carpenter et al., 2004), and sleep quality (Cohen & Williams, 1988). Therefore, it can be said PSS is a very important tool in assessing stress.

Built on Lazarus & Flokman’s (1984) transactional model of stress, the development of PSS is to assess one’s perceived nonspecific stress in a given situation or a daily life situation. Lazarus & Flokman’s (1984) transactional model of stress contends that an individual’s stress perception derived from the imbalance between one’s appraisal of situational demands and coping resources. If one perceives that situational demands over resources and the consequences of such failure will be severe; then, it will lead to psychophysiological responses such as fast heartbeats, pale face, cold and sweaty hands, tense muscles, and so on. Cohen and colleagues (1983); Cohen & Williams, 1988) constructed this global stress perception by two important components—something that one can control (i.e., counter stress) and something that one can’t control (i.e., perceived stress). In such manner, PSS is not only a measure to assess an extant of how a given situation might hurt oneself but also to assess the degree of how this give situation is controllable or uncontrollable (Golden-Kreutz et al., 2004; Örücü & Demir, 2009; Roberti, Harrington & Storch, 2006).

Given that the applicability of PSS in assessing perceived stress in many domains, the number of the item, the factorial structure, and the reliability of PSS have been intensively examined by many researchers. For example, although the initial version of PSS (Cohen, Kamarack & Mermelstein, 1983) has been developed as 14-item with a unidimensional measure, Cohen and colleagues (1988) continuingly examined the appropriateness of the item’s number and factorial structure of PSS. They sampled 960 male and 1,427 female US residents (M_age = 42.8 ± 17.2 years) and examined its factorial structure, criterion validity, and internal consistency of the PSS. Results found 10-item, by deleting item 4, 5, 12, and 13, 2-factor PSS-10 can be a better measuring tool of perceived stress because they found the revised version of PSS-10 accounted 48.9% of the variance, and had better reliabilities (Cronbach’s α = .84 ∼.86), and correlated with anxiety, depression and life events which indicated good construct validity.

Followed Cohen and colleagues (1988), Hewitt and colleagues (1992) sampled psychiatry patient as participants and examined factorial structure and reliability of PSS, the exploratory factor analysis (EFA) found PSS had two factors—perceived distress (Cronbach’s α = .81) and perceived coping (Cronbach’s α = .72) with 11 items. Similar findings also found in a Mexico sample where Ramírez & Hernández (2007) found two factors PSS-10 had better reliability and accounted 48.02% of the variance. Recently, Barbosa-Leiker and colleagues (2013) examined measurement invariance of PSS across gender and time with a clinical sample. Results indicated the 2-factor 10-PSS model provided acceptable fit in both men and women at each time point.

Based on above literature, we understand that earlier version of one-factor 14-PSS (Cohen, Kamarack & Mermelstein, 1983) was designed to measure a global perceived stress without considering the dimensionality. However, when researchers continuingly examined the psychometric properties of PSS (e.g., Barbosa-Leiker et al., 2013; Cohen & Williams, 1988; Hewitt, Flett & Mosher, 1992; Ramírez & Hernández, 2007) it has found that the items, the factorial structure and the reliability of PSS need further examination. Especially, researchers concerned about whether 2-factor 10-PSS or 14-PSS can be an ideal tool in assessing perceived stress. Further, since PSS was to assess individuals perceive normal stress in a given situation or a daily life situation, however, the stress of an athlete on the training and competing situation is different from non-athletes. We doubt whether PSS is suitable to be used in sports and the measurement is invariant comparing to non-athletes. Therefore, it is imperative to examine the psychometric properties of PSS in sports and compare the measurement invariance between athletes and non-athletes.

In sports, stress is an important issue that has been received much of attention; specifically in the studies of athletic burnout and athletic injury. For example, Smith (1986) proposed a cognitive-affective model of athletic burnout to explain the influences of stress on burnout. Smith (1986) explains that in the stressful sports settings, athletes keep appraising contextual stressors and personal coping resources. If athletes perceived situational demands surpass personal resources and consequences will be severe, the cognitive appraisal leads to severe physiological and psychological responses—anxiety, tension, insomnia, and illness, which eventually lead to burnout. With the same line of conceptualization, Andersen & Williams (1988) also proposed a ‘stress-athletic injury model’ which contends that athletic injury is the interaction between personality, history of stressors, coping resources and cognitive appraisal. In the stress appraisal process, athletes’ perceived stress is influenced by above mentioned factors such as personality, history of stressors, and coping resources. The consequences of this interaction can lead to either attenuate or deteriorate the perceived stress and eventually cause athletic injury.

Many researchers borrow either stress-burnout or stress-injury concepts to examine the role of perceived stress on athletes’ burnout and injury. Researchers use different measures to do their studies. For example, Rushall (1990) used Daily Analysis of Life Demands for Athletes (DALDA) to measure athletes’ perceived environment stimuli and overtraining. Petrie (1992) used the Life Event Scale for Collegiate Athletes (LESCA) to measure athlete’s life stressors and its relations to sports injury. Lu and his colleagues (2016) used College Student-Athlete Life Stress Scale (CSALSS) to examine the relationships between coaches’ social support, athletes’ resilience and burnout. The general problem of these measures is they are only used for assessing life stressors and examining its’ relationship with athlete burnout or sports injury. Although some report adequate psychometric properties (e.g., Lu et al., 2012), but some have been questioned regarding its reliability and validity (e.g., DALDA, LESCA). If they are used for assessing global perceived stress they are not appropriate.

To assess global perceived stress in sports, some researchers used PSS in examining its’ relationship with sports injury and burnout. For example, in examining whether stress and affect as the mediator of hope-burnout relationship, Gustafsson and colleagues (2013) administered 238 Swedish soccer players with trait hope, Swedish version of PSS (i.e., PSS-10), positive and negative affect and athlete burnout scales. Results found athletes’ hope and burnout were fully mediated by stress and positive affect. Similarly, Tashman, Tenenbaum & Eklund (2010) sampled 177 college coaches and examined the relationship between coaches’ perfectionism and burnout. Results indicated that perceived stress (measured by PSS-14) mediated the relationship between self-evaluative perfectionism and burnout, and a significant direct link to burnout, accounting for 56% of its variance. Similar sport burnout studies that using either PSS-10 or PSS-14 can be found in Raedeke & Smith (2004), Smith, Gustafsson & Hassmeín (2010), and Gustafsson & Skoog (2012) studies.

In terms of stress- athletic injury relationship, Galambos and colleagues (2005) investigated 845 Australian youth athletes’ injury rates and its relations with psychological variables. Participants (males = 433; females = 412) completed a demographic questionnaire, health history, Brunel Mood Scale and PSS-10. Results found mood and stress collectively predicted injury characteristics. In a similar study, Malinauskas (2010) sampled 123 college athletes and administered with social support scale, PSS-10, and life satisfaction scale. Results found greater perceived stress was associated with diminished life satisfaction for major injury athletes than minor injury athletes. Also, the interaction between perceived stress and perceived social support was associated the most with diminished life satisfaction for athletes with a major injury.

As previously mentioned, although sports researchers used either PSS-10 or PSS-14 in examining their relationships with athletic burnout or injury, the psychometric properties of PSS have never been examined in sports. Thus, there are several questions remained. First, whether two-factor or unidimensional PSS will be suitable for sports? Second, if we compare athletes and non-athletes will measurement characteristics remains the same? Third, what is the reliability and validity of PSS in sports? Therefore, the existing knowledge gap of the application of PSS in sports is that we don’t know which type of PSS is suitable in terms of dimensionality, construct validity, reliability, and group comparisons between athletes and non-athletes. To fill the gap, there are three purposes in this study. First, we intended to examine the factorial structure of the PSS-10 and PSS-14 and internal consistency. Second, we intended to examine the measurement invariance of PSS between athletes and non-athletes. Third, we attempted to examine the construct validity of PSS and test-retest reliability of PSS.

Study 1

The purpose of study1 was threefold: (a) to compare the factorial structure of the PSS-10 and PSS-14; (b) to examine internal consistency of PSS-10 and PSS-14; (c) to examine measurement invariance of PSS between athletes and non-athletes.

Study 2

The purpose of study 2 was to examine the construct validity of 2- factor PSS-10, which is the convergent and discriminant validity, via correlational analyses surrounding the relationships among PSS-stress, PSS counter-stress, college student-athletes’ life stress, coping self-efficacy and burnout experiences.

Results

The results of factorial structure suggest that 2-factor PSS-10 has better measuring quality than unidimensional PSS-10 or PSS-14. The two factors measurement model of the PSS-10 indicated a good fit of the instrument according to the fit indices in study2 (RMSEA = .056, SRMR = .061, χ²/DF = 1.607, CFI = .960, NNFI = .903, GFI = .955). The factor loadings for the 10 items range from .33 to .87 (Fig. 1). Cronbach’s α coefficients of stress was .77, the coefficient for the “counter-stress” was .68. The composite reliability (Fornell & Larcker, 1981) for each subscale was calculated: stress (.78), and counter-stress (.73) indicating that each was above the .70 standard. The average variance extracted was also calculated: stress (.59), and counter-stress (.52), which were all above the acceptable standard (.50). As for concurrent and discriminant validity, the Pearson correlations indicated that PSS-stress negatively correlated with CSE total scores and CSE subscales, and positively related to ABQ total scores, ABQ subscales, CASLSS total scores, and CASLSS subscales. Moreover, PSS-counter stress negatively or not correlated with ABQ total scores, ABQ subscales, CASLSS total scores, and CASLSS subscales, in contrast, positively related with CSE (see Table 3).

Study 3

Previous studies had found that the retest reliability was a better index of the reliability than the internal consistency coefficients (McCrae, 2014; McCrae et al., 2011). Therefore, the purpose of study 3 was to examine the test and retest reliability of 2- factor PSS-10.

Discussion

In line with past research examining the validity of PSS the purpose of this study was to examine psychometric properties of PSS in sports settings. Specifically, this study attempted to examine factorial structure, measurement invariance, reliability, and construct validity of PSS. By three studies we found 2-factor PSS-10 has a better fit of the model. Also, we found 2-factor PSS-10 had appropriate internal consistency, testretest reliability and measurement invariance across athletes and non-athletes. Further, we found 2-factor PSS-10 positively correlated with athletes’ life stress and burnout but negatively correlated with coping self-efficacy which indicated appropriate construct validity.

Therefore, the psychometric properties of PSS gain solid supports in the sports contexts. The results of factorial structure suggest that 2-factor PSS-10 has better measuring quality than 2-factor PSS-14, or 1-factor PSS-10/PSS-14. The 2-factor PSS-10 reduces four items allows researchers collect data in a short period of time (Shacham, 1983). Although researchers have different arguments regarding dimensionality of PSS (Hewitt, Flett & Mosher, 1992; Mitchell, Crane & Kim, 2008; Örücü & Demir, 2009) it is for sure that 2-factor PSS-10 receive better support in sports contexts. The results are consistent with the earlier study by Hewitt and colleagues (1992). Past research in sports generally just assessed life stressors (e.g., CSALSS, DALDA, LESCA) or arbitrarily used PSS-14/PSS-10 in examining the relationship between stress and related psychological responses (e.g., Raedeke & Smith, 2004; Smith, Gustafsson & Hassmeín, 2010; Gustafsson & Skoog, 2012). They did not exactly understand the psychometric properties of PSS. Our results provide robust evidences that 2-factor PSS-10 can be an appropriate tool in the sports settings. Thus, researchers not only have a better tool in examining the relationship between perceived stress and related psychological responses but also practitioners can use 2-factor 10-PSS in evaluating athletes’ existing perceived stress. Also, both researchers and practitioners can use two factors—perceived stress and counter stress to further understand what factor plays an important role on athletes’ psychological responses. We suggest future researchers may use 2-factor PSS-10 to examine gender differences, or to compare which variable predict related psychological disorders/constructs the most.

In terms of measurement invariance, we found configuration invariance, metric invariance, and variance/covariance invariance are all equivalent except error variance invariance. Therefore, it means that the same level of measurement error for each item between athletes and non-athletes is not the same. However, Lee (2006) suggested that most research that using CFA focus on the equivalence of the factor loadings and factorial covariance. If these indicators meet criteria they can assure that means measurement invariance across observed groups is held, while residual restrain model may be too critical to be reached. Tabachnick & Fidell (2001) also suggest that when factor loadings and factorial covariance are equivalent across group it is indicated that measurement invariance holds true.

A significant feature of this study is the sample recruited from Taiwanese student-athletes. Although PSS has been validated in different culture such as Spanish (Remor, 2006), Swedish (Eskin & Parr, 1996), Japanese (Mimura & Griffiths, 2004), Portugal (Ramírez & Hernández, 2007), Turkish (Örücü & Demir, 2009), or Chinese (Leung, Lam & Chan, 2010), this is the first study using healthy young athletes as participants in examining psychometric properties of PSS and its relationship with athlete burnout, coping self-efficacy and life stress. Our study not only confirms Leung and colleagues’ (2010) psychometric properties of the Chinese version of PSS which indicated that 2-factor 14-PSS, 10-PSS and 4-PSS have better measurement quality but also extends the applicability of PSS into sports contexts. The results of psychometric validation and measurement invariance can be forwarded to those researchers who interested in psychometric properties of PSS. Further, our construct validity analyses found 2-factor PSS-10 correlated with athlete burnout and life stress, but negatively correlated with coping self-efficacy, are worthy of forwarding these messages to sports professionals. As we all know the young athletes face many challenges in their life (Lewis, 1991). They engage in heavy and intensive training/competition all year round. The intensive and heavy training bring lots of stress for young athletes (Weinberg & Gould, 2015). If coaches and sports professionals fail to monitor athletes’ training loading and arrange them with appropriate competition plans they may induce lots of stress and cause burnout. Therefore, teaching young athletes effective coping skills (e.g., fostering social support and time management) and teaching psychological skills (e.g., goal-setting, relaxation, and imagery) are very important because they can help young athletes to cope with stress from training and competitions.

Further, as previously mentioned that most research in sports either using Smith (1986) cognitive-affective model of athletic burnout or Andersen & Williams (1988) stress-athletic injury model to examine the effects of stress on athlete burnout/injury, we suggest future research may extend research beyond this area. Especially, we suggest that future research may examine the antecedents of perceived stress in sports. For example, sports literature suggests that motivational climate created by coaches and teams (e.g., ego-involving climate) is one of the major sources of athletes’ stress (Hogue et al., 2013). Researchers may use 2-factor PSS-10 in examining how a sports team’s motivational climate predicts athletes’ perceived stress. In addition, it is well-documented that the leadership style implemented by coaches (e.g., autocratic) may produce stress for athletes (Horn et al., 2011). We suggest researchers may examine how coaches’ leadership and situational conditions (e.g., competition performance) predicting athletes’ perceived stress. Further, since the athletic world is a challenging setting it is found that some athletes become substance abuse and eating disorder to cope stress (Ansel, 2010). We suggest researchers may use 2-factor PSS-10 as a measuring tool in assessing athletes’ perceived stress during off- season, pre-season, and after-season in order to understand their stress level and provided with appropriate interventions.

As to the application of 2-factor PSS-10 in other domains, we suggest it can be a useful tool in assessing perceived stress in an intervention study. For example, if researchers want to know whether a stress management program reducing participants’ perceived stress (e.g., Shapiro et al., 2005). In such condition, researchers can use 2-factor PSS-10 before and after intervention so to understand the effectiveness of the intervention. Also, many researchers adopt a psychophysiological approach to examined participants’ electroencephalography (EEG) during stressful conditions (e.g., Cavanagh & Shackman, 2015; Grupe & Nitschke, 2013; Proudfit, Inzlicht & Mennin, 2013). In such case, if researchers use 2-factor PSS-10 combined with EEG it can be a sound approach to detect participants’ real psychological reactions. Further, many researchers in medicine intend to examine whether there is an association between perceived stress and coping strategies (e.g., Najam & Aslam, 2010). For this type of study 2-factor, PSS-10 can be a quick and efficient tool for collecting data.

There are several limitations that need to be addressed. First, our sample is all recruited from Division I college student-athletes whether our results could be generalized to other athletes, such as professional athletes or junior athletes, need to be further examined. Additionally, the data was collected from Taiwanese student-athletes; hence, the results may not be generalizable to different cultures. We recommend researchers adopt similar approaches to test measurement invariance in different cultures and populations.

Conclusion

To acknowledge that PSS is a widely used measure in assessing stress, we have conducted three studies to examine the factor structure, measurement invariance between athletes and non-athletes, internal reliability, test-retest reliability and construct validity in the sports contexts. Results indicated that 2-factor PSS-10 can be an ideal measure for the research in sports. We suggest future research may use 2-factor PSS-10 in conducting various stress research.

Supplemental Information