The effect of social anxiety on threat acquisition and extinction: a systematic review and meta-analysis

Literature search and information sources

The papers included in this review were identified via three separate literature searches and a previously published meta-analysis. The first literature search (Search 1) was conducted in July 2017 to identify articles that aimed to investigate the relationship between anxiety (across all anxiety subtypes) and threat acquisition and extinction. All studies identified as part of this search that focused on social anxiety were retained for the present review. Search 1 excluded studies that recruited patient samples because we were able to extract studies that had investigated the relationship between social anxiety disorder (SAD) and threat acquisition and extinction from the previously published meta-analysis carried out by Duits et al. (2015). We then conducted the second literature search (Search 2) in December 2019 to capture any additional relevant articles, including those focusing on either social anxiety disorder or trait levels of social anxiety, published between 2013 (after the end date of the article search for the Duits et al. (2015) meta-analysis) and December 2019². Further, to account for the passing of time since Search 2, a third literature search was conducted to identify any relevant articles that had been published between December 2019 and January 2022. By combining these approaches, we were able to capture all papers examining threat acquisition and extinction in relation to either social anxiety disorder or social anxiety conceptualised continuously published up to and including January 2022.

Search 3 (January 2022)

Search 3 was identical to Search 2 but search terms were set to capture articles that had been published between December 2019 and January 2022, to update the search. After removing duplicates between databases, 168 abstracts were extracted and screened. Thirteen full-text articles were then reviewed, of which seven additional were included in the review (Fung & Alden, 2020; Fyer et al., 2020; Reichenberger, Pfaller & Mühlberger, 2020; Savage et al., 2020; Stegmann et al., 2020; Wake et al., 2021a; Wake, Van Reekum & Dodd, 2021b).

Across all searches, twenty-one articles were identified as relevant for inclusion in the current systematic review (see Fig. 1).

Inclusion and exclusion criteria/eligibility criteria

Abstracts and full-text articles were only included in the review if they met the following eligibility criteria: (1) the study must be written in English, (2) the study must be an original, peer-reviewed experiment (i.e., not a dissertation, case study or review article), (3) participants must be human, (4) classical aversive conditioning and extinction must be examined, (5) the study must recruit patients diagnosed with social anxiety disorder according to past or current DSM criteria (Search 2 and Search 3) or social anxiety must be measured using a standardised self-report measure of social anxiety (Search 1, Search 2, and Search 3), (6) the study must not incorporate a treatment or intervention, including medication and/or therapy, (7) there must not be a mood/emotion/stress/pain induction or manipulation, including training, at any point during the study, (8) Subjective (i.e., ratings of valence/anxiety/expectancy) and/or psychophysiological (i.e., startle/skin conductance/corrugator EMG/heart rate/pupil) outcome measurements must be used to index the conditioned fear response, and (9) the experiments must use a correlational or between-subjects design.

Outcome measures relating to brain imaging (i.e., functional magnetic resonance imaging/positron emission tomography) were considered to be beyond the scope of the current review.

Study selection

Study selection adhered to PRISMA guidelines (Liberati et al., 2009) (see Fig. 1). First, abstracts from all sources were screened against eligibility criteria (outlined above), followed by the full texts. The eligibility criteria for Search 1, Search 2, and Search 3 were identical, except for criterion 5 (see above). During the study selection of Search 1, articles were excluded if patients with a formal diagnosis of SAD had been recruited. Articles from Search 2 and Search 3 were not excluded based on the recruitment of patient populations. An article could be excluded at any stage of the screening process based on a ‘no’ response to any of the eligibility criteria; the first criterion that was not met was recorded as the reason for the rejection. Where criteria were coded as unclear (in the absence of any definite ’no’ codes) at the abstract stage, articles were put through to the full-text screening.

Piloting and inter-rater reliability

Eligibility criteria and search terms were piloted before Search 1 was conducted, and eligibility criteria were altered accordingly (criterion 6 and 7 were added). After completion of piloting, two coders (the first author, SW, and a postgraduate student, IH) checked the first 2,000 abstracts of Search 1 (33%). Based on these 2,000 abstracts, a high level of inter-rater reliability was found between coders for accept/reject decisions (K = .95, p < .001). The remaining abstracts were coded by SW.

To ensure the reliability of the criteria for the full-text screening, SW and a second postgraduate student (MA) both assessed the first 249 full-texts (55%) against the eligibility criteria. Again, there was good inter-rater reliability (k = .69, p < .001) between coders for accept/reject decisions on the basis of these 249 full texts. SW coded the remaining articles.

Any disagreements between coders at either stage of the screening were discussed to reach a consensus.

SW coded all abstracts and full texts for Search 2 and Search 3 given that reliability had already been established in the context of Search 1.

Risk of bias in individual experiments

The Standard Quality Assessment Criteria for Evaluating Primary Research Papers was used to indicate the overall quality of all the included studies (Kmet, Cook & Lee, 2004) (Table S1). Studies were scored on fourteen items on a checklist that assessed study design, methodology, accurate reporting of results, and conclusions made. Depending on the extent to which they met the criteria (2 = yes, 1 = partly, 0 = not addressed), a summary score was calculated for each study. Kmet, Cook & Lee (2004) suggest that a relatively conservative cut off point for article inclusion is 75%. One of the selected papers (Veit et al., 2002) had a score of 64% and therefore, based on this criterion, was excluded from the current review, leaving a total of twenty-one research articles. Otherwise, the studies included were of high quality, with scores ranging from 77%–95%. Only articles that fulfilled the criteria of the risk of bias assessment are referred to in the remainder of the review (i.e., n = twenty-one published articles).

Data extraction

Once all of the full texts had been screened, the first author extracted all of the relevant data, including (1) sample characteristics, (2) methodology, (3) statistical analyses conducted, and (4) the studies outcome measures that were relevant to the aims of the review. The above information was used to conduct a qualitative systematic review of the twenty-three included experiments in relation to the aims of the study.

To calculate the effect sizes included in the meta-analyses, the first author extracted the means and standard deviations of responses towards the CS+ and the CS- separately for each group of participants (i.e., individuals with higher social anxiety and individuals with lower social anxiety). Where available, means and standard deviations were extracted from the text, tables, or figures (using WebPlotDigitizer software; Drevon, Fursa & Malcolm (2017)) included in the individual papers.

Means and standard deviations were extracted from fourteen of the twenty-three experiments during the acquisition phase and seven of the twenty-three experiments during the extinction phase. We were unable to extract the required information to calculate effect sizes from the remaining studies, either because the means and standard deviations of the effects of interest were not reported, or because the other reported statistics (F, p or t values) did not allow a straightforward conversion to an effect size due to complex, or unclearly reported model specifications, or lacked descriptive statistics to interpret the direction of the effect. As such, fourteen experiments were included in meta-analyses of acquisition effects and seven experiments are included in meta-analyses of extinction effects (See Table 1).

The primary effect size index used to quantify effects during threat acquisition and extinction was Cohen’s d, i.e., the difference between responses towards the CS+ versus the CS- between two groups (i.e., individuals with higher social anxiety and individuals with lower social anxiety). Initially, a Cohen’s d was calculated for the within-subject difference between responses towards the CS+ and CS- (i.e., for individuals with higher social anxiety and individuals with lower social anxiety separately) using the following Cohen’s d calculation for within-subject designs that uses the mean difference as the numerator average and standard deviation as the denominator (Cumming, 2012; Lakens, 2013). The overall effect size (formula X) was thus calculated by subtracting the Cohen’s d for the difference between responses towards the CS+ versus the CS- for individuals with lower social anxiety from the Cohen’s d for the difference between responses towards the CS+ versus the CS- for individuals with higher social anxiety. A positive overall value of Cohen’s d represents an increased conditioned response for individuals with higher social anxiety relative to lower social anxiety.

${\displaystyle d=\frac{M_{CSdiffHA}}{\frac{S{D}_1+S{D}_2}{2}}-\frac{M_{CSdiffLA}}{\frac{S{D}_1+S{D}_2}{2}}}$

Formula X

Meta-analyses model

Effect size outcomes were modelled with a random effects model due to its tolerance of heterogeneous effect sizes and conservative nature of estimation (Schmidt, Oh & Hayes, 2009). Heterogeneity across effect sizes was measured by Cochran’s Q statistic. Unless reported otherwise, parameter estimates were obtained via restricted maximum likelihood estimation, owing to its superior accuracy given a smaller number of studies (López-López et al., 2014).

Many samples contributing to the analysis produced multiple effects, entailing a nested structure. We modelled this by incorporating a hierarchical model specification that nested effects within their corresponding samples, thereby estimating random effects at both the effect and sample level. We thus used the following random effects structure ‘∼1 — sample_id/effect_id’. All models were fit using the ‘rma.uv’ function in the R package ‘metafor’ (Viechtbauer, 2010).

Moderator analyses

To model heterogeneity in effect sizes, we proceeded in two ways. We first fit single-moderator models to examine the effect of each moderator in isolation. This phase allowed us to determine a subset of potentially explanatory variables for further exploration. Statistical tests of model coefficients were computed Wald-type chi squared statistics.

To test multiple combinations of moderators, we employed automated model selection and fit all combinations of models up to and including two-way interactions. These models were compared and ranked the basis of Akaike’s information criterion (AIC). This information-theoretic approach (Burnham & Anderson, 2002) allowed us to determine the ‘importance’ of each coefficient on the basis of their summed Akaike weights across the population of models. This approach, implemented in the ‘MuMin’ R package (Bartoń, 2020), depends on repeated evaluations of a ‘full model’ formula that includes all moderators. If such a model could not be specified on account of being underdetermined (too many coefficients), we used the single moderator phase to remove the most uninformative coefficients based on their p value.

Sample characteristics

The twenty-one articles identified and included in this review were published between 1999 and 2022. The majority of experiments (n = 9) contained German samples, while the remaining experiments included samples from the USA (n = 6), the UK (n = 3), Australia (n = 1), Canada (n = 1), Spain (n = 1) and The Netherlands (n = 1) (see Table 2). Fourteen experiments used a self-report measure to quantify levels of social anxiety, including the Liebowitz Social anxiety scale (n = 2; Blechert et al., 2015; Tinoco-González et al., 2015), a pre-screening questionnaire based on DSM-IV criteria (n = 1; Ahrens et al., 2014), the Brief Fear of Negative Evaluation Scale (n = 1; Ly & Roelofs, 2009), a maternal report of Social Reticence (n = 1; Michalska et al., 2019), the Social Interaction Anxiety Scale (n = 1; Fung & Alden, 2020), the German version of the Social Phobia and Anxiety Inventory (n = 1; Stegmann et al., 2020), the Rejection Sensitivity Questionnaire (n = 1; Olsson et al., 2013), and the Social Phobia Inventory (n = 6; Pejic et al., 2013; Reichenberger et al., 2017; Reichenberger, Pfaller & Mühlberger, 2020; Shiban et al., 2015; Wake et al., 2021a; Wake, Van Reekum & Dodd, 2021b). The remaining experiments (n = 8) included samples of adults diagnosed with SAD and healthy control participants (see Table 2). Tinoco-González et al. (2015) reported two separate experiments within their paper, one of which recruited a clinical sample of patients with SAD and the other measured social anxiety using the Liebowitz Social Anxiety Scale. Wake et al. (2021a) also included two separate experiments in their paper. Only one experiment was included from all other articles, so a total of twenty-three separate experiments from twenty-one research articles are referred to in this review. All experiments, apart from Michalska et al. (2019), recruited adult samples.

Systematic review

Please see Tables 1, 2, and 3 for specific details of the sample characteristics, outcome measures and experimental parameters employed across the twenty-three experiments.

Dependent variables

Psychophysiology

Subjective ratings

Experimental parameters

Meta-analyses

Threat acquisition effects

Acquisition effects were taken from 14 studies. These studies contributed 14 samples, which comprised of 454 healthy controls and 399 participants with social anxiety. Ten of these samples contributed multiple effects, entailing 41 effects in total.

Global outcomes.

Figure 2A depicts a forest plot of the data and random-effects model outcomes. The random-effects model was not significant and estimated a very small, negative effect d = −.009 [−.212,.195], p = .934, AIC = 71.62. Expressed differently, the probability of superiority, i.e., that a person with social anxiety picked at random would demonstrate an increased conditioned response compared to a person picked at random from the HC group, was estimated to be 50% (i.e., chance level). The test for heterogeneity was significant Q(40) = 123.76, p < .001. According to Cook’s distances, only one strongly negative effect (which is clearly visible on the left of the forest plot, see Fig. 2A) was determined as being particularly influential (taken from Hermann et al., 2002). Further inspection revealed no obvious coding errors and so this study was retained in the analysis.

Publication bias.

A funnel plot of effect sizes can be seen in Fig. 2B. Evidence of publication bias was limited: rank-sign test revealed no evidence of a relationship between sampling variances and effect magnitudes (T = .044, p = .696).

Moderators.

No effect of experimental design was detected Q(1) = 1.2948, p = .255, there was no difference between studies that examined trait-level of social anxiety (d =  − 0.112, [−0.384, 0.160], p =.421) and clinical group designs (d = 0.13, [−0.185, 0.444], p =.419). No effect of DV was detected Q(9) = 5.89, p = 0.751. Of the DVs, only US expectancy elicited a detectable effect overall and this was in the negative direction (d =  − 0.593, [−1.176, −0.011], p = .046).

No effect of CS type was detected (Q(1) = 0.276, p = .599), there was no difference in effect magnitude between studies employing a social CS (d = 0.017, [−0.216, 0.251], p = .884) and non-social CS (d =  − 0.133, [−0.643, 0.376], p = .609). Similarly, no effect of US type was detected (Q(2) = 2.701, p = .259), there was no difference in effect magnitude between studies employing a social US (d = 0.090, [−0.201, 0.381], p = .546) and non-social US (d =  − 0.300, [−0.703, 0.103], p = .144) or both (d = .094, [−0.309, 0.496], p = .648). No effect of reinforcement schedule was detected Q(1) = 0.840, β = 0.004, [−0.005, 0.012], p = .359

Moderator importance.

A ‘full model’ with all moderators could not be fit to the data. We therefore removed CS type from the analysis on the basis of its high p value. We additionally binarised the DV factor into a moderator that coded whether the DV was subjective or physiological to reduce the number of coefficients to be estimated.

The best model, according to AICc, included reinforcement schedule, US type and their interaction AICc(7) = 76.51. However, the model-averaged importance of each moderator value indicated none that exceeded 0.8, which is often used as a cutoff to differentiate between important and unimportant variables. Given this, and also considering that the second-ranking model included only the intercept, we opted to be conservative and retain this empty (no moderator) model as the final model (AICc(3) = 78.17). The model-averaged coefficients are shown in Table S2.

Threat extinction effects

Threat extinction effects were taken from seven studies. These studies contributed seven samples, which comprised of 231 healthy control subjects and 193 participants with SAD. Five of these samples contributed multiple effects, entailing 23 effects in total.

Global outcomes.

Figure 2D depicts a forest plot of the data and random-effects model outcomes. The random-effects model estimated a very small, positive effect d = 0.047 [−.131, .224], p = .607, AIC = 33.27. The probability of superiority, i.e., that a person with social anxiety picked at random would demonstrate compromised threat extinction compared a healthy control subject, was estimated to be 51%. The test for heterogeneity was significant Q(22) =49.90, p < .001. Cook’s distances did not indicate any influential effects indicative of outliers.

Publication bias.

A funnel plot of effect sizes can be seen in Fig. 2E. Evidence of publication bias was limited: rank-sign test revealed no evidence of a relationship between sampling variances and effect magnitudes (T =.044, p = .696).

Moderators.

No effect of experimental design was detected Q(1) = 1.036, p = 0.309, there was no difference between studies that examined trait-levels of SA (d =  − 0.268, [−0.157, 0.692], p = 0.216) and clinical group designs (d =  − 0.140, [−0.352, 0.324], p = 0.935). An effect of DV was detected Q(8) = 18.724, p = 0.016. Of the DVs, anxiety ratings produced the largest detectable effects in the positive direction d = 0.987, [0.332, 1.642], p = 0.003. Holm-corrected post-hoc tests indicated significantly larger effects with anxiety ratings as the DV relative to US Expectancy t(14) =4.07, p = 0.041.

No effect of CS type was detected (Q(1) = 1.639, p = 0.201), there was no difference in effect magnitude between studies employing a social CS (d =  − 0.185, [−0.669, 0.298], p = 0.452) and non-social CS (d =  − 0.181, [−0.102, 0.464], p = 0.211). An effect of US type was detected (Q(2) = 6.648, p = 0.036). Holm-corrected post-hoc tests revealed that this was mostly driven by social US stimuli (Q(2) = 0.487, [0.116, 0.859], p = 0.010) having larger effect sizes than both non-social stimuli β = 0.558, t(20) = 2.538, p = 0.059 and combinations of both social and non-social stimuli β = 0.497, t(20) = 2.047, p = 0.108. An effect of reinforcement schedule was also detected, such that more reinforcement led to larger effects Q(1) = 0.129, β = 0.013, [0.003, 0.023], p = .0128

Model selection and moderator importance.

A ‘full model’ with all moderators could not be fit to the data. We therefore removed CS type from the analysis on the basis of its high p value. The model-averaged importances of each moderator are depicted in Fig. 2F. The best model, according to AICc, included reinforcement schedule as the sole moderator AICc(4) = 33.506. This model is depicted in Fig. 2G. The model-averaged coefficients are shown in Table S3.

Registration and Protocol

This review was not registered and a protocol was not registered on PROSPERO.