Stage 2 registered report: investigating a preference for certainty in conversation among autistic adults

Intolerance of uncertainty

A “preference for certainty and explicit communication” may link to the widely-researched cognitive trait, Intolerance of Uncertainty (IU). IU has been defined as a tendency to negatively evaluate uncertain situations and information (Shihata et al., 2016). We use the term “Intolerance of Uncertainty” in line with previous research and intend to convey a value-neutral meaning in using it, as we recognise that high levels of IU may be an understandable, even adaptive, response where individuals have experienced mishaps in confusing situations. IU has mostly been investigated as a transdiagnostic construct that plays a central role in emotional disorders across the general population (see Shihata et al., 2016 for a review), but it also seems especially relevant to autism, with autistic children and adults showing significantly elevated levels of the trait compared to the general population (e.g.,  Hwang et al., 2020; Vasa et al., 2018). IU has been closely linked to anxiety in autistic people (Jenkinson, Milne & Thompson, 2020), and also relates to core features of autism, including social difficulties, sensory sensitivities, insistence on sameness and repetitive behaviours (e.g.,  Hwang et al., 2020; Vasa et al., 2018; Wigham et al., 2015).

A possible link between IU and communication in autistic people remains largely unexplored. However, there are reasons to believe that a link is plausible. First, inferential models of communication, such as Relevance Theory (Sperber & Wilson, 1986), propose that communication inherently involves uncertainty. Under Relevance Theory, language comprehension is not simply a process of “understanding what the words mean”, as there are often indeterminacies and ambiguities in uses of language; instead, the words are used as evidence by the listener in supporting a hypothesis about what the speaker probably means in the context, i.e., inferring intended meaning under uncertainty. Relevance Theory suggests that there is a gradient of uncertainty in communication. Sometimes, the listener can rely mostly on the explicit content of the utterance to compute the intended meaning, but in other situations there is a greater reliance on inferential processing to understand the speaker’s probable meaning by integrating the utterance with contextual cues and world knowledge. Compare for instance the utterances “No, let’s stay inside” and “It’s quite cold today” as responses to a suggestion to go outside. In the second example, the speaker communicates implicitly, leaving the listener to process the implicature that they would probably prefer to stay inside. In a previous study, we provided evidence for cognitive differences between autistic and non-autistic people in processing implicatures (Wilson & Bishop, 2020c). Crucially, it seemed that a cognitive preference for certainty and explicit communication, and not simply reduced ability, may account for some of the differences. Participants completed the Implicature Comprehension Test, which required individuals to listen to short conversational interchanges that are followed by a comprehension question to assess whether an implied meaning has been processed; test-takers responded with “yes”, “no” or “don’t know”. Controlling for grammar/vocabulary ability, we found that autistic adults (N = 66) were 6.19, 95% CI [3.63–10.39], times more likely to select the “don’t know” rather than the correct response compared to non-autistic people (N = 118), and also 2.56, 95% CI [1.76–3.77], times more likely to choose the “incorrect” response (Wilson & Bishop, 2020c). Group differences were large, and performance on the test gave 76% sensitivity and specificity for differentiating between autistic and non-autistic groups. On the face of it, these results suggest that autistic people have difficulties inferring the gist of a speaker’s meaning, as predicted by the ‘central coherence’ theory, which proposes that autistic people may show less tendency than non-autistic people to process information at a global level (Frith, 1989).

However, in an alternative version of the test without a “don’t know” response, autistic individuals showed high accuracy for items for which they had selected “don’t know” first time round. This marked tendency to select “don’t know” when given a chance, but to process the inference as intended when constrained by the task, suggested reduced confidence in the face of uncertain information and a preference for explicit communication. This could be due to possible difficulties around metacognition in autistic people, who may experience a mismatch between performance and confidence in their performance due to differences in self-monitoring. There is evidence that autistic people may show such a mismatch (Grainger, Williams & Lind, 2016; Nicholson et al., 2019) although there is some concern about the replicability of these results (Maras, Norris & Brewer, 2020). An alternative view would be that it is less an issue of metacognitive “ability”, and more about differences in personality/cognitive preference, with the well-replicated elevated levels of IU in autistic people (e.g.,  Hwang et al., 2020; Vasa et al., 2018) accounting for this apparent preference for explicit communication observed in our previous study. In the present study, we aim to replicate this finding with more refined methods. In an adapted version of the Implicature Comprehension Test, individuals will respond using a 4-point scale of “yes”, “maybe yes”, “maybe no”, and “no”, allowing us to capture accuracy and confidence in the same measure. We hypothesise that confidence is likely to be affected specifically in a pragmatic language task (i.e., where the individual needs to make flexible context-dependent inference about uncertain implied meanings), and not on tasks focused on more structural, codified aspects of language such as grammatical competence. As such, we present another metalinguistic task, the Grammaticality Decision Test, as a control task with a similar response format to the pragmatic task to test the specificity of any differences.

We propose that reduced confidence on the Implicature Comprehension Test will be a marker of IU in autistic people, and may be a more influential factor in the communication difficulties diagnostic of autism, as opposed to a “deficit” in understanding social meanings. If this claim is borne out, it would have a couple of implications for psychological practice. First, it would suggest that interventions targeting IU may be useful for autistic people wanting support with communication challenges. Current interventions for communication focus on explicit instruction in social skills, and reviews suggest modest effectiveness although there are questions about the extent to which skills transfer to daily life (Gates, Kang & Lerner, 2017; Spain & Blainey, 2015). A focus on IU may be a useful alternative target. Existing cognitive interventions involve integrating psychoeducation and cognitive challenge techniques to target a client’s beliefs about (un)certainty, and these have shown some effectiveness for treating mental health difficulties and particularly anxiety in the general population (Shihata et al., 2016). It remains to be seen whether such interventions could be adapted to support autistic individuals with distressing communication experiences, although this is a promising possibility given that early studies suggest that such interventions may be feasible and acceptable for autistic groups (Rodgers et al., 2018; Rodgers et al., 2017). Second, if a cognitive preference for certainty is especially significant as an explanation for social difficulties, then it supports an autism-positive approach to intervention which focuses on awareness of cognitive differences across communities. In addition, if performance on the Implicature Comprehension Test is a sensitive marker of IU, that in itself might have clinical and research utility, since measurement of IU is currently limited to self- and informant-report questionnaires.

A remaining question is whether any differences observed on our tasks are specific to autism or might also be relevant to other neurodevelopmental diagnoses. This is certainly plausible in the light of dimensional models of neurodiversity, where features of autism, developmental language disorder, dyslexia, ADHD, etc., might show some overlap and exist as a continuum in the general population (Thapar, Cooper & Rutter, 2017). To test the specificity of any cognitive differences observed on our tests, we will compare performance by autistic people to both a dyslexic and a general population sample. As neurodevelopmental conditions are often co-occurring, we view these three groups as defined less by a specific diagnostic label but rather as varying along a communication continuum. As such, one group is defined by social communication differences potentially alongside co-occurring language/literacy impairments (the autism group), a second group by language/literacy impairments but no diagnosed social communication difficulties (the dyslexia group), and a final group without any communication, language or literacy related diagnosis. It is possible that dyslexic adults may show some difficulty on our pragmatic task (i.e., the Implicature Comprehension Test), as previous research has documented some limited evidence for pragmatic difficulties in dyslexic individuals (e.g.,  Cappelli et al., 2018; Cardillo et al., 2018; Griffiths, 2007). An alternative possibility is that adults with dyslexia will show greater difficulty with tasks focused more on structural language skills compared to pragmatics. For instance, a meta-analysis has found that dyslexic adults perform less well on language measures, such as vocabulary, speeded naming, verbal memory and phonological processing, than people without a diagnosis of dyslexia, with moderate to large effect sizes (Swanson & Hsieh, 2009). Given that there is no clear reason to support one of these possibilities over the other, we will take a more exploratory approach with the dyslexic group to examine how they compare with autistic adults.

In summary, we propose the following hypotheses:

1. Autistic adults will score lower on a pragmatic language task when responses are coded purely in terms of confidence (number of yes and no responses, regardless of polarity) than when responses are coded in terms of accuracy (with yes and maybe yes, and maybe no and no responses, combined according to polarity), compared to adults without any neurodevelopmental diagnosis, but will not show this same disparity between accuracy and confidence on a core language task.

2. The number of less confident responses (maybe responses) on the pragmatic language task, the score on the Intolerance of Uncertainty Scale, and self-reported social communication difficulties will significantly intercorrelate across the full sample.

Power calculation

We determined power to detect the three-way interaction described in Hypothesis 1 using simulations. We used data reported in Wilson & Bishop (2020c) to estimate the likely size of fixed and random effects in the mixed model described in Data Analysis below. Using R package simr (Green & MacLeod, 2016) we ran 1,000 simulations with a sample size of 200 people (100 autistic, 100 non-autistic) and a significant three-way interaction was found in 9,830 simulations, indicating that power was over 98% to detect our effect of interest at an alpha level of .05. Effectively, this allowed us to detect a significant difference where approximate Cohen’s d values in favour of the non-autistic group were 0.70 and 1.10 for the implicature accuracy and confidence variables and 0.20 for the grammar variables, as suggested by our previous data. Allowing for exclusion of up to 10% of participants during the outlier exclusion phase described in Data Analysis, power remained very high (98% in a sample of 180). For Hypothesis 2, a sample of 200 was powered at over 99% to detect a correlation of .3.

Participants

We recruited individuals with autism, self-reported reading difficulties, and no neurodevelopmental diagnosis. Based on the power calculation, we aimed to recruit 100 autistic adults and 100 adults without a neurodevelopmental diagnosis in order to run the confirmatory analysis. In addition, we set out to recruit 50 dyslexic adults as a clinical control group for exploratory analysis. All participants met the following eligibility criteria: (i) age of 18 years or over, (ii) native-level fluency in English, (iii) no history of acquired brain injury, (iv) no significant uncorrected sensory impairment, and (v) access to a computer with internet and audio.

Individuals were recruited into three groups defined by communication and language/literacy problems. One group was recruited on the basis of a clinical diagnosis of autism; participants needed to declare where, by whom and what label was used for their diagnosis on the Study Questionnaire. For inclusion, the diagnosis must have been made in a clinical service by appropriately trained individuals, such as clinical psychologists, psychiatrists or developmental paediatricians. We recruited autistic individuals through Autistica, the research network for families and individuals with autism, as well as support groups arranged privately and by the National Autistic Society, and through social media. A second group included individuals reporting dyslexia or specific reading difficulties. For inclusion in this group, individuals needed to score below the clinical threshold of 6 on the Autism Spectrum Quotient (AQ) and at 6 or above on the Reading Scale of the Adult Reading Questionnaire (ARQ); this latter score translates to over 1.5 SDs above the mean in individuals not self-reporting dyslexia in the original validation study (Snowling et al., 2012). Individuals were recruited through charitable organisations such as the British Dyslexia Association and social media. Other neurodevelopmental diagnoses were not grounds for exclusion from these groups. A third group had no neurodevelopmental diagnosis, and was recruited via the online participant platform, Prolific (https://prolific.co). Individuals were excluded from this third group if they scored above threshold on either the AQ or ARQ (i.e., 6 or above on either) or if they have ever been diagnosed with: a global or specific learning disability, attention deficit hyperactivity disorder, dyspraxia/developmental coordination disorder, a genetic variation (such as Down’s syndrome or Fragile-X) or a neurological condition (such as epilepsy).

Procedure

The study was presented online using Gorilla, the online platform for behavioural experiments and surveys (https://gorilla.sc/). Individuals completed an online set of tasks and questionnaires in one sitting at a time and place of their choosing. After providing informed written consent to participate, individuals completed a Study Questionnaire (please see the OSF link) on which they reported on demographics and any neurodevelopmental diagnoses. Then they completed questionnaires/tasks in two sections. The first section included the experimental tasks required for the hypothesis-testing, and the second included several brief measures for characterising the sample. The two experimental tasks were randomized between participants, and all other measures were administered in the order set out below.

Measures

A priori data analysis plan

Individuals were excluded from the dataset if they had an outlying score for either (a) accuracy on the GDT or the positive control items of the ICT-2, or (b) total number of timeouts across the ICT-2 and GDT. Outliers were defined according to the method of Hoaglin & Iglewicz (1987): more than 2.2 times the interquartile range below the first quartile. In previous work, these criteria led to exclusion of approximately 5% of participants, and captured individuals scoring below approximately 50% on the GDT and 70% on the positive control items of the original version of the ICT (Wilson & Bishop, 2020a).

Data were analysed in R (R Core Team, 2019). After exclusions, total scores on the two experimental tasks for the groups with autism and no neurodevelopmental diagnosis were turned into long format, and each participant’s total was dummy-coded for task (ICT-2 or GDT), group (autistic or no neurodevelopmental diagnosis), response (accuracy or confidence) and participant. We ran a mixed effects linear regression using the lme4 R package (Bates et al., 2015). The model included three fixed effects (task, group and response) and the interactions between these, as well as a random effect (participant). The significance level of the three-way interaction offered a test of Hypothesis 1. We also computed correlations between confidence on the ICT-2, self-reported communication challenges on the CC-SR and total score on IUS-12 across the full sample; this tested Hypothesis 2 and represented a dimensional analysis of the relationship between communication difficulties and sensitivity to uncertainty. Table 1 shows a summary of our planned analyses, linking research questions, hypotheses, tests and power calculations.

In exploratory analysis, we examined how the group with self-reported reading difficulties compared to the autistic group and the group without a neurodevelopmental diagnosis on the ICT-2 and GDT in terms of accuracy and confidence.

Testing hypothesis 1

For this hypothesis, we focused on performance on our two key language tests (the ICT-2 and GDT) when comparing Groups 1 and 3 (our autistic and control groups). Please see Table 4 which shows effect sizes (Cohen’s d) for group differences on the four variables drawn from these tests. We can see that there were significant differences on the pragmatic language test (ICT-2) but not the core language test (GDT), with the autistic group showing somewhat lower scores on the ICT-2 but not the GDT. We checked whether these differences on the ICT-2 remained when controlling for gender. We included Group (autistic vs control) and Gender (female vs not female) as factors in a two-way ANOVA for each variable extracted from the ICT-2. There remained a significant effect of Group for both accuracy, F(1, 208) = 8.05, p = .005, partial η² = 0.04, and confidence, F(1, 208) = 9.37, p = .003, partial η² = 0.05, whereas Gender did not have a significant effect for either variable.

We hypothesised that the autistic group would show lower confidence on the pragmatic language test, controlling for their accuracy on the test and their performance on the core language test. We ran a mixed model with three fixed effects for Group (autistic vs control), Task (ICT-2 vs GDT), and Response (Accuracy vs Confidence) to test this hypothesis. The three-way interaction was not significant, p = .235, indicating that the hypothesis was not supported. As we can see from the effect sizes in Table 4, there is no evidence that the autistic group showed substantially lower confidence on the ICT-2 in relation to accuracy on the test. In fact, the autistic group gave responses on the ICT-2 that were both somewhat less accurate and less confident. They were in line with the control sample in their performance on the core language test (the GDT). See plots in Fig. 1 which show all data for each variable extracted from these tests.

In addition to this pre-registered analysis, we also looked at performance of the group with self-reported reading difficulties compared to the control group on the two tests. Overall, this group seemed closely comparable to the control group. The only variable extracted from the ICT-2 and GDT that showed a significant difference was Accuracy on the GDT, i.e., grammatical ability. The group with self-reported reading difficulties had significantly lower scores, t(116.16) = 2.24, p = 0.027, d =  − 0.35.

Testing hypothesis 2

Please see Table 5 for correlations between variables taken from the ICT-2 and GDT and self-reported communication difficulties on the pragmatic scale of the CC-SR and intolerance of uncertainty as measured by the IUS-12. We hypothesised that confidence on the ICT-2 and scores on the CC-SR and IUS-12 would significantly inter-correlate across the sample. This was the case, p < .001. Interestingly, accuracy on the ICT-2 showed similar correlations with these variables too, p < .001.