Serotonin 5-HT_1A receptor binding and self-transcendence in healthy control subjects—a replication study using Bayesian hypothesis testing

Aims of the study

The aim of the present study was to perform a replication of our original finding of a negative correlation between 5-HT_1A receptor BP_ND and ST/SA in a larger sample. In addition to traditional frequentist statistics, we made use of Bayesian hypothesis testing, which allows us not only to test a hypothesis, but also to quantify the relative probability of the observed data under competing hypotheses. Recently replication Bayes factors (BF) have been introduced (Verhagen & Wagenmakers, 2014; Wagenmakers, Verhagen & Ly, 2016), allowing researchers to evaluate replication success by taking the outcome of the previous study fully into account. In this way, we aimed to evaluate the relationship between 5-HT_1A receptor binding and ST/SA from the perspective both of hypothesis testing without consideration of the magnitude of previous results, and of replication success.

Subjects

The sample consisted of 50 healthy men: 12 were enrolled as control subjects in a series of different pharmacological studies (for details see Matheson et al. (2015); 38 in a twin study (Borg et al., 2016). Age ranged from 21 to 55 (Mean = 30, SD = 5 years). The studies were approved by the Regional Ethics Committee in Stockholm and the Radiation Safety Committee of the Karolinska Hospital, and all subjects provided written informed consent prior to their participation in the studies (IRB 2008/60-31/3; for serotonin markers 2013/136-32).

MR and PET data acquisition (5-HT_1A binding potential)

Magnetic Resonance Imaging (MRI) images were acquired using a 1.5TGE Signa system (Milwaukee, WI, USA). T1- and T2-weighted MRI images were acquired for all subjects. The PET system used was Siemens ECAT Exact HR 47 (CTI/Siemens, Knoxville, TN, USA). All subjects were examined using [¹¹C]WAY-100635; The injected radioactivity was 276 ± 35 MBq (mean; SD). BP_ND values were calculated for the same regions as examined in the original study (Borg et al., 2003): frontal cortex, hippocampus (using the simplified reference tissue model - SRTM) and dorsal raphe nucleus (using a wavelet-based method using the non-invasive Logan plot in order to reduce the noise in this small region). For detailed description see Matheson et al. (2015). Other regions were not included in the analysis as they were not part of the original study. However, since [¹¹C]WAY100635 BP_ND is highly correlated between regions, the inclusion of more regions would therefore be unlikely to provide unique information from the three included regions (Bose et al., 2011).

Personality assessment

The Swedish translation of the TCI self-report questionnaire was used (Brändström et al., 1998). It consists of 238 true/false items covering four temperament dimensions (novelty seeking, harm avoidance, reward dependence, and persistence) and three character dimensions (self-directedness, cooperativeness, and self-transcendence). Individual scores were calculated for ST and its subscale SA.

Statistical analysis

Pearson’s correlation coefficients and their corresponding p-values were calculated for the correlation between ST/SA and 5-HT_1A BP_ND in the frontal cortex, hippocampus and dorsal raphe nucleus. Two BF tests were performed for each comparison. Firstly, we calculated a default correlation BF for the association between BP_ND and the ST/SA scores in frontal cortex, hippocampus and dorsal raphe nucleus respectively. Since we specifically wanted to test a negative correlation, we choose a one-sided default Bayes factor test, with a negative Beta prior of width 1 (i.e., flat between −1 and 0) using JASP (JASP Team, 2017). This test compares the predictive adequacy of the null hypothesis H₀ (i.e., no correlation) with an alternative hypothesis H_- (i.e., a negative correlation) (for more details on Bayes factors, see (Ly, Verhagen & Wagenmakers, 2016; Wagenmakers, Morey & Lee, 2016). Second, we calculated a replication BF for the correlations for each region as a measure of replication success. This test compares the predictive adequacy of the null hypothesis H₀ (i.e., no correlation) with an alternative hypothesis H_r. The alternative hypothesis is defined as the posterior distribution of the correlation coefficient derived from the original study, assuming a uniform prior before seeing the data of the original study (Wagenmakers, Verhagen & Ly, 2016). We slightly modified of the following source code http://www.josineverhagen.com/wp-content/uploads/2013/07/RepfunctionscorrelationFINAL1.txt (for plotting purposes) to the code which can be found online at the following address: https://osf.io/x9gjj/. This code was executed using RStudio (RStudio Team, 2017) with R 3.3.2 (R Core Team, 2015). We also reanalysed the results of Karlsson et al. (2011) with these methods. Bayes factors assess the relative likelihood of the observed data under competing hypotheses, yielding a ratio of the relative evidence for one hypothesis over the other. For instance, a BF₀₁ below 3 indicates weak or anecdotal evidence, a BF₀₁>3 moderate and a BF₀₁ >10 strong evidence in favour of the null against the alternative (Jeffreys, 1961). In this paper, all BFs are presented as the likelihood of the null hypothesis relative to the alternative hypothesis (i.e., BF₀₋ specifying a negative correlation as alternative; BF_0r specifying the posterior probability distribution of the original correlation as alternative). The differences between the default and the replication BF tests can be expressed as follows: the default test addresses the question of whether an effect was present or absent given relatively little prior knowledge of the effect size, while the replication test asks whether the effect was similar to what was found before, or absent (Wagenmakers, Verhagen & Ly, 2016).

Two potential sources of bias for this analysis were the inclusion of twin pairs, and the use of cerebellar grey matter as the reference region (Hirvonen et al., 2007). We therefore performed two additional analyses by (1) randomly excluding one twin from each twin pair (using http://www.random.org), resulting in a sample size of 31, and (2) using the white matter as a reference region for hippocampus and frontal cortex.

Limitations

Our data consisted of males only. We excluded women from the analysis since the literature is conflicting about the effect of gender and menstrual cycles on 5-HT_1A receptor binding (Palego et al., 1997; Tauscher et al., 2001; Cidis Meltzer et al., 2001; Parsey et al., 2002; Costes et al., 2005; Jovanovic et al., 2008; Stein et al., 2008; Moses-kolko et al., 2011) and gender influences ST scores on TCI (Brändström, Richter & Przybeck, 2001; Garcia-Romeu, 2010). Additionally, we wanted to replicate our original study, which contained only males, as closely as possible. Therefore, caution must be exercised when generalizing the present finding in male subjects to the female population. Karlsson and co-authors studied a gender mixed sample (11 males/nine females) in their previous negative study (Karlsson et al., 2011), and in genetic studies the association between serotonin genes and ST/SA has in some studies been reported to differ between gender (Nilsson et al., 2007; Aoki et al., 2010) whereas others found no difference (Lorenzi et al., 2005; Saiz et al., 2010).

The same is true for age: we had a similar sample to the original study, with limited range, and age might influence both ST scores and 5-HT_1A binding (Kirk, Eaves & Martin, 1999; Brändström, Richter & Przybeck, 2001; Moses-kolko et al., 2011).

As in the original study, we used the cerebellar grey matter as a reference region, which is not considered the gold standard due to small levels of specific binding in this region (Shrestha et al., 2012). However, using arterial plasma to calculate BP_P and BP_ND using cerebellar white matter as reference, Karlsson and co-authors could not replicate the original findings either (Karlsson et al., 2011). In addition, our analysis using cerebellar white matter showed similar results (see Supplemental Information 1).

Strengths

Where Karlsson and co-authors could only conclude that they did not find a significant correlation between ST/SA and 5-HT_1A receptor binding (Karlsson et al., 2011), using Bayesian hypothesis testing, we were able to conclude that the data supplied more evidence in favour of the null hypothesis (i.e., no correlation) for both our data and for that of Karlsson et al. (2011). Furthermore, the replication BF allowed us to take the magnitude of our previous results and its uncertainty fully into account. In this way, using the current data, the replication BF results suggest that the effect reported by the original study was likely either to be overestimated or a false positive. As such, these results support the conclusion that there is little to no association between ST/SA and 5-HT_1A receptor binding.

Of wider interest in the field of molecular imaging is that Bayesian hypothesis testing provides more informative conclusions than traditional p-values, thus offering pragmatic advantages for analysis of expensive neuroimaging studies, where limited sample sizes are common. For instance, Bayesian hypothesis testing allows for collecting data until the evidence is sufficiently strong to make a conclusion for one or the other hypothesis without requiring correction for multiple comparisons with sequential analyses. In this way, both costs and radiation exposure can be decreased.