Factors of influence in prisoner’s dilemma task: a review of medical literature

Opponent parties’ composition

Opponent parties’ composition refers to the synthesis of the competing PD parties. We use the term “parties” to include PD experiments where the opponents are individuals as well as groups of individuals. Most commonly the PD game is used as a research analogue of social interactions. Such interactions mainly occur between individuals or groups, which constitute the opponent parties. The main difference between individual and group interactions is that in the between-groups condition, members of each group have to settle to an agreement about their representative PD choice.

The vast majority of the PD literature concerns interactions between individuals. There are however, three additional types of PD experiments where interactions cannot be strictly categorized as group or individual interactions. The first of these additional PD types, places the individual in a network of others, where their every PD choice simultaneously determines the outcome of multiple PD interactions between them and their neighbours. This type of PD is mainly used to investigate the dynamics of social networks, for example how the members’ psychopathic traits could influence the group or how dynamic partner updating would influence cooperation levels (Wang, Suri & Watts, 2012). In the second type (IPD-MD: Intergroup Prisoner’s Dilemma-Maximum Difference) (Halevy, Bornstein & Sagiv, 2008), the individual is a member of a group and their PD decision affects both members of their and other conflicting groups. This type of PD has been used in research to investigate specific characteristics of the players behaviour such as the motives behind altruism, including the desire to help one’s group or harm other groups (De Dreu, Dussel & Velden, 2015; Weisel & Böhm, 2015). The third type is that of a multiplayer PD where both the individuals’ and the groups’ rewards depend on the combination of the members’ decisions without involving any direct interaction between individuals. This PD type is a tool for investigating altruistic behaviour and cooperation as it highlights the dilemma between personal and social benefits (Sheldon, Sheldon & Osbaldiston, 2000; Locey, Safin & Rachlin, 2012).

From the research articles included in this review, the majority used interactions between individuals (207 experiments) with fewer studies looking at interactions between groups (10 experiments).

Opponent perceived type

Opponent perceived type refers to the players’ beliefs about the nature of their opponents. The subject’s perception of their opponent’s nature is of great importance for the PD game methodology. This is mainly due to the contrasting natures of the PD and that of modern methodology. The PD paradigm is mainly used to investigate the characteristics of social behaviour which per definition requires human interactions, modern methodological environments however, are machine-based and aim to minimize human involvement.

In order to compromise the two, players have to either (a) truly believe that machines could accurately mimic or simulate all aspects of human behaviour and assign to them innate human characteristics like compassion or (b) believe that their opponent is an actual human. This becomes even more complicated when taking into account the fact that machines are often used as a proxy for individual interactions. As a result, when someone confronts a soulless screen, they can never be sure about what it reflects/represents. For that purpose, a common practice is to ask participants, at the end of the experiment, what they believed about their opponent’s nature.

We have identified two categories for the opponent perceived type parameter: human agents playing as experimental subjects and agents with predefined strategy or answers. The first opponent type, human vs. human, is a more convenient and realistic model when we intend to study the interaction itself while the second opponent type, human vs. agent with predefined strategy, is a more controlled setting and allows the study of individual subject characteristics as well as the comparison between subjects. The latter category could be further separated in order to distinguish whether or not the subject was falsely informed about the human nature of their opponent or manipulated into thinking so. This manipulation is usually achieved by the ‘supposed’ presentation (i.e., introduction or photo) or actual presence of a human opponent, signs of communications (i.e., messages) and signs of human behaviour (i.e., delays). Whether or not this is truly important, it remains controversial. On one hand, Miwa & Terai (2012) have shown that participant’s behaviour during the PD game is influenced by the instructions given about the partner’s nature while Knight & Kagan (1977) stated that playing against an imaginary opponent strongly correlates with decision-making when an opponent is actually present. On the other hand, it has been shown that humans activate brain areas related to mentalizing when they knowingly face computers as co-players (Rilling et al., 2004a; Krach et al., 2008; Kircher et al., 2009). Finally, interaction (cooperation) significantly changes when comparing human and computer opponents, only when communication between the opponents is avoided (Kiesler, Sproull & Waters, 1996).

Another factor to consider when the ‘opponent type’ parameter is concerned is the sex balance of the players or opposing parties which could influence the game decisions (Rapoport & Chammah, 1965). Male opponents usually appear more cooperative than women in same-sex interactions while women cooperate more than men in mixed-sex settings (Rapoport & Chammah, 1965; Balliet et al., 2011; Colman, Pulford & Krockow, 2018). Social proximity is another important factor. Rewards to socially closer persons are valued more than equivalent and or identical rewards to socially distant ones, which in turn alters subjects’ behaviour as the cost of cooperation is compared to its socially discounted benefit (Locey & Rachlin, 2011; Locey, Safin & Rachlin, 2012; Safin, Locey & Rachlin, 2013; Safin, Arfer & Rachlin, 2015). A possible explanation for this, is perceived similarity which signals some degree of kinship (Park & Schaller, 2005), trustworthiness (Koch et al., 2020), likability (Alves, Koch & Unkelbach, 2017, 2018), and opportunity to form a group (Hehman, Flake & Freeman, 2018) and could thus explain cooperation in social dilemmas (Rand & Nowak, 2013).

Only 40% of the PD experiments, included in this review involved interactions between real human subjects. From these, only a few involved a vis-a-vis setting (mostly EEG experiments). The majority of the studies have used computers to conduct interactions (117 experiments). From the 123 PD experiments that have used algorithms, in at least 97 participants were manipulated to believe they were interacting with real humans.

Interaction flow

Interaction flow describes the order in which the players take turns throughout the PD game. Based on the interaction flow parameter, PD experiments can be distinguished into ‘simultaneous’ and ‘sequential’. In simultaneous PD, which is the most common type of PD, both parties choose their actions without knowing each others current decision. In the sequential PD, one party makes the first move (Player A) and the other the subsequent move (Player B). In this case, Player B has the benefit of already knowing Player’s A choice before deciding upon their move and thus Player B has a strategic advantage over Player A who is aware of this. Kiyonari, Tanida & Yamagishi (2000) has shown that the cooperation levels in the sequential PD are higher when compared to the simultaneous PD. Locey & Rachlin (2011) implemented a modified version of the sequential type where each participant has the role of both Player A and Player B, as each decision is the answer to the opponent’s last choice and at the same time the first move in the new round.

Using the interaction flow parameter, PD experiments can also be distinguished based on the total number (sum) of rounds. When only one round is involved the game is called ‘one-shot’, else, the game is called ‘iterated’. There is a great difference between the two types, as in ‘one-shot’ PD, the player does not have any information/knowledge concerning their opponent’s strategy that would be based on any previous behaviour—nor the opportunity to be strategic. In this case, defection seems to be the most rational choice. Experiments involving human subjects however, have proven that this is far from true since individuals often choose cooperation in ‘one shot’ trials (Shafir & Tversky, 1992; Wedekind & Milinski, 1996; Wong & Hong, 2005). This justifies the use of ‘one-shot’ PD as a means to evaluate the altruistic nature of the agents. In contrast to ‘one-shot’ PD, ‘iterated’ PD, encompasses the essential time frame for strategy to be assessed and implemented and thus it is more suitable for studying the specifics of the evolution of cooperation (i.e., reputation, hierarchy, strategy). Another form of ‘iterated’ PD is that involving multiple ‘one-shot’ games (Yang & Yue, 2019). In this case, the possibility of a future rematch with the same opponent can induce higher cooperation levels (Bó, 2005; Duffy & Ochs, 2009; Arechar, Kouchaki & Rand, 2018; Rand & Nowak, 2013).

From the articles included in our review, only 29 papers involve a sequential type of PD while 186 involve a simultaneous one.

Number of rounds

The ‘number of rounds’ parameter determines the length of the PD game and it is relevant only for the ‘iterated’ type of PD as ‘one-shot’ PD consists of only one round. Our literature search revealed a great variability in the number of rounds used in research, with most studies using 10 to 30 rounds. The total number of rounds in a PD experiment, is considered important for achieving cooperation. The fewer the rounds, the bigger the risk of unsuccessful negotiation and loss. Lave (1965) suggests a minimum number of 3 * (P − S)/(R − P ) rounds, where R stands for reward, P for punishment and S is for sucker payoff (see introduction and ‘reward matrix and payoffs’ section) to achieve mutual cooperation based on the reward matrix and Chang et al. (2010) demonstrate that 12 rounds are sufficient for individuals to achieve stable levels of cooperation.

As a PD game approaches to the end, the number of remaining rounds decreases and this can influence the participants’ behaviour; end-game effect. As the margin for retaliation is getting smaller, temptation for opportunistic behaviour increases along with the uncertainty concerning the opponent’s attitude towards this temptation. Although mitigating the end-game effect is not a common practice in the PD literature, there are two prominent strategies to address the issue. Instead of explicitly defining the exact number of rounds to participants (finite round type) either this information is withheld, or the number of rounds is stochastically determined. The first can be achieved by simply not revealing to participants the total number of rounds of the game (Wedekind & Milinski, 1996; Melamed, Simpson & Harrell, 2017; Grujić & Lenaerts, 2020) while the second by pre-defining a specific probability for the game to be terminated after each round (Shafir & Tversky, 1992; Wedekind & Milinski, 1996; Kiyonari, Tanida & Yamagishi, 2000; Hehman, Flake & Freeman, 2018). Keeping the termination of the game vague, could help prevent the common human practice of defecting in the final rounds and thus avoid the ‘Nash equilibrium’ (Colman, Pulford & Krockow, 2018).

In the literature, the range of the total number of rounds used in experiments is vast, spanning from 2 to 500 with a median of 20 and an average value of around 45. The most commonly selected numbers of rounds which were used in more than half of the experiments, range between 10 and 30. In order to avoid end game effects, in 28 experiments, researchers have either chosen not to reveal the total number of rounds to participants or to assign, in each round, a specific probability for the game to be terminated (stochastic determination of the end of the game).

Instructions, narrative and options

The task’s instructions, narrative and options refer to the way and the context under which the PD game is presented to participants and are provided to participants before the beginning of the task or throughout the paradigm. The options that are available for the players and the representation of the payoffs matrix, the social context of the experiment’s implementation, the definition of the game’s purpose, the opponent type perception and knowledge of outcome’s inter-dependency (Martin et al., 2013) are aspects of the task that can influence the player’s decisions.

The typical PD narrative (criminal’s story), implements ‘cooperation’, ‘defect’ or ‘betray’ as available choices. This, however, is not the case for the majority of PD studies where letters (i.e., A/B, X/Y), numbers and other combinations are commonly used as analogues of cooperation and defection. These options are neutral enough to prevent priming competitive behaviour and also spare participants from the social moral weight of ‘defection’ or ‘betrayal’ which seem to promote cooperative decisions (Deutchman & Sullivan, 2018; Capraro et al., 2019), implement social bias (Capraro & Cococcioni, 2015) and are too diverse to represent common social interactions. Moreover, extra options (i.e., punishment, reward, withdrawal, 1–7) may be available to participants, either explicitly, as by payoff matrix definition, or more dynamically (i.e., ‘decide how many units to give to the other party’) depending on the PD design.

Researchers, when first introducing participants to the PD game, in addition to simply explaining the rational of the game and the matrix payoffs, which is the common practice, could use different imaginary scenarios (Table 1) in accordance with methods specific to the PD design. These scenarios usually lack the social bias of the original narrative but maintain their social context which contributes to the task’s understanding. It has been shown that socially-framed tasks are easier to grasp (Cosmides, 1989; Alekseev, Charness & Gneezy, 2017), and an in-depth comprehension of the task’s design and rules is crucial for the reliability of research. There is, however, the possibility of unexpected framing effects. For example, unpredictable associations between the imaginary scenarios presented in the PD experiment and past experiences of the players as well as intuitive or familiar preferences (Capraro, Jordan & Rand, 2014; Capraro & Cococcioni, 2015; Capraro et al., 2019). Additionally, Capraro & Rand (2018) and Capraro et al. (2019) investigating the role of morality as an influence in PD choices have shown that pro-social people tend to choose the option that is presented as being morally right in the given situation.

PD instructions commonly state that the purpose of the experiment is to achieve the maximum amount of points alternatively, they set the monetary reward as a function of the participants’ performance. This is done in order for the players to be motivated, the experiment to be an accurate representative of both real life and the human nature, and also to avoid the ‘always defect’ strategy which is the optimum for just winning but would not yield the maximum monetary rewards. Finally, to establish the adequate understanding of the task’s design and rules, most researchers require from participants to answer comprehension questions prior to PD trials.

Experimenters mainly prefer socially neutral (letters) or positive (cooperate vs. no-cooperate) vocabulary in their narrative as only 47 experiments use terms such as ‘defection’, ‘betray’, ‘punish’, ‘steal’ or ‘tell-on’ as options. A total of 19 experiments have implemented more than two options and 35 experiments utilised a different game design from the typical straightforward PD, like joint-invest, chest-keys etc.

Strategy

Strategy refers to the decision making process that unfolds during the PD game. The implementation of a strategy is necessary when the subject competes against an agent whose answers are controlled by the experimenter (algorithms). In ‘one-shot’ PD (see ‘number of rounds’ section), strategy implementation is limited to one decision since there is only one round. There are no priors of interaction and no planning involved as no future interaction will occur. This, however, is not the case for multiple ‘one-shot’ PDs which are played with the same opponent.

Strategy implementation mainly concerns the ‘iterated’ PD and by far the most common strategy is ‘tit-for-tat’ (Axelrod, 1984) and its variations. Other frequently used strategies include ‘always-cooperate’, ‘always-defect’ and ‘random’ (for a more detailed description see Table 2). ‘Tft’ starts with cooperation and then the strategy entails replicating the opponent’s previous move. In the famous Axelord’s competitions, this strategy has proven to be most successful while also promoting cooperation (Kuhlman & Marshello, 1975; Axelrod & Dion, 1988). ‘Tft’ has however, two main weaknesses, it cannot reverse an established mutual defection, meaning that any mistake or environmental noise would be of great influence during the course of the game and secondly, it cannot maximally exploit an unconditional cooperator. In order to eliminate the first weakness, two main subtypes of ‘tft’ have been proposed. The first subtype is called ‘forgiving-tft’ or ‘generous-tft’ (Nowak & Sigmund, 1992) and it involves a percentage of cooperation after the opponent’s defection. The second subtype is called ‘tit-for-two (or more)-tats’ (Boyd & Lorberbaum, 1987) where defection is chosen following continuous defections by the opponent.

Another strategy which is underrepresented in the literature (Martin et al., 2013), is ‘win-stay-lose-shift’ (wsls) or ‘pavlov’ (Nowak & Sigmund, 1993) where the players switch their choice whenever they receive a bad payoff, S or P. ‘Wsls’ can be exploited by unconditional defectors but overcomes the ‘tft’ weaknesses. In order to promote cooperation, the reward matrix (see following section) in this strategy must fulfill the condition 2 * R > T + P where R stands for reward, T for temptation and P for punishment. Otherwise the player has no motive to cooperate (C) since continuous defection (D) (DD-DC loop) would gather at least the same amount of reward as cooperation (CC-CC or CD-CD).

Other interesting strategies mentioned in the literature are Zero Determinant (ZD) (Press & Dyson, 2012; Pansini, Shi & Wang, 2016), ‘pavlov-tft’ (Juvina et al., 2012) and ‘Raise the stakes’ (Roberts & Sherratt, 1998). ‘Pavlov-tft’ is considered to better fit human behaviour compared to its consisting strategies while ‘Raise the stakes’ can be applied to more dynamic invest-like PD designs. These designs, include ‘Give doubled—keep tripled’ and escalates the players’ investment when partners match or better their last move (for details on the different PD designs refer to Table 1). Other more straightforward, commonly implemented strategies include predefined choices (Papageorgiou et al., 2013), defecting in a number of rounds (Bitsch et al., 2018a) or decisions based on probabilities (i.e., 40% defection) (Haruno & Kawato, 2009). When PD is used for fMRI experiments, most strategies are based on Rilling et al. (2004b) work on collected human data and Rilling’s (Rilling et al., 2004a, 2004b, 2012) and McClure’s (McClure et al., 2007) strategies are the most frequently used, see Table 2.

As expected, the lion’s share in the medical literature (40 experiments) belongs to ‘tft’ based strategies while other ‘famous’ strategies where non-human participants are involved, like ‘wsls’ appear to be underrepresented (1 experiment). A total of 58 experiments involve non-typical and mixed strategies.

Table 2 summarizes the strategies used in literature.

Reward matrix and payoffs

The reward matrix and payoffs determine the gains and loses that different PD choices could bring to the players. The reward matrix could be considered the most important part of the PD as it determines the motive as well as the pros and cons of the agent’s decisions since different strategic choices are reflected upon the expected reward. There is no point in cooperating or defecting if there is no profit from it, or no punishment for doing otherwise. As a result, the amount of payoffs for each possible outcome and the degree of difference between these payoffs strongly influence the course of the game.

In a PD game, the reward matrix typically consists of four values, ‘T’ (temptation) is the reward of someone’s defection over cooperation, ‘R’ (reward) is the reward of mutual cooperation, ‘P’ (punishment) the payoff of mutual defection and ‘S’ (sucker) the payoff of cooperation over defection. In order for a reward matrix to accurately represent PD’s dilemma, the following condition must, at least, be met, T > R > P > S (Axelrod & Hamilton, 1981). The above is not sufficient for the ‘iterated’ version of the game, where 2 * R > T + S is another mandatory condition which gives mutual cooperation a greater reward value thus preventing individuals from alternating between cooperation and defection strategies (Axelrod & Hamilton, 1981). Moreover, as stated in the ‘strategy’ section, for the ‘wsls’ strategy to be meaningful, the 2 * R > T + P condition must also be met. Although the above conditions are standard for a typical PD game (Axelrod & Hamilton, 1981) a lot of papers deviate from these mainly by implementing the P < S condition, in order to promote cooperation (Bitsch et al., 2018b; Acosta, Straube & Kircher, 2019). The P < S condition, however, transforms such paradigms into a snowdrift or hawk-dove game which deviate from the typical PD task since mutual defection has the minimum payoff.

But how do the different combinations of those four matrix values, influence participants’ decisions? In order to answer that question, one factor that needs to be considered is the cooperation index ratio ((R − P )/(T − S)) used by Rapoport & Chammah (1965) and Rapoport (1967). Greater ratio values indicate that defection is less tempting and cooperation less risky, thus defining a more cooperation-friendly environment. In addition to the cooperation index ratio, Sherman (1967) proposes two other ratios derived from the values of the reward matrix parameters. These ratios represent attractiveness of cooperation ((R − P )/(P − S)) and attractiveness of defecting from a cooperative solution ((T − R)/(R − P )). A higher value for the first ratio represents relatively higher gain from cooperation while lower values for the second ratio relatively low gain from defecting, resulting, in both situations, in greater cooperation levels. The index of riskiness ((R − S)/((R − S) + (T − P ))), as proposed by Au et al. (2011) is a measure of the risk of cooperation, with values ranging from 0 to 1. Finally, Baas et al. (2019) and Huang et al. (2019) propose that (T − R) and (P − S) values could be representative of greed (gain the maximum from exploiting opponents cooperativeness) and threat (fear of being exploited) respectively, linking them both (mainly greed in older adults) to non-cooperative behavior.

Matrix payoffs are not just numbers, they represent units of reward. These rewards could either be real or hypothetical and whether that would influence the participants’ decisions remains unclear. PD experiments that tried to test this (Locey, Jones & Rachlin, 2011) produced ambivalent results and in general, the literature of decision making research examining hypothetical over real rewards has produced inconclusive results (Hertwig & Ortmann, 2001; Madden et al., 2003; Mieth et al., 2021). It is generally believed however, that real monetary rewards offer greater incentives compared to equivalent in value but hypothetical rewards (Hertwig & Ortmann, 2001; Bruner, Goodnow & Austin, 2017). However, how these would alter decisions in a PD setting, remains unknown. Nonetheless, Furlong and Offer suggest that what influences cooperation, is not the real value of payoffs but their numeric representation, meaning for example, that a reward of 300c would induce greater cooperational behaviour compared to that of 3$ (Furlong & Opfer, 2009).

In practice, in the literature, the most commonly encountered matrix values include quadruples T:3 R:2 P:1 S:0, T:5 R:3 P:1 S:0 and their multiples (i.e., T:30 R:20 P:10 S:0, T:15 R:9 P:3 S:0) as well as adding or subtracting a value from each of these payoff numbers (i.e., T:10 R:5 P:0 S:–5) (Supplemental Material).

Many experiments in the literature use negative values for P and S. Locey & Rachlin (2011) have shown that actual losses as punishment, lead to higher cooperation levels than gain reductions a finding that is consistent with previous literature (Gray & Tallman, 1987). Additionally, positive payoffs compared to losses (i.e., negative values for T, R, P and S), also favour cooperation (Sun et al., 2016; Deutchman & Sullivan, 2018).

Another aspect of the matrix that can influence the task’s outcome is the way it is presented (Evans & Crumbaugh, 1966) or in other words the context in which the conflict situation is presented to the participants. This could be either a grid representation of outcomes of every decision combination, i.e., T:3 R:2 P:1 S:0, or a non-grid representation, i.e., ‘either take 1 or give opponent 2’ which results to the same aforementioned grid. In Evans & Crumbaugh (1966), the non-grid representation which appeared to be better understood from participants, favoured cooperation.

Playing an ‘iterated’ PD of many rounds or multiple ‘one-shot’ games all with an unchanged reward matrix, can lead to boredom, loss of focus or strategic decisions that are made without adequate consideration. This is an important issue especially in psychophysiological experiments. Our literature search, revealed two ways to circumvent those issues and keep the participants interested in the task. Both ways involve the use of a dynamic matrix that entails changing payoffs in every round either by keeping the T − R difference the same (Emonds et al., 2013; Lambert et al., 2017) or by slightly altering the T, R and P values keeping however, the mean value of each parameter constant (Zhang et al., 2019).

Another reason for different amounts of reward associated with each round, is that the available amount might not be fixed for each round. This could be the case in more dynamic PD designs where the player decides how many units to give to the opponent (i.e., ‘Give doubled’, ‘Joint invest’) or in a network or multiplayer PD experiment where the final payoff of a round depends on the decision of other parties. Finally, in some cases, the payoffs can change as a result of decisions that occur between rounds such as the decision to punish the opponent after each round. (Fehl et al., 2012; Bone et al., 2015, 2016; Mieth et al., 2021). Payoffs can also change as the result of environmental variability, for example, a change in the reward values (Taheri, Rotshtein & Beierholm, 2018) or random decision switch (Rodebaugh et al., 2011, 2013; Grujić et al., 2012) can simulate uncertainty, which facilitates forgiveness and cooperation.

Based on payoff asymmetries, PD is distinguished in ‘symmetric’ and ‘asymmetric’. Most experiments implement a symmetric PD, there are however, a few experiments with asymmetric PD games (Gerbasi & Prentice, 2013; Aksoy & Weesie, 2014; Antonioni et al., 2018) where the reward asymmetry is used as a means of implementing inequality and hierarchy like in most social interactions.

Some PD literature uses matrices with more than four outcomes or in other words with a bigger payoff grid than 2 × 2. Wang et al. (2018) include for example, a reward option which plays the role of a more exploitable and beneficial for the opponent choice that is not associated with a greater payoff compared to cooperation. Pansini, Shi & Wang (2016) add the punishment option which is similar to cooperation in respect to one’s benefit but is most costly for the opponent. Finally, Insko et al. (2005) use the withdrawal option which results to the same payoff for both sides, irrespective of the opponents’ decision. Inserting one extra option as in the examples above leads to 9 (3 × 3 grid) possible rewards. Others (Houston et al., 2000; Knez & Camerer, 2000; Bone et al., 2016) use five or seven possible choices resulting to 25 or 49 possible payoffs. The rational behind these multichoice matrices is to create a reward gradient that could allow different cooperation and defection levels to be expressed without deviating too much from the typical PD game since taking into account only the first (defect) and last (cooperation) option of these multichoice matrices, still results in a valid typical PD.