Oliver Will (2009)

Resolving a Replication That Failed: News on the Macy & Sato Model

Journal of Artificial Societies and Social Simulation 12 (4) 11
<https://www.jasss.org/12/4/11.html>

For information about citing this article, click here

Received: 29-Aug-2009    Accepted: 27-Sep-2009    Published: 31-Oct-2009

Abstract

The paper at hand aimes at identifying the assumptions that lead to the results presented in an article by Michael Macy and Yoshimichi Sato published in PNAS. In answer to a failed replication, the authors provided the source code of their model and here the results of carefully studying that code are presented. The main finding is that the simulation program implements an assumption that is most probably an unwilling, unintended, and unwanted implication of the code. This implied assumption is never mentioned in Macy and Sato's article and if the authors wanted to program what they describe in their article then it is due to a programming error. After introducing the reader to the discussion, data that stem from a new replication based on the assumptions extracted from the source code is compared with the results published in Macy and Sato's original article. The replicated results are sufficiently similar to serve as a strong indicator that this new replication implements the same relevant assumptions as the original model. Afterwards it is shown that a removal of the dubious assumption leads to results that are dramatically different from those published in Macy and Sato's PNAS article.

Keywords:

Replication, Social Dilemma Situations, Trust, Simulation Methodology, Cooperation

Introduction

1.1

In Will and Hegselmann (2008a) we reported to JASSS on how we failed to replicate the results presented in Macy and Sato (2002). We considered several aspects of the model as causes of our failure. The most plausible explanation we could find, was that we had missed the way in which transaction costs were implemented. Lacking the opportunity to analyse the source code of the model, we were not able to verify this conjecture. Shortly after the publication of our paper, the authors' reply appeared in the same journal (Macy and Sato (2008)). No clear answer on why our replication might have failed was given in this text (Will and Hegselmann 2008b) but Macy and Sato published their source code in answer to our article. In the present text, I am presenting the insights I gained from studying the code carefully, trying to uncover those assumptions of the original model that we got wrong in our replication.

1.2

Macy and Sato's model is quite appealing. It implements a social structure that distinguishes between interactions in (a) more or less small groups of people that know each other comparatively well and (b) usually larger groups of strangers. This is a structural scenario that corresponds well to real-world interactions of people. The model predicts that social mobility, as long as it is not extremely high, has a large positive effect on trust among strangers. In a world with probably increasing social mobility this is an interesting finding. Thus Macy and Sato's work is of some interest to scholars of simulation but it needs to be clarified what assumptions lead to the published results.

1.3

In the following section a condensed overview of the assumptions that I extracted from the source code of the model is provided. The main finding is that there is an exclusion of certain agents from reinforcement learning that is most probably an unmeant, unintended, and unwanted implication of the code. This implied assumption is never mentioned in Macy and Sato (2002) and if the authors wanted to program what they describe in their article then it is due to a programming error. The subsequent section compares data that stem from a new replication based on these assumptions with the results published in Macy and Sato's PNAS article. The replicated results are sufficiently similar to serve as a strong indicator that the replication implements the same relevant assumptions as the original model. Afterwards I provide data from simulations in which the seemingly unmeant exclusion of agents from reinforcement learning mentioned above is repaired. This modified version leads to results that are dramatically different from those published in Macy and Sato (2002).^[1]

Assumptions of the model in Macy and Sato (2002) extracted from the source code

2.1

The following subsections give an overview of the assumptions that I extracted from the source code provided in Macy and Sato (2008). They are ordered according to the sequence of events in a simulation and each time step.

Initializing

2.2

• We have 1000 agents i.
• Agents are randomly distributed among N neighbourhoods n.
• The number of neighbourhoods is varied in order to generate average neighbourhood sizes N_s of 10, 20, ... , 90, 100. For this purpose, N is set to the integer part of 1000/N_s, i.e. takes on values of 10, 11, 12, 14, 16, 20, 25, 33, 50, 100.^[2]
• The agents' probabilities to enter the market (P_market(i), see 2.4), cooperate (P_cooperate(i), see 2.6) and to check telltale signs of character when deciding on trust or distrust (P_checkTTS(i), see 2.7) are set to a uniformly distributed random value between 0 and 1.

Moving

2.3

• In each time step, each agent moves to a randomly chosen neighbourhood (that is different from his previous one) with a certain probability.
• The probability to move is given by the exogenous parameter mobility rate and is varied from 0 to 1 in steps of 0.1.
• Agents that moved are classified as newcomers until they interacted once in their neighbourhood (and not on the market, see 2.4). This is very important since all newcomers are strangers and agents may decide not the interact with strangers.

Entering the market

2.4

• Each agent enters the market with an agent-specific probability P_market(i).
• Agents that entered the market are classified as strangers.
• Newcomers that enter the market remain newcomers in the next round.
• Newcomers that interact in the neighbourhood become neighbours.

Matching

2.5

• Agents that entered the market can only be matched with agents that chose to enter the market as well.
• Agents that chose not to enter the market can only be matched with agents within their own neighbourhood that did not enter the market.
• We thus have N + 1 pools of agents from which matches are drawn randomly.
• If and only if the number of agents in a pool is uneven then there remains one agent that does not have partner in this pool.

Cooperation

2.6

• Each agent i decides to cooperate (in a possible prisoner's dilemma situation with her partner, see 2.8) in the current time step with her agent-specific probability P_cooperate(i).

Trust

2.7

• In Macy and Sato's model, to trust means to enter a prisoner's dilemma. Mutual trust is needed for a PD. Whenever at least one agent does not trust, both agents receive an "exit payoff" that is set to -0.2 (see 2.8). Agents have two strategies to decide on whether or not they trust:
  • Agents that follow the parochial strategy distrust whenever their partner is a stranger. Strangers are all agents that either entered the market or are newcomers in local interaction (see 2.3, 2.4)). In local interactions with neighbours (agents within their own partition that are not newcomers, see 2.4) they trust.
  • Agents that act according to the signal-reading strategy trust their partner j with a probability that is given by the probability to cooperate, P_cooperate(j), of that partner.^[3]
• Each agent i chooses the signal-reading strategy with her agent-specific probability P_checkTTS(i) and the parochial strategy otherwise.

Payoffs

2.8

• Agents that remained without a partner during the matching procedure gain a payoff of -0.2 (see 2.5).
• Agents that do not trust or whose partner does not trust gain an "exit payoff" of -0.2.
• Agents in matches of mutual trust play the prisoner's dilemma. Given their decisions on cooperation (see 2.6) they end up with one of the following payoffs:^[4]
  • T: 1 - 0.5 · O(n)
  • R: 0.7 - 0.5 · O(n)
  • P: -0.2
  • S: -0.5
• O(n) represents the opportunity costs that agents pay for being restricted to a subset of all agents. They are given by O(n) = 1 - (n-1) / (N - 1) where n is the size of the pool of agents from which the respective match was drawn (see 2.5) and N is the total number of agents.
• Besides the payoff in each time step, the agents' cumulated payoff is tracked. The cumulated payoff of agent i in time step t is the sum of all the payoffs that agent i earned until the end of time step t. Note that there is no discount factor.

Learning

2.9

There are two types of learning in the model: Social learning from a role model and reinforcement learning based on the agents' own experience. Agents that do not have a partner do not learn at all.

Social Learning

2.10

• In each neighbourhood the agent with the highest cumulated payoff serves as the role model.
• If the role model is not a newcomer then all agents in the same neighbourhood whose cumulated payoff is strictly smaller than that of the role model copy each of the role model's propensities with a probability of 0.5,^[5] i.e. for each of those agents
  • P_market(role model) is copied with a probability of 0.5,
  • P_cooperate(role model) is copied with a probability of 0.5,
  • and P_checkTTS(role model) is copied with a probability of 0.5
• In case that the role model is a newcomer, there is no social learning in the respective neighbourhood and time step.

Reinforcement Learning

2.11

• Agents that are a role model or whose cumulated payoff is equally high, change each of their propensities, P_market(i), P_cooperate(i), P_checkTTS(i), according to

  (2)

  where P_a,t is the current probability of deciding in a certain way a, e.g. to defect, π_a,t is the payoff gained by that behaviour, e.g. -0.2 in mutual defection, and P_a,t+1 is the probability to decide for the same action a in the next time step.^[6]

• If i is a newcomer then the probability to enter the market P_market(i) of an agent i is never changed by reinforcement learning. (!!!)

2.12

This last assumptions is never mentioned in Macy and Sato's paper though their central claim crucially depends on it (see section 4.2). There is no obvious justification for it and - assuming that the authors wanted to program what they describe in their article - the most plausible diagnosis is, that it is an unmeant, unwantend and unintended consequence of some lines of code: in short a programming error.

2.13

A look at the code shows us that the authors experimented with agents that have two probabilities to enter the market: one in case of being newcomers and one if not.^[7] Both of these probabilities can change via reinforcement learning but not both during the same time step since an agent can be either a newcomer or not but not both and only the respective probability can be changed. To remove the assumption of two probabilities, the probability to enter the market as a newcomer is set to the value of the probability for entering the market as a neighbour after initialising the agents^[8] and after each learning event.^[9]

2.14

This causes a serious problem: For all agents that are newcomers and whose probability to enter the market in case of being a newcomer was changed by reinforcement learning, this probability will afterwards be overwritten by the (unchanged) probability of entering the market in case of not being a newcomer. Thus for agents that are newcomers reinforcement learning does not have any effect on their probability to enter the market.

2.15

At first sight, I did not expect this problem to have any substantial effect on the results of the model. However, the results I present in section 4 tell a very different story.

Successful replication

3.1

Using the following figures you can compare plots that stem from the replicated model based on the assumptions described in section 2 with plots from the original paper.^[10] Information on the figures is given in the captions of Macy and Sato's figures. Note that each combination of a mobility rate and a neighbourhood size was repeated 20 times.

3.2

When comparing the plots, one can easily see that the results from the replicated model match those from the original model very well. Not only in qualitative but also in quantitative terms they are very much the same. This indicates that it is very likely that the assumptions I extracted from the source code match at least the relevant ones of those that were implemented by Macy and Sato. Anyhow, this is only an argument to the best explanation. It might still be that there are other reasons for the good fit of the results but, in my opinion, this is quite unlikely. Furthermore, the high degree of quantitative similarity indicates that 20 repetitions of each parameter combination are sufficient to control for random effects of the model.

Figure 1. Plots of data from replicated (left) and original model (right).

Figure 2. Plots of data from replicated (left) and original model (right).

Figure 3. Plots of data from replicated (left) and original model (right).

Figure 4. Plots of data from replicated (left) and original model (right).

Figure 5. Plots of data from replicated (left) and original model (right).

Modified assumptions

4.1

Now, that I have good reason to believe that I uncovered the assumptions that lead to the results in Macy and Sato (2002), I want to explore the effects of changing some of them. As the implementation of the reinforcement learning seems to imply an assumption that is implemented unintentionally, the most urgent issue to check is how the model reacts if this problem is corrected. I therefore took the code of the successful replication and changed it such that newcomers can learn by reinforcement learning on whether or not they should enter the market as well. Afterwards exactly the experiments of Macy and Sato described in the previous section were repeated.^[11]

4.2

It turns out that after this seemingly small issue concerning the set of agents that might learn from reinforcement learning is changed, the model behaves dramatically different. Figure 6 shows the most interesting plot. We can see that for small levels of mobility the plots that show the indicators of market interaction over time are very similar in both cases. But for rates of mobility above 0.4 the results become more different the larger the rate of mobility gets. This is consistent with the modification of the code since the rate of mobility determines the number of newcomers and what was changed was whether or not newcomers can learn from reinforcement learning on whether they should enter the market.

4.3

The most important point is that trust in strangers does not increase with the rate of mobility. Actually the level of trust in strangers remains below or near 0.2 for all levels of mobility. This is very different from Macy and Sato's finding that for mobility rates between 0.1 and 0.9 the expected level of trust in strangers grows (approximately) linearly from a level of 0.1 to 0.7.^[12] Figure 7 shows us that there are actually some runs in which a high level of trust in strangers was reached. However, (ignoring mobility rates 0 and 1) it does not look as if there is a correlation between the probability of high levels of trust and the rate of mobility.

Figure 6. Plots of data from "corrected" (left) and original model (right).

Figure 7. Plots of data from "corrected" (left) and original model (right).

Conclusion

5.1

The previous sections suggest that the results presented in Macy and Sato (2002) stem from a source code that implied an assumption the authors did not intend to implement. If the model were robust against this assumption, everything would be fine. But it turns that the model crucially depends on that assumption.

5.2

The results presented here account for the importance of replication. Computer programs may imply assumptions we did not want to implement and this might even happen to the best in social simulation. Rigourous replication can detect such problems and furthermore uncover the assumptions that lead to published results. Thus replication is an important issue in the field social simulation and it is good to see that there is an increasing number of publications on replicating models.^[13]

Notes

¹ Both implementations were written in FORTRAN and can be downloaded from http://pe.uni-bayreuth.de/?coid=21&q=detail&mid=125. The download also contains the Mathematica notebooks that were used to generate the plots presented in this paper.

² This is a minor difference to our former repplication since in Will and Hegselmann (2008a) our variation of N was more fine-grained. E.g., to get an average neighbourhood size of 30, we conducted 7 runs with 34 neighbourhoods and the remaining 13 with 33 instead of taking 33 neighbourhoods in all 20 repetitions.

³ This is equivalent to the signalling mechanism suggested in Will and Hegselmann (2008a).

⁴ I use the standard notation here: If one agents defects and the other cooperates, the defector gains a payoff given by T(emptation) and the cooperator one given by S(ucker). In case of mutual cooperation both gain the R(reward payoff) and if both defect they receive the P(unishment) outcome.

⁵ In Will and Hegselmann (2008a) we assumed that newcomers can be role models.

⁶ In Will and Hegselmann (2008a) we assumed that all agents that have a partner learn either by social or reinforcement learning and that both options are equally likely.

⁷ The variables are named "PropMktN" and "PropMktS" in the source code. Line 492 shows their application.

⁸ Line 348 of the source code.

⁹ Line 753 of the source code. Here, you also find a note, that the distinction should be eliminated.

¹⁰ Here is a description of what the values in the figures mean. The exact definition of each value was extracted from the source code of Macy and Sato's model.

Market size:

Is the share of agents that entered the market (see 2.4).

Trust strangers:

Of all agents that have a partner and entered the market, "Trust strangers" gives the share that trust.

Reading signals:

Is the mean value of all agents' probability to conduct the signal-reading instead of the parochial strategy when deciding on trust or distrust (see 2.7).

Trust newcomers:

Of all agents that have a partner, are newcomers, and interact in the neighbourhood, "Trust newcomer" gives the share that trusts

Trusts neighbours:

Of all agents that have a partner, are not newcomers, and interact in the neighbourhood, "Trust neighbours" gives the share that trusts

¹¹Not only were the parameters the same but I also used the same random seeds to initialise the model.

¹²Note that this modified model produces results that are not only different from those in Macy and Sato's article but also from those of the very first replication in Will and Hegselmann (2008a). Theses differences are mainly due to further differences concerning the learning algorithm mentioned in footnotes [5] and [6]. The model's high sensitivity to small modifications of the learning algorithm will be adressed in future work.

¹³See for example van de Rijt, Siegel and Macy (2009).

References

MACY M and Sato Y (2002) Trust, cooperation and market formation in the US and Japan. Proceedings of the National Academy of Sciences, 99, pp. 7214-7220.

MACY, M and Sato Y (2008). 'Reply to Will and Hegselmann'. Journal of Artificial Societies and Social Simulation 11 (4) 11 https://www.jasss.org/11/4/11.html.

VAN DE RIJT, A, Siegel, D and Macy, M (2009). 'Neighborhood Chance and Neighborhood Change: A Comment on Bruch and Mare'. American Journal of Sociology 114, 4, pp. 1166-80.

WILL, O and Hegselmann, R (2008a). 'A Replication That Failed — on the Computational Model in 'Michael W. Macy and Yoshimichi Sato: Trust, Cooperation and Market Formation in the U.S. and Japan. Proceedings of the National Academy of Sciences, May 2002''. Journal of Artificial Societies and Social Simulation 11 (3) 3 https://www.jasss.org/11/3/3.html.

WILL, O and Hegselmann, R (2008b). 'Remark on a Reply'. Journal of Artificial Societies and Social Simulation 11 (4) 13 https://www.jasss.org/11/4/13.html.

Return to Contents of this issue

© Copyright Journal of Artificial Societies and Social Simulation, [2009]