Abstract

An update to Volker Grimm and colleagues' Overview, Design concepts and Details (ODD) protocol for documenting individual and agent based models (I/ABM) has recently been published in Ecological Modelling. This renames the 'State variables and scales' element to 'Entities, state variables and scales', and the 'Input' element to 'Input data', introduces two new Design concepts ('Basic principles' and 'Learning'), and renames another ('Fitness' is now generalised to 'Objectives'). The Design concepts element can now also be shortened such that it is not required to include any design concept that is irrelevant to the model, and expanded to include new design concepts more appropriate to the model being described. Other clarifications of intentions in the original protocol have been made.

Keywords:

ODD, Individual Based Models, Agent Based Models, Replication, Documentation

Introduction

1.1

Grimm et al. (2006) originally developed the Overview, Design concepts and Details (ODD) protocol for documenting individual and agent based models (I/ABM) to address frustrations with the lack of consistency in the text describing such models in journal articles (mainly in the field of ecology). In particular, such descriptions were often incomplete in the sense that they did not provide enough information to replicate the results, and/or were difficult to read because the information was not presented in a logical order. Similar frustrations have been expressed by authors concerned with agent-based social simulations (Treibig and Klugl 2009; Janssen et al. 2008; Richiardi et al. 2006), some making their own proposals for documentation standards. The ODD protocol has been widely adopted in ecological models (about 70% of articles citing Grimm et al. (2006) are in that area (Grimm et al. 2010)).

1.2

Issues with replication are commonplace in social simulation just as they are in ecology, and there have been various activities in the community that have been specifically aimed at addressing them. Three model-to-model workshops have been held, two of which have been the subject of special sections in this journal (Hales et al. 2003; Rouchier et al. 2008). Furthermore, the forum of JASSS has been allocated to reports on replications of agent-based models (recent examples include Miodownik et al. (2010), Macy and Sato (2010) and Will (2009)). Further, the annual Volterra prize for replication has been issued with the aim of demonstrating the robust use of agent-based models, the first winner of which was Radax and Rengs (2010).

1.3

In the meantime, the use of ODD in the social simulation community is small but growing. Grimm et al.'s (2010) review of uses of ODD categorised just two of 54 papers as being in the social sciences, though two of the other papers might also be regarded as social simulations. Polhill et al. (2008) demonstrated ODD for three agent-based social simulations of land-use change. Between the review being conducted and October 2010, five further papers documenting social simulations have used ODD: Griffith et al.'s (2010) simulation of how habitat changes could have influenced populations of early hominids; Hartig and Drechsler's (2010) model of farmers trading in a biodiversity market; Le et al.'s (2010) application of their Land Use Dynamic Simulator to policy scenarios; Naivinit et al.'s (2010) documentation of a model co-constructed with rice farmers in Thailand; and Valbuena et al.'s (2010) model of farmers' responses to liberalisation and regionalisation scenarios in the Netherlands.

1.4

With growing interest in, and use of, ODD, the time is right to update the protocol in the light of experiences with it to date.

Updating the protocol

2.1

In the update to ODD, Grimm et al. (2010) review citations of Grimm et al. (2006) on Web of Science at 14 December 2009, checking their use of ODD against the protocol, and finding that whilst 75% of uses were (almost) correct, 11% of the papers had significantly deviated from it. Based on this review, and feedback from users, including Polhill et al. (2008), Grimm et al. (2010) make the following changes to the protocol:

Purpose

2.2

Many articles were found to have left out 'Purpose', one possible motivation being that it repeats material that may already have been provided in the introduction to the article. Though the new protocol retains 'Purpose' there is an emphasis on keeping the statement short.

Entities, state variables and scales

2.3

This element has been renamed from 'State variables and scales' to 'Entities, state variables and scales', to move beyond a point where all that matters is the variables in the model; often grouping the variables into entities allows the model to be more clearly described. It is also clarified that 'state variables' can include non-numeric discriminants of objects, such as variables determining behaviour or strategies.

Process overview and scheduling

2.4

The review found that schedules were often still not described in enough detail to enable replication, despite using ODD. The definition of 'Process overview and scheduling' has therefore been specified more precisely, and recommends the use of pseudo-code to describe the schedule, which should include descriptions of how execution is ordered when a process is conducted for multiple agents.

Design concepts

2.5

Two new headings were introduced to this element: 'Basic principles' and 'Learning', and the 'Fitness' design concept was generalised to 'Objectives'. However, the main change to the 'Design concepts' element was to introduce greater flexibility. For smaller models, some of the design concepts are not relevant, in which case they can be ignored. For other models, authors may feel that important concepts underlying the design are not in the ODD protocol. Here, these concepts should be given a short heading and included at the end of the element, making it clear that they are not 'ODD-standard' design concepts.

2.6

The 'Objectives' design concept encompasses a more general idea than that of 'Fitness', which applies to any aspect of an individual's behaviour that is aimed at increasing their success in achieving one or more objectives. Here, the protocol stipulates that authors record those objectives, how success is measured, and how agents choose between several options with differing expected success (a particular issue where there are multiple objectives). The 'Learning' design concept was introduced to allow authors to describe how agents adapt their decision-making processes to improve the expected success with which objectives are achieved. The distinction between 'Objectives' and 'Learning' is somewhat subtle; if an agent decides what it will do using a set of rules R, then the 'Objectives' are what the agent is trying to do by applying R. By contrast, 'Learning' refers to how R is changed in the light of experience to improve the agent's ability to achieve its objectives.

2.7

The 'Basic principles' design concept is provided to allow authors to describe theories, hypotheses, and modelling approaches underpinning the design.

Initialisation

2.8

No significant changes were made to the 'Initialisation' element.

Input data

2.9

The name and description of the 'Input' element were changed slightly to clarify that it refers to time-series data used while the model is running. (Initialisation, by contrast, describes how values are found for the state variables at time 0.) Such data might be referred to as scenario, or driving data, and the variables they give values for as exogenous.

Submodels

2.10

The review found a mismatch between the processes named in the 'Process overview and scheduling' element and subheadings of the 'Submodels' element. It was also not clear in the original protocol specification that the factual description of a submodel (i.e. its equations, rules and algorithms) and its rationale should be separated. The protocol description was amended to make appropriate clarifications.

Conclusion

3.1

Hopefully the revisions to ODD increase its relevance and utility to the social simulation community. Indeed, at the 2010 World Congress on Social Simulation in Kassel, Janssen's (2010) plenary talk announced progress on initiating a new journal emphasising empirically embedded uses of agent-based models, and that this journal would encourage use of ODD when describing models in its articles. However, an issue acknowledged in Polhill et al. (2008) was also raised in the ensuing debate: whilst the revised ODD has attempted to distance itself further from a focus that might be deemed better suited to describing models built with object-oriented programming languages, it may be that it is still not a good fit for models built in declarative programming languages popular with many researchers in agent-based social simulation (e.g. Alam et al. 2007; Edmonds and Hales 2004; Harbers et al. 2009; Moss 1998; Pajares et al. 2004). To assess this properly will require such researchers to evaluate ODD and see where it presents obstacles to the more natural description of declarative models. One area likely to be difficult is the 'Process Overview and Scheduling' element: in declarative models the order of execution of the rules is determined by the question asked of the model and the inference engine's mechanics^[1]. One suggestion is that this element be used to give such information instead, with, perhaps, an interaction diagram showing which rules fire which other rules^[2], albeit that possibly the title of the element wouldn't match quite with the content. Similarly, though the element title and information content don't quite match, the 'Submodels' element could be used to give details on the rules.

3.2

This is unlikely to be the final version of ODD, and should it become popular in the social simulation community there will be more feedback on which to base future revisions. In the meantime, you are encouraged to use and try ODD in your own articles, and invited to comment through the openabm.org forum.

Acknowledgements

This work was funded in part by the Scottish Government Rural and Environment Research and Analysis Directorate.

Notes

¹ 'Pure' declarative models, where the issues described here apply, should be distinguished from 'mixed' models whereby the schedule is procedural, but the model uses declarative methods for agent cognition (as is the case with some of the models cited). Though it cannot be confirmed until ODD is more widely used, 'mixed' models should be no more difficult to describe using ODD than models built using more conventional object-oriented programming languages.

² Thanks to Scott Moss for this suggestion.

References

ALAM S J, Meyer R, Ziervogel G and Moss S (2007). The impact of HIV/AIDS in the context of socioeconomic stressors: An evidence-driven approach. Journal of Artificial Societies and Social Simulation 10 (4), 7. https://www.jasss.org/10/4/7.html

EDMONDS B and Hales D (2004). When and why does haggling occur? Some suggestions from a qualitative but computational simulation of negotiation. Journal of Artificial Societies and Social Simulation 7 (2), 9. https://www.jasss.org/7/2/9.html

GRIFFITH C S, Long B L and Sept J M (2010). HOMINIDS: An agent-based spatial simulation model to evaluate behavioural patterns of early Pleistocene hominids. Ecological Modelling 221 (5), 738-760. [doi:10.1016/j.ecolmodel.2009.11.009]

GRIMM V, Berger U, Bastiansen F, Eliassen S, Ginot V, Giske J, Goss-Custard J, Grand T, Heinz S K, Huse G, Huth A, Jepsen J U, Jørgensen C, Mooij W M, Müller B, Pe'er G, Piou C, Railsback S F, Robbins A M, Robbins M M, Rossmanith E, Rüger N, Strand E, Souissi S, Stillman R A, Vabø R, Visser U and DeAngelis D L (2006). A standard protocol for describing individual-based and agent-based models. Ecological Modelling 198 (1-2), 115-126. [doi:10.1016/j.ecolmodel.2006.04.023]

GRIMM V, Berger U, DeAngelis D L, Polhill J G, Giske J and Railsback S F (2010). The ODD protocol: A review and first update. Ecological Modelling 221 (23), 2760-2768. [doi:10.1016/j.ecolmodel.2010.08.019]

HALES D, Rouchier J and Edmonds B (2003). Model-to-model analysis. Journal of Artificial Societies and Social Simulation 6 (4), 5. https://www.jasss.org/6/4/5.html

HARBERS M, Meyer J-J and van den Bosch K (2009). Explaining simulations through self explaining agents. Journal of Artificial Societies and Social Simulation 12 (3), 6. https://www.jasss.org/12/3/6.html

HARTIG F and Drechsler M (2010). Stay by thy neighbor? Social organization determines the efficiency of biodiversity markets with spatial incentives. Ecological Complexity 7 (1), 91-99. [doi:10.1016/j.ecocom.2009.07.001]

JANSSEN M (2010). Towards collective action among social simulation. Presentation to the Third World Congress on Social Simulation, Kassel, Germany, September 6-9, 2010.

JANSSEN M A, Alessa L N, Barton M, Bergin S and Lee A (2008). Towards a community framework for agent-based modelling. Journal of Artificial Societies and Social Simulation 11 (2), 6. https://www.jasss.org/11/2/6.html

LE Q B, Park S J and Vlek P L G (2010). Land Use Dynamic Simulator (LUDAS): A multi-agent system model for simulating spatio-temporal dynamics of coupled human-landscape system. 2. Scenario-based application for impact assessment of land-use policies. Ecological Informatics 5 (3), 203-221. [doi:10.1016/j.ecoinf.2010.02.001]

MACY M and Sato Y (2010). The surprising success of a replication that failed. Journal of Artificial Societies and Social Simulation 13 (2), 9. https://www.jasss.org/13/2/9.html

MIODOWNIK D, Cartrite B and Bhavnani R (2010). Between replication and docking: "Adaptive agents, political institutions, and civic traditions" revisited. Journal of Artificial Societies and Social Simulation 13 (3), 1. https://www.jasss.org/13/3/1.html

MOSS S (1998). Critical incident management: An empirically derived computational model. Journal of Artificial Societies and Social Simulation 1 (4), 1. https://www.jasss.org/1/4/1.html

NAIVINIT W, Le Page C, Trébuil G and Gajaseni N (2010). Participatory agent-based modeling and simulation of rice production and labor migrations in Northeast Thailand. Environmental Modelling & Software 25 (11), 1345-1358. [doi:10.1016/j.envsoft.2010.01.012]

PAJARES J, Hernández-Iglesias C and López-Paredes A (2004). Modelling learning and R&D in innovative environments: A cognitive multi-agent approach. Journal of Artificial Societies and Social Simulation 7 (2), 7. https://www.jasss.org/7/2/7.html

POLHILL J G, Parker D, Brown D and Grimm V (2008). Using the ODD protocol for describing three agent-based social simulation models of land-use change. Journal of Artificial Societies and Social Simulation 11 (2), 3. https://www.jasss.org/11/2/3.html

RADAX W and Rengs, B (2010). 'Prospects and Pitfalls of Statistical Testing: Insights from Replicating the Demographic Prisoner's Dilemma'. Journal of Artificial Societies and Social Simulation 13(4),1. https://www.jasss.org/13/4/1.html.

RICHIARDI M, Leombruni R, Saam N and Sonnessa M (2006). A common protocol for agent-based social simulation. Journal of Artificial Societies and Social Simulation 9 (1), 15. https://www.jasss.org/9/1/15.html.

ROUCHIER J, Cioffi-Revilla C, Polhill J G and Takadama K (2008). Progress in model-to-model analysis. Journal of Artificial Societies and Social Simulation 11 (2), 8. https://www.jasss.org/11/2/8.html

TREIBIG C and Klugl F (2009). Elements of a documentation framework for agent-based simulation models. Cybernetics and Systems 40 (5), 441-474. [doi:10.1080/01969720902922459]

VALBUENA D, Verburg P, Veldkamp A, Bregt A K and Ligtenberg A (2010). Effects of farmers' decisions on the landscape structure of a Dutch rural region: An agent-based approach. Landscape and Urban Planning 97 (2), 98-110. [doi:10.1016/j.landurbplan.2010.05.001]

WILL O (2009). Resolving a replication that failed: News on the Macy & Sato model. Journal of Artificial Societies and Social Simulation 12 (4), 11. https://www.jasss.org/12/4/11.html.