Editorial: Meeting Grand Challenges in Agent-Based Models

Introduction

The use of agent-based models (ABMs)^[1] has increased rapidly among various scientific communities over the last two decades. According to an online survey, the number of articles reporting the development or use of ABMs has been steadily increasing in an exponential rate since the mid-1990s, ranging across such research fields as ecology, epidemiology, human-environment science, land systems science, sociology, and political science (An et al. 2017).

This increasing interest in and use of ABMs is rooted in a set of challenges humans face: Virtually all major current challenges of humankind involve complex systems, such as economies, political and social-ecological systems or financial markets, comprised of autonomous, decision-making agents, including animals, people, families, parties, or companies. Due to their complexity, researchers cannot handle these systems in a controlled way common to science. We need models that understand the internal organization and processes of these systems sufficiently to address, for example, sudden regime shifts and collapse. Statistical and mathematical models cannot fully capture many key features of complex systems: agents, including non-human organisms and social groups, among others, are different in that they interact not globally but locally or within complex networks, and they adapt their behavior to the current state of themselves and their spatially and temporally varying environment. Agent-based modeling (ABM) can capture these aspects and has therefore become a widely used tool.

ABM Concept

Agent-based modeling (ABM), or individual based modeling (IBM, named so largely by ecologists), is rooted on the fundamental philosophy of methodological individualism, which focuses on the uniqueness of individuals and interactions among them or between them and the associated environment(s). Technologically, agent-based modeling has emerged and prospered with the advent of increasingly available computing power and data and, a decade or so later, with the advent of object oriented programming and specific software packages for ABMs (Askenazi et al. 1996).

ABMs have intellectual origin from, and substantially contribute to, complexity science. Complexity science aims to understand complex systems, which often include heterogeneous subsystems, autonomous entities, nonlinear relationships, and multiple interactions among them (Arthur 1999; Axelrod & Cohen 1999; Crawford et al. 2005; Levin et al. 2013). ABM’s basic paradigm of representing the entities and subsystems as agents (often at various, often hierarchical levels) and employing flexible rules to mimic many complex relationships and interactions—just satisfies the needs of understanding complex systems. Such systems feature path-dependence, self-organization, difficulty of prediction, and emergence that cannot be inferred from system components and their attributes alone (Bankes 2002a; Manson 2001). Hence it is suggested that the ABM approach be employed to understand, harness, and improve (rather than fully control) the system’s structure and function, taking innovative actions to steer the system in beneficial directions (Axelrod & Cohen 1999).

ABMs have witnessed increasing adoption when studying social systems, ecological systems, and social-ecological systems (Eliassen et al. 2016; Eliassen et al. 2009; Giske et al. 2014; Grimm et al. 2006). A range of empirical studies also show that many social-ecological systems manifest complexity features: heterogeneity, reciprocal effects and feedback loops, nonlinearity and thresholds, surprising outcomes (observable as a result of human-nature couplings), legacy effects and time lags, and resilience (An et al. 2017; Egli et al. 2019; Grimm et al. 2005; Liu et al. 2007; Zvoleff & An 2014).

ABM is capable of addressing these complexity-related challenges because of its intrinsic capacity to incorporate individual-level (e.g., heterogeneous subsystems, autonomous entities) information, to allow for multiple nonlinear relationships and interactions such as feedbacks, learning, and adaptation, to account for spatially and temporally variable information, and to integrate cross-scale and cross-discipline data and methods (An et al. 2005; National Research Council 2014). Another key value of ABM is the ability to represent human behavior more realistically by accounting for bounded rationality, heterogeneity, agent-agent and agent-environment interactions, evolutionary learning and adaptation, among others (An 2012; Filatova et al. 2013; Groeneveld et al. 2017; National Research Council 2014; Parker et al. 2003).

Since the NAS Sackler Colloquium addressing the topic in 2001 (Bankes 2002a, 2002b), agent-based modeling has made major advances in many areas, including the establishment of platforms supporting user-friendliness, diversity, usability, and available open-source resources (e.g., https://www.comses.net/). Because the ABM methodology explores dynamic paths, ABMs are especially useful when the processes under investigation involve abrupt changes, crises, and critical transitions related to social interactions and adaptive behavior, answering many “what—if” questions that shed light into the system’s paths or trajectories under given conditions. Therefore, according to National Academy of Science, ABMs are useful at the intervention design stage because they provide a means for projecting possible effects of policies or decisions ex ante (National Research Council 2014).

ABM Challenge

Agent based models were greeted with enthusiasm initially (Bankes 2002a; Huston, DeAngelis, & Post 1988), but this response faded quickly: “Scientists sometimes tend to rush to a new approach that promises to solve previously intractable problems, and then revert to familiar techniques as the unanticipated difficulties of the new approach are uncovered” (Grimm & Railsback 2005, p. xi). Critiques of ABMs asked for the identification of outcomes that differ from or are better than other types of models and for validation of ABMs. Interestingly, such responses are not always asked of other, non-ABM model types. A number of difficulties (many of them are common to any type of modeling, not necessarily unique to ABM), in turn, may account for, at least partially account for, some hesitance, misuse, misunderstanding, or doubt about ABM (Couclelis 2002; Roughgarden 2012).

A number of challenges regarding ABM still warrant in-depth exploration and research, however, including difficulties in communication of ABMs due to lack of common standards or protocols (with a few exceptions, e.g., Grimm et al. 2006, 2010), model verification and validation, and telling signal from noise in model structure and output (Grimm & Berger 2016). ABM challenges also entail making model design and analysis coherent and efficient instead of ad hoc, and identifying general principles underlying the systems’ internal organization, steep learning curves for non-modeling experts (novices in particular), difficulty to scale ABM models or findings from one level to another, high sensitivity to detail (the other side of the problem is ABM’s data hunger) as well as stochastic elements, alternative decision models, and representation of spatial structure (e.g., An 2012; An et al. 2005; O’Sullivan et al. 2016; Parker et al. 2003). Equally important, ABMs are largely developed either on a PC (for exceptions see Tang et al. 2011; Tang and Bennett 2011) or on an ad hoc basis with substantial variation in platforms, programming languages, model details and sophistication, and the modeler’s preferences. These variations reduce ABM’s capacity to facilitate high performance computing and handle big data (especially spatially and potentially temporally explicit data). According to the feedback we received from an NSF-sponsored conference (An et al. 2018), the following challenges are of particular attention.

1. Modeling human decisions, especially decisions regarding their interaction with the environment. There is increasing evidence that this topic is of prime interest. For instance, one review paper about modeling human decisions using ABMs (An 2012) has become highly cited in the past 7 years (547 citations as of December 28, 2019) according to Google Scholar. A more recent publication (Groeneveld et al. 2017) shows that most models of land use/land cover change use simple utility functions to represent decision making and that, while sometimes more sophisticated representations are used, in none of the 134 reviewed article alternative decision models were compared. However, such comparisons are needed to select and develop the most flexible and predictive decision models (“pattern-oriented theory development”, Railsback & Grimm 2019; see also the framework for systematic comparison and development of human decision models suggested by Schlüter et al. 2017).
2. ABM transparency and reusability. These issues have been mentioned as one of the bottleneck problems for the ABM community (An et al. 2014; National Research Council 2014; Parker et al. 2003), even though important work suggests advances (e.g., the COMSES node at www.comses.net). Without adequate transparency and reusability, it is not only difficult to verify and validate ABMs, but also wastes a large amount of resources, such as modules and programming libraries that have been developed and tested by ABM experts. When ABMs are largely not reusable, non-transparent, and difficult to be validated, it is challenging, if not impossible, to compare ABMs from different sites or applications and to generalize commonalities out of locale-specific details. As a result, the usefulness of ABM in hypothesis testing and theory formulation is reduced (An et al. 2014; Rindfuss et al. 2008). Correspondingly, it is sometimes problematic to convince people what insights may come uniquely from ABMs instead of traditional equation-based models, such as various types of regression models. Currently, the ODD protocol (Grimm et al. 2006, 2010; Polhill 2010; Polhill et al. 2008^[2]) is the most widely used standard format for describing ABMs, which is also designed to facilitate replication; it has already contributed to integrating research from different disciplines using ABMs (Vincenot 2018). Likewise, the benefits of starting a modelling project from the replication of existing models, rather than starting from scratch, are increasingly acknowledged (Thiele & Grimm 2015; Hauke et al. 2020).
3. Validation of ABM. This has been a problem besetting ABM modelers and users for many years (An et al. 2005; National Research Council 2014; Parker et al. 2003), and many doubts arise from this difficulty (Couclelis 2002; Roughgarden 2012). Without robust model validation, the reliability of ABM cannot be established, limiting its usefulness and application in various contexts. Pattern-oriented modelling has been suggested as a strategy that allows us to compare model output to multiple, instead of a single, patterns, observed a different scales and levels of organization. Each pattern is used to reject unrealistic process representations or parameter combinations (Grimm et al. 2005; Railsback & Grimm 2019). By this multi-criteria design and testing, the structural realism of ABMs is increased and trust in the model’s validity is also increased.
4. ABM Software and Big data ABM. This emphatic topic comes from the increasing availability of big data, such as high resolution remote sensing imagery and large detailed, human socioeconomic datasets (Wang et al. 2013). Currently ABMs are largely based on relatively small or local scales, limiting the usefulness of ABM in large (spatial extent) and high-resolution contexts. A small set of exceptions, however, exists in parallel computing, (Tang & Bennett 2010b, 2010a; Tang et al. 2011).
5. ABM theories. We need testable, generative theories to understand how complex systems patterns and dynamics may emerge. Intuitively, buffer and recovery mechanisms based on diversity, heterogeneity, and adaptation are key to sustained functioning of systems, but it is challenging to quantify and understand their interaction (Egli et al. 2019).

A key to the further advancement of the theory of agent-based complex systems is a systematic development of theories of human behavior. It is critical to distinguish between imposed behaviors, which are based on empirical and thus not transferable rules, and emergent behaviors, which emerge from first principles underlying each agent’s decision making in response to changing conditions. While in ecology such first principles, such as fitness seeking and energy budgets, are increasingly used, corresponding principles still need to be identified for human behavior. A promising avenue for future research in this field is acknowledging the context-dependency of human behavior. In panic reaction, humans can be represented as “Brownian agents”, i.e. like physical particles, in other situations simple utility functions may be suitable, while in “hedonic modelling”, emotions such as fear are the central concept (Eliassen et al. 2016). Only if we are getting better in capturing the emergence and context dependency of human behavior will we be able to better understand and predict the dynamics of agent-based complex systems and develop theories for these systems.

This Special Issue

The aforementioned challenges regarding ABM may lead to inadequate or inappropriate ABM uses, constraining the advancement of the science in question if not addressed with high priority in ABM research. This special issue features three standalone, yet interrelated goals: (1) defining complex systems as agent-based systems, the object of a new, much-needed generic systems theory; (2) summarizing the generic features of such systems and the key questions about them; and (3) providing case studies where these features are relevant and these questions are addressed using agent-based modeling.

Ligmann-Zielinska et al. (2020) offer a comprehensive overview of sensitivity analysis, providing a roadmap for ABM modelers or users to choose the most appropriate methods when performing sensitivity analysis. The methods for sensitivity analysis may vary, depending on whether the goal is to identify a proof of concept, to produce predictive outcomes (or to increase decision-making capacity), or to conduct exploratory modeling and gain insight into the effects of complexity features on a complex system of interest. Despite increasing use of ABM in many disciplines, broader adoption is hindered by a set of methodological and conceptual challenges. This context explains well the impetus of Manson et al. (2020) to explore methodological issues of spatial ABMs (SABM): space and time representation, scale and space, predictive or explanatory use of ABM, balance between model simplicity and complexity, qualitative modeling and collaboration, and the role of ABM in advancing theory or generative science. Manson et al.(2020) also provide a summary of common platforms that facilitate SABM development. On the other hand, An et al. (2020) provide an empirical study of employing ABM to integrate data and models from various domains (bearing different spatial and temporal scales by nature) in an agent-based complex system. With simulation of human and monkey activities, An et al. (2020) performed ABM-based experiments to examine the social and ecological impacts of a conservation policy, showing a surprising outcome that cannot be explained by common statistical or equation-based models. Another empirically calibrated ABM by Tang & Yang (2020) seeks to identify space-time locations of land developments at critical thresholds of water quality in eight North Carolina counties. In this model, land developers interact with land owners and decide where and how to develop land parcels, producing complex landscape patterns that drive spatiotemporal patterns of water quality over space. The work by Hauke et al. (2020) shows that replicated simulation models can be used to develop theory. In a context where most agent-based models have not been replicated, it is often difficult to identify sources of information or insights in ABMs (e.g., from ad hoc assumptions or parameters or from theoretical insights). Applying the ODD (Overview, Design concepts, and Details) protocol and DOE (design of experiments) principles, the authors were able to develop experiments and generalize some theoretical insights built in a previous ABM, showing a promising use of ABM in generative science. The model by Dou et al. (2020) represents a new type of ABM—a systems simulation tool that quantifies the causes and effects of local land-use changes on distant locations using hierarchical modelling structure and the telecoupling framework. The model shows that potential subsidies from Brazilian government to local soybean farmers could reshape land uses in China through international soybean markets.

This special issue addresses ABM problems and challenges at a time in which the use of ABMs is exploding. It assembles seven exceptional teams (including modelers, users, and domain experts) to disentangle the ABM challenges and advance the ABM science. This special issue also aims to identify a set of impending tasks in several topical subareas, such as model testing and validation through sensitivity analysis (Ligmann-Zielinska et al. 2020) or replicability test (Meyer et al. 2020), ABM-enabled theory development (Manson et al. 2020; Hauke et al. 2020), modeling human decisions (An et al. 2020; Dou et al. 2020; Tang & Yang 2020), and ABM transparency and reusability (Hauke et al. 2020). We hope this endeavor will help prioritize specific resources and activities in relation to ABM advances, leading to coordinated, joint efforts and initiatives to advance the science and technology behind ABM. It is our sincere hope that all papers in this special issue help outline a clearer picture of ABM, including strengths, weaknesses, available resources, and future directions. The ABM science will advance more steadily when more users, developers, and commercial companies are engaged, allocating more resources to the science, technology, and application of ABM, enhancing ABM software and capabilities.

References

AN, L. (2012). Modeling human decisions in coupled human and natural systems: Review of agent-based models. Ecological Modelling, 229, 25–36. [doi:10.1016/j.ecolmodel.2011.07.010]

AN, L., Linderman, M., Qi, J., Shortridge, A., and Liu, J. (2005). Exploring complexity in a human-environment system: An agent-based spatial model for multidisciplinary and multiscale integration. Annals of the Association of American Geographers, 95(1), 54–79. [doi:10.1111/j.1467-8306.2005.00450.x]

AN, L., Mak, J., Yang, S., Lewison, R., Stow, D. A., Chen, H. L., Xu, W., Shi, L. and Tsai, Y. H. (2020). Cascading Impacts of Payments for Ecosystem Services in Complex Human-Environment Systems. Journal of Artificial Societies and Social Simulation, 23 (1), 5: https://www.jasss.org/23/1/5.html. [doi:10.18564/jasss.4196]

AN, L., Manson, S., Jankowski, P., Wang, S., and Turner II, B. L. (2017). (NSF proposal) ABM’17: The usefulness, uselessness, and impending tasks of agent-based models in social, human-environment, and life Sciences. Retrieved from https://www.nsf.gov/awardsearch/showAward?AWD_ID=1638446&HistoricalAwards=false

AN, L., Manson, S., Jankowski, P., Wang, S., and Turner II, B. L. (2018). Project final report of ABM’17: The usefulness, uselessness, and impending tasks of agent-based models in social, human-environment, and life Sciences. San Diego, CA.

AN, L., Zvoleff, A., Liu, J., and Axinn, W. (2014). Agent based modeling in coupled human and natural systems (CHANS): Lessons from a comparative analysis. Annals of Association of American Geographers, 104(4), 723–745. [doi:10.1080/00045608.2014.910085]

ARTHUR, W. B. (1999). Complexity and the economy. Science, 284(5411), 107–109.

ASKENAZI, M., Burkhart, R., Langton, C., and Minar, N. (1996). The Swarm simulation system: A toolkit for building multi-agent simulations (SFI Working Paper No. 96-06–042). Santa Fe, NM: Santa Fe Institute.

AXELROD, R., and Cohen, M. D. (1999). Harnessing Complexity: Organizational Implications of a Scientific Frontier. New York: The Free Press.

BANKES, S. C. (2002a). Agent-based modeling: A revolution? Proceedings of the National Academy of Sciences of the United States of America, 99(10), 7199–7200. [doi:10.1073/pnas.072081299]

BANKES, S. C. (2002b). Tools and techniques for developing policies for complex and uncertain systems. Proceedings of the National Academy of Sciences of the United States of America, 99 Suppl 3(10), 7263–7266. [doi:10.1073/pnas.092081399]

COUCLELIS, H. (2002). Why I no longer work with agents: A challenge for abms of human-environment interactions. In D. C. Parker, T. Berger, and S. M. Manson (Eds.), Meeting the Challenge of Complexity: Vol. 1.1 (pp. 14–16). Retrieved from http://www.csiss.org/events/other/agent-based/additional/proceedings.pdf.

CRAWFORD, T. W., Messina, J. P., Manson, S. M., and O’Sullivan, D. (2005). Complexity science, complex systems, and land-use research. Environment and Planning B: Planning & Design, 32(6), 792–798. [doi:10.1068/b3206ed]

DOU, Y., Yao, G., Herzberger, A, Bicudo da Silva, R. F., Song, Q, Hovis, C., Batistella, M., Moran, E., Wu, W. and Liu, J. (2020). Land-Use Changes in Distant Places: Implementation of a Telecoupled Agent-Based Model. Journal of Artificial Societies and Social Simulation 23(1), 11: https://www.jasss.org/23/1/11.html. [doi:10.18564/jasss.4211]

EDMONDS, B., Le Page, C., Bithell, M., Chattoe-Brown, E., Grimm, V., Meyer, R., Montañola-Sales, C., Ormerod, P., Root, H. and Squazzoni, F. (2019). Different modelling purposes. Journal of Artificial Societies and Social Simulation, 22(3), 6: https://www.jasss.org/22/3/6.html. [doi:10.18564/jasss.3993]

EGLI, L., Weise, H., Radchuk, V., Seppelt, R., and Grimm, V. (2019). Exploring resilience with agent-based models: State of the art, knowledge gaps and recommendations for coping with multidimensionality. Ecological Complexity, 40. [doi:10.1016/j.ecocom.2018.06.008]

ELIASSEN, S., Andersen, B. S., Jørgensen, C., and Giske, J. (2016). From sensing to emergent adaptations: Modelling the proximate architecture for decision-making. Ecological Modelling, 326, 90–100. [doi:10.1016/j.ecolmodel.2015.09.001]

ELIASSEN, S., Jørgensen, C., Mangel, M., Giske, J., and Whitlock, A. E. and E. M. C. (2009). Quantifying the Adaptive Value of Learning in Foraging Behavior. The American Naturalist, 174(4), 478–489. [doi:10.1086/605370]

FILATOVA, T., Verburg, P. H., Parker, D. C., and Stannard, C. A. (2013). Spatial agent-based models for socio-ecological systems: Challenges and prospects. Environmental Modelling & Software, 45, 1–7. [doi:10.1016/j.envsoft.2013.03.017]

GISKE, J., Eliassen, S., Fiksen, Ø., Jakobsen, P. J., Aksnes, D. L., Mangel, M., and Jørgensen, C. (2014). The emotion system promotes diversity and evolvability. Proc. R. Soc. B, 281(1791), 20141096. [doi:10.1098/rspb.2014.1096]

GRIMM, V., and U. Berger (2016). Robustness analysis: Deconstructing computational models for ecological theory and applications. Ecological Modeling, 326, 162-167. [doi:10.1016/j.ecolmodel.2015.07.018]

GRIMM, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S., Huse, G., Huth, A., Jepsen, J. U., Jørgensen, C., Mooij, W. M., Müller, B., Pe'er, G., Piou, C., Railsback, S. F., Robbins, A. M., Robbins, M. M., Rossmanith, E., Rüger, N., Strand, E., Souissi, S., Stillman, R. A., Vabø, R., Visser, U., and DeAngelis, D. L. (2006). A standard protocol for describing individual-based and agent-based models. Ecological Modelling, 198(1), 115–126. [doi:10.1016/j.ecolmodel.2006.04.023]

GRIMM, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J., and Railsback, S. F. (2010). The ODD protocol: A review and first update. Ecological Modelling, 221(23), 2760–2768. [doi:10.1016/j.ecolmodel.2010.08.019]

GRIMM, V., and Railsback, S. F. (2005). Individual-based Modeling and Ecology. Princeton, NJ: Princeton University Press.

GRIMM, V., Railsback, S. F., Vincenot, C. E., Berger, U., Gallagher, C., DeAngelis, D. L., Edmonds, B., Ge, J., Giske, J., Groeneveld, J., Johnston, A. S. A., Milles, A., Nabe-Nielsen, J., Polhill, G., Radchuk, V., Rohwäder, M.-S., Stillman, R. A., Thiele, J. C. and Ayllón, D. (2020). The ODD Protocol for Describing Agent-Based and Other Simulation Models: A Second Update to Improve Clarity, Replication, and Structural Realism. Under review. [doi:10.1016/j.ecolmodel.2010.08.019]

GRIMM, V., Revilla, E., Berger, U., Jeltsch, F., Mooij, W. M., Railsback, S. F., Thulke, H.-H., Weiner, J., Wiegand, T., and DeAngelis, D. L. ( (2005). Pattern-oriented modeling of agent-based complex systems: Lessons from ecology. Science, 310(5750), 987–991. [doi:10.1126/science.1116681]

HAUKE, J., Achter, S. and Meyer, M. (2020) Theory Development Via Replicated Simulations and the Added Value of Standards. Journal of Artificial Societies and Social Simulation 23(1),12: https://www.jasss.org/23/1/12.html. [doi:10.18564/jasss.4219]

GROENEVELD, J. J., Müller, B., Buchmann, C. M., Dressler, G., Guo, C., Hase, N., Hoffmann, F., John, F., Klassert, C., Lauf, T., Liebelt, V., Nolzen, H., Pannicke, N., Schulze, J., Weise, H., and Schwarz, N. (2017). Theoretical foundations of human decision-making in agent-based land use models – A review. Environmental Modelling & Software, 87, 39–48. [doi:10.1016/j.envsoft.2016.10.008]

HUSTON, M., DeAngelis, D., and Post, W. (1988). New computer models unify ecological theory. BioScience, 38(10), 682–691. [doi:10.2307/1310870]

LEVIN, S., Xepapadeas, T., Crépin, A.-S., and Norberg, J. (2013). Social-ecological systems as complex adaptive systems: Modeling and policy implications. Environment and Development Economics, 18(2), 111–132. [doi:10.1017/s1355770x12000460]

LIGMANN-ZIELINSKA, A., Siebers, P.-O., Magliocca, N., Parker, Dawn C., Grimm, V., Du, J., Cenek, M., Radchuk, V., Arbab, N. N., Li, S., Berger, U., Paudel, R., Robinson, D. T., Jankowski, P. and An, L. (2020) 'One Size Does Not Fit All': A Roadmap of Purpose-Driven Mixed-Method Pathways for Sensitivity Analysis of Agent-Based Models. Journal of Artificial Societies and Social Simulation 23 (1), 6: https://www.jasss.org/23/1/6.html. [doi:10.18564/jasss.4201]

LIU, J., Dietz, T., Carpenter, S. R., Alberti, M., Folke, C., Moran, E., Pell, A.N., Deadman, P., Kratz, T., Lubchenke, J., Ostrom, E., Ouyang, Z., Provencher, W., Redman, C.L., Schneider, S.H., and Taylor, W. W. (2007). Complexity of coupled human and natural systems. Science, 317(5844), 1513–1516. [doi:10.1126/science.1144004]

MANSON, S. M. (2001). Simplifying complexity: A review of complexity theory. Geoforum, 32(3), 405–414. [doi:10.1016/s0016-7185(00)00035-x]

MANSON, S., An, L., Clarke, K. C., Heppenstall, A., Koch, J., Krzyzanowski, B., Morgan, F., O'Sullivan, D., Runck, B. C., Shook, E. and Tesfatsion, L. (2020) Methodological Issues of Spatial Agent-Based Models. Journal of Artificial Societies and Social Simulation 23(1), 3: https://www.jasss.org/23/1/3.html. [doi:10.18564/jasss.4174]

MEYER, M., Hauke, J. and Achter, S. (2020) Theory Development Via Replicated Simulations and the Added Value of Standards. Journal of Artificial Societies and Social Simulation 23(1),12: https://www.jasss.org/23/1/12.html. [doi:10.18564/jasss.4219]

NATIONAL Research Council. (2014). Advancing Land Change Modeling: Opportunities and Research Requirements. Washington, DC: The National Academies Press.

O’SULLIVAN, D., Evans, T., Manson, S., Metcalf, S., Ligmann-Zielinska, A., and Bone, C. (2016). Strategic directions for agent-based modeling: Avoiding the YAAWN syndrome. Journal of Land Use Science, 11(2), 177–187. [doi:10.1080/1747423x.2015.1030463]

PARKER, D. C., Manson, S. M., Janssen, M. A., Hoffmann, M. J., and Deadman, P. (2003). Multi-agent systems for the simulation of land-use and land-cover change: A review. Annals of the Association of American Geographers, 93(2), 314–337. [doi:10.1111/1467-8306.9302004]

POLHILL, J. G. (2010). ODD updated. Journal of Artificial Societies and Social Simulation, 13(4), 9. [doi:10.18564/jasss.1700]

POLHILL, J. G., Parker, D., Brown, D., and Grimm, V. (2008). Using the ODD protocol for describing three agent-based social simulation models of land-use change. Journal of Artificial Societies and Social Simulation, 11(2), 3: : https://www.jasss.org/11/2/3.html [doi:10.18564/jasss.1563]

RAILSBACK, S. F., and Grimm, V. (2019). Agent-Based and Individual-Based Modeling: A Practical Introduction. 2^nd Edition. Princeton, New Jersey: Princeton University Press.

RINDFUSS, R. R., Entwisle, B., Walsh, S. J., An, L., Badenoch, N., Brown, D. G., … Verburg, P. H. (2008). Land use change: Complexity and comparisons. Journal of Land Use Science, 3(1), 1–11.

ROUGHGARDEN, J. (2012). Individual based models in ecology: An evaluation, or how not to ruin a good thing. Presented at the Philosophy of Science Association Biennial Meeting, San Diego, CA. Retrieved from http://philsci-archive.pitt.edu/9434/1/RoughgardenPSA2012IBMLecture.pdf.

SCHLÜTER, M., Baeza, A., Dressler, G., Frank, K., Groeneveld, J., Jager, W., … Wijermans, N. (2017). A framework for mapping and comparing behavioural theories in models of social-ecological systems. Ecological Economics, 131, 21–35. [doi:10.1016/j.ecolecon.2016.08.008]

TANG, W., and Bennett, D. A. (2010a). Agent-based modeling of animal movement: A review. Geography Compass, 4(7), 682–700. [doi:10.1111/j.1749-8198.2010.00337.x]

TANG, W., and Bennett, D. A. (2010b). The explicit representation of context in agent-based models of complex adaptive spatial systems. Annals of the Association of American Geographers, 100(5), 1128–1155. [doi:10.1080/00045608.2010.517739]

TANG, W. and Yang, J. (2020). Agent-Based Land Change Modeling of a Large Watershed: Space-Time Locations of Critical Threshold. Journal of Artificial Societies and Social Simulation 23(1),15: https://www.jasss.org/23/1/15.html. [doi:10.18564/jasss.4226]

TANG, W., Wang, S., Bennett, D. A., and Liu, Y. (2011). Agent-based modeling within a cyberinfrastructure environment: A service-oriented computing approach. International Journal of Geographical Information Science, 25(9), 1323–1346. [doi:10.1080/13658816.2011.585342]

THIELE, J. C., and Grimm, V. (2015). Replicating and breaking models: Good for you and good for ecology. Oikos, 124(6), 691–696. [doi:10.1111/oik.02170]

VINCENOT, C. E. (2018). How new concepts become universal scientific approaches: Insights from citation network analysis of agent-based complex systems science. Proceedings of the Royal Society B: Biological Sciences, 28(1874), 20172360. [doi:10.1098/rspb.2017.2360]

WANG, S., Anselin, L., Bhaduri, B., Crosby, C., Goodchild, M. F., Liu, Y., and Nyerges, T. L. (2013). CyberGIS software: A synthetic review and integration roadmap. International Journal of Geographical Information Science, 27(11), 2122–2145. [doi:10.1080/13658816.2013.776049]

ZVOLEFF, A., and An, L. (2014). The effect of reciprocal connections between demographic decision making and land use on decadal dynamics of population and land use change. Ecology and Society, 19(2), 31. [doi:10.5751/es-06243-190231]