Abstract

This article reports on an agent-based simulation of public participation in decision making about sustainability management. Agents were modeled as socially intelligent actors who communicate using a system of symbols. The goal of the simulation was for agents to reach consensus about which situations in their regional environment to change and which ones not to change as part of a geodesign process for improving water quality in the greater Puget Sound region. As opposed to studying self-organizing behavior at the scale of a local 'commons', our interest was in how online technology supports the self-organizing behavior of agents distributed over a wide regional area, like a watershed or river basin. Geographically-distributed agents interacted through an online platform similar to that used in online field experiments with actual human subjects. We used a factorial research design to vary three interdependent factors each with three different levels. The three factors included 1) the social and geographic distribution of agents (local, regional, international levels), 2) abundance of agents (low, medium, high levels), and 3) diversity of preconceptions (blank slate, clone, social actor levels). We expected that increasing the social and geographic distribution of agents and the diversity of their preconceptions would have a significant impact on agent consensus about which situations to change and which ones not to change. However, our expectations were not met by our findings, which we trace all the way back to our conceptual model and a theoretical gap in sustainability science. The theory of self-organizing resource users does not specify how a group of social actors' preconceptions about a situation is interdependent with their social and geographic orientation to that situation. We discuss the results of the experiment and conclude with prospects for research on the social and geographic dimensions of self-organizing behavior in social-ecological systems spanning wide regional areas.

Keywords:

Social Actors, Public Participation, Decision Making, Sustainability Management, Geodesign, Geographic Information Systems (GIS)

The Three Domains of Sustainability: Sustainability Science, Sustainability Information Science, and Sustainability Management

1.1

A widely accepted theory is that when people are left to their own devices they will simply consume the resources at their disposal and deteriorate their environment unless governments impose a control system to prevent an unavoidable tragedy of the commons. One of the interesting developments in sustainability science is the emergence of an alternate theory. Based on extensive case studies, sustainability science finds that human resource users may sometimes self-organize as a control system to make sure that the social-ecological system of which they are a dependent part remains resilient in a way they prefer (Ostrom 2009; Agrawal 2001). Systems theorists have long held that complex systems exhibit the capacity to evolve internal control systems and essentially self-regulate (Bennet and Chorley 1978). In fact, sustainability science and environmental history have both shown through historical case studies that government intervention into the self-organizing capabilities of an otherwise resilient social-ecological system sometimes accelerates its deterioration (Earle 1988).

1.2

Ostrom (2009) highlighted ten subsystem variables explaining self-organizing behavior leading to a sustainable social-ecological system. Of particular interest to us are four subsystem variables that scale the social and geographic dimensions of a decision making situation in different ways. The four variables of interest include the size of the resource system, the number of users, the amount of knowledge sharing among different resource user's mental models, and finally, the level of importance of the resource to each user. The probability that a group of resource users will self-organize as a control system is higher when these subsystem variables fall within a certain range. For example, it is unlikely that resource users will self-organize in systems spanning very large areas because of the burdens of managing extensive flows of resources. On the other hand, it is also unlikely that resource users will self-organize over very small areas that typically do not generate flows of substantial value. In sum, sustainability science holds that given the size of the area and the number of resource users, the more that resource users are able to share their mental models about the preferred attributes of the system of which they are a dependent part, and the more important the resource or ecosystem service is to the users themselves, the more likely a set of resource users will invest time and energy in managing the attributes of their system to maintain a preferred state or identity.

Figure 1. Three knowledge domains of sustainability

1.3

When it comes to the actual design and testing of information technology platforms to support self-organizing behavior among geographically-distributed resource users, one can turn to sustainability information science. Figure 1 illustrates the three knowledge domains of sustainability including science, technology, and management (Kates et al. 2001; Clark 2007). While not meant to be mutually exclusive, these domains describe three kinds of expertise at work in assessing what is known about a social-ecological system, and deciding whether to intervene. Assessment and intervention is an organizational process with multiple roles and feedback loops. However, at minimum the process involves at least three activities including measuring the properties of real-life elements of a social-ecological system; secondly, processing those measurements into information and then reasoning about relationships between elements to explain the apparent character, state, or identity of a system as a whole; and then thirdly, generating an informed understanding or consensus about how to manage certain relationships so as to ensure that the preferred attributes of the system as a whole remain resilient to disturbance and change over long periods of time. Ideally, an organizational process of assessment and intervention involves public participation and takes into consideration all affected parties.^[1] Regardless of whether or not the term sustainability is embraced, a set of activities aimed at changing an existing situation into a more preferred one that includes future generations as affected parties, so as to "meet present needs without compromising the ability of future generations to meet their needs," represents a special class of geodesign work we call sustainability management (WECD 1987).

1.4

The practice of sustainability involves at least three overlapping work activities to describe, assess, and manage the resilience of a social-ecological system (Walker and Salt 2012). On the one hand, describing a system requires conceptual work and literacy with the most enduring ideas about sustainability and resilience (e.g., see Agrawal 2001; Beisner et al. 2003; Cumming 2011; Liu et al. 2007; 2007, 2009). We call this expertise in the domain of sustainability science. On the other hand, work activities spent managing a system require a host of skills ranging from performing technology-supported work to displaying personal and professional competencies working in an organizational setting like a public agency. We call this expertise in the domain of sustainability management. In between these two lies a special body of knowledge focused on the design, testing and implementation of geospatial information capable of modeling a social-ecological system inside of a computer in order to better represent the potential consequences of changing existing situations into more preferred ones (e.g., Kersten et al. 2000; Hilty et al. 2005; Campagna 2006; NRC 2005; Klinsky et al. 2010; NRC 2012). We call this last body of knowledge expertise in the domain of sustainability information science.

1.5

One of the dilemmas faced by experts in sustainability information science is that providing information for a decision making situation can sometimes do more harm than good, drowning people in a sea of information or generating conflicts and confusion because the information provided does not match preexisting conceptions hardened by exposure to different information (NRC 1996; NRC 2005). These challenges were what led Herbert Simon (1976, 1981) to call for a science of information processing and a science of the artificial. Simon sought a general set of relations determining success or breakdown in any workflow mixing two very different kinds of information processors, i.e., people with different levels of expertise on the one hand, and computers on the other. In addition to extensive arguments in favor of agent-based modeling, Simon's calls for research have inspired work on social intelligence, human-computer-human-interaction, and social-computational systems. Interest in what has been called participatory geographic information science (Jankowski and Nyerges 2001) has been similarly motivated. Over the past decade, researchers in participatory geographic information science have tried to understand how large groups of people can use geographic information technology to address existing spatial problems and improve future well-being in decision making situations allocating public funds for land use, transportation, and water resource management (Nyerges and Jankowski 2010). Likewise, geodesign has emerged as a way of thinking about how to integrate GIS and methods like agent-based modeling to provide information about changing an existing situation into a preferred one, where the spatial scale of interest spans beyond neighborhoods and urban growth areas to watershed and basins (Steinitz 2012).

1.6

Enhancing overlaps between the three domains of sustainability is a practical goal. Practitioners of sustainability management regardless of their chosen substantive area should be well-versed in the methods of sustainability information science and the concepts of sustainability science. To that end there are now professional graduate programs like the Professional Master's Program in GIS (PMPGIS) for sustainability management at the University of Washington. The Professional Master's Program in GIS for sustainability management at the University of Washington takes the greater Puget Sound region as a large-scale field laboratory or "commons" to explore the use of methods like agent-based modeling for sustainability science and sustainability management. Speaking about the drivers, pressures, state, impact, response (DPSIR) conceptual framework, the Washington State Academy of Sciences (2012) recently stated, "If the millions of people in the Puget Sound region could be represented by one individual—or one collective mind—then the assumptions that underpin the DPSIR model might be a realistic representation of interactions between humans and the environment …. Human communities, however, are not simply the sum of atomistic individuals … [and] no simple model can map societal characteristics on environmental pressures." In response to sentiments like that above posed by the Washington State Academy of Sciences, we integrated GIS with an agent-based model to simulate how self-organizing behavior might emerge among a socially and geographically diverse set of agents from the greater Puget Sound region using a geodesign platform.

1.7

The remainder of the paper proceeds as follows. In Section 2 we describe the properties of agents and the agent-based model of public participation in a geodesign decision making process. In Section 3 we present our factorial research design calling for 27 experimental treatments varying the social and geographic distribution of agents, the number of agents, and the diversity of agent preconceptions. In Section 4 we present our findings from 18 of the 27 originally planned treatments. We conclude in Section 5 with future prospects for design, testing, and implementation of agent-based modeling and online platforms in the study and enabling of self-organizing behavior among social actors given a common resource area.

Modeling an Agent Object for Public Participation in Decision Making

2.1

Our interest in agent-based modeling comes from having worked with actual human subjects in two field experiments, one concerning regional transportation planning in the central Puget Sound region and the other the regional impacts of global climate change on the Oregon coast. Essentially, an experimental research design involving hundreds or thousands of human subjects repeated over a widely-distributed area would be impossible. Esri's Agent Analyst is particularly interesting for future educational purposes given its integration with ArcGIS using a middleware approach and a programming language called Not Quite Python (Brown et al. 2005; Johnston 2013). However, for the simulation in this article we chose a Java-based application called AnyLogic based on our impressions of its customer and technical support for new users, a thorough test of its graphical user interface and functionality, and the fact that it was promoted as one of the only systems designed to work with GIS software and external databases while supporting system dynamics, discrete-event, and agent-based modeling.

2.2

We began the process of designing and building a simulation by considering a single common-sense narrative statement:

People make decisions about substantive things, such as courses of action aimed at changing existing situations into sustainable ones, through a process of participatory group interaction.

Similar to semantic modeling or entity-relationship modeling, we proceeded by parsing the narrative statement above into basic entities and relationships. For example, any general or abstract noun that functions as a subject, object, or part of a noun phrase could describe a class of entity or relationship. Verbs, adjectives, and other parts of speech could describe actions or states of entities and relationships. We made abstract words describing real-world entities more "specific" by distinguishing substantively relevant classes or subtypes, and more "concrete" by giving entities properties or attributes based on a realistic domain of values. An important caveat in conceptual modeling is that when carried to logical extremes, making elements and relationships more specific and concrete does not necessarily result in a more realistic computational simulation, particularly when it comes to modeling complex systems. A pragmatic approach based on a simple linear model may produce acceptable results when compared with reality, even when the entities and relationships in the model do not faithfully represent what we would assume to be the true complexity of the entities and relationships in the system under investigation (Bennet and Chorley 1978).

2.3

Parsing the sentence above into its component parts of speech suggested five principal entities or relationships to consider for the agent-based model, outlined in bold:

2.4

The first entity to consider is "people," the subject of the sentence, which we distinguish as social actor entities with different mental models. Taking the words "make decisions," the main verb and its object, social actor entities use their mental models to think, learn, and make decisions through a process of analysis and deliberation using symbols of communication. The words "substantive things," a noun phrase right after the main verb, represents what social actor entities are thinking, learning, or making decisions "about" through their use of symbols, referring to any set of real-life entities and relationships composing a situation within the social-ecological system of which the social actor itself is a component part. For human social actors, a situation represents any real-life social-ecological relationship to which that social actor also has a certain social and geographic orientation or stake. Social actors may be direct users or harvesters of some tangible resource produced by a situation, or they may be an indirect beneficiary of an intangible resource, ecosystem service, or social savings produced by a situation. The fourth potential element of the simulation comes from the words "a process of participatory group interaction," another noun phrase. The participatory group process was modeled as agents filtering, sorting, and reasoning about each other's use of symbols through an online platform specifically developed to support the six-step process typically convened in geodesign (Steinitz 2012). The fifth and last element comes from the words "in a spatial and temporal context," a noun phrase we added at the end in curly brackets, in part to simply cover everything else but also as a way of justifying use of a simulated client-server event log as our primary set of observations, as described in Aguirre and Nyerges (2011) and Nyerges and Aguirre (2011).

Figure 2. An agent active object class whose properties, states, and behaviors are implemented in AnyLogic as parameters, plain variables, Java collections, state charts, action charts, functions, or presentations.

Figure 3. An example of a state chart, in UML for Real Time (UML-RT), used to implement agent states and transitions.

Figure 4. An example of an AnyLogic action chart used to implement agents interactions with symbols.

2.5

After parsing a narrative statement into model elements, we implemented social actor agents as an active object class in AnyLogic. Within that active object class we defined agent properties, states and behaviors using the software features of AnyLogic including parameters, plain variables, Java collections, state charts, action charts, functions, and presentations (see AnyLogic 2013). There are a number of standards for documenting an agent-based model to ensure its reproducibility. Such standards include entity-relationship diagrams, Unified Modeling Language (UML) diagrams, various other object-oriented (OO) diagramming techniques, and the Overview, Design concepts, and Details (ODD) protocol for agent-based models (Grimm et al. 2010; Polhill 2008). For matters of ease of production and detail, we documented the physical implementation of the model itself with the documentation tools available in AnyLogic. The AnyLogic documentation tools list the complete descriptions of all model elements, e.g., parameters, plain variables, Java collections, state charts, action charts, functions, graphics, etc., in PDF, DOCX, or HTML form for ease of distribution.

2.6

Figure 1 is a schematic representation describing how agents were implemented in AnyLogic as an active object class. State charts were modeled using computable Unified Modeling Language for Real Time (UML-RT) diagrams. Figure 2 is an illustrative example of the UML-RT state chart used to specify and implement agent behavioral states and rules for transitions between states during the simulation. For instance, in Figure 2, after an agent transitions from a state of being logged in to the online platform (state "A"), to being active (state "A1"), to being ready to create deliberative content in the form of a vote, post, or reply (state "A1b" as marked with an asterisk); consequently they enter an action chart that determines what kind of deliberative behavior they will likely take. Action charts are structured programming blocks that implement code snippets using graphical Java operators. Figure 3 is an example of an action chart implementing voting behavior for a social actor agent operating in an "executive (EX)" mental model, which itself was implemented as a Java collection. In the action chart in Figure 3, there is an equal chance the agent will either vote in favor of situations that best match their preconceptions, or vote against those that least match their preconceptions. Further examples in the paper provide illustrative examples of agent object voting behavior, whereas full details about state chart transition rules and action chart algorithms used in the simulation are available in our model documentation.

Research Design for a Simulated Online Field Experiment

3.1

A prime concern in experimental research is limiting the number of variables being considered all at once. For example, in a factorial research design the number of different treatments required equals the cross-product of the number of interdependent factors being considered. Based on the theory of self-organizing behavior in sustainability science, we took four subsystem variables of interest including size of the resource system, the number of users, the amount of knowledge sharing among different resource user's mental models, and the level of importance of the resource to each user and then developed three simple sets of agent-based properties:

• Social & Geographic Properties: Agents have a certain social and geographic orientation to situations in their environment
• Conceptual Properties: Agents carry preconceptions organized into mental models, which they use to reason about situations in their environment
• Symbolic Properties: Agents are socially intelligent and can communicate their preconceptions to one another using a system of symbols

3.2

Each set of properties were further categorized into three levels and a number of qualifications had to be made when it came to implementing the properties of agent objects in a relational database integrated with the agent-based model (Figure 6), explained in more detail below. Thus using a factorial research design, after cross-tabulating three interdependent factors each with three different levels the result was 27 experimental treatments, not including parameter variation experiments and replication experiments to evaluate random effects.

Social & Geographic Properties of Agents

3.3

Our first task was to create a population of agents with social and geographic properties and then set target values for recruiting a certain number of these agents (low, medium, and high) from within the boundaries of regional areas representing a resource system (local, regional and international). We established the boundaries representing local, regional, and international areas using a combination of political jurisdictions and drainage areas and then used ArcGIS to generate a population of potential agents in Washington State and British Colombia, Canada (Figure 4). The local scale for the simulation was an area formed by the nine counties intersecting the watersheds of the greater Puget Sound region of Washington State, including the City of Seattle and King County, encompassing 228 strata (ZCTAs) with a population of 3.7 million people in the year 2000. The regional scale for the simulation was an area created by the 85 major watersheds (areas conforming to an 8-digit HUC, or USGS hydrologic unit code, and Canadian equivalents) contributing to the water body defined as the Salish Sea, which encompassed 804 strata (ZCTAs and CSDs) with a total population of 7.1 million. Finally, the international scale was Washington State and British Columbia, encompassing 1423 strata (ZCTAs and CSDs) with a total population of 9.8 million. To represent a population of agents we used counts from the most easily available year, the year 2000, enumerated in zip code tabulation areas (ZCTAs) in the United States and census subdivisions (CSDs) in Canada. We then used the centroids of each ZCTA and CSD as the coordinate location for each agent object instance, in the same way we used self-reported zip code information to represent the location of human subjects in prior experiments (Nyerges and Aguirre 2011; Aguirre and Nyerges 2011). Lastly, we set target values for low, medium, and high numbers of agents at approximately 25, 100, and 1000, respectively.

Figure 5. Maps illustrating the three different scales of agent distribution (local, regional, international) used in the experiment.

Figure 6. Map showing a detailed view of the regional scale of the simulation. The gray area represents coastal and fluvial drainage basins emptying into to the water body defined as the Salish Sea. The total population of each geographic strata (ZCTAs and CSDs) available for sampling are represented as proportional size symbols. Affected party preferences of social actors are represented as a color range from blue (more oriented to the coast) to red (less oriented to the coast).

3.4

Gastil et al. (2007) suggest using a Citizen Jury recruitment strategy using small, randomly-selected groups as representative of larger populations (see also Ferguson 2007). The Jefferson Center (2009) similarly used randomly sampled participants as representative groups on the basis of demographic characteristics. Other authors advocate non-randomly sampled groups of participants, pointing out from somewhat anecdotal evidence that participation worked best when participants were nominated by their community to represent their preferences or beliefs (Carson and Martin 2002; Rayner 2003). Still others point out the reality of online situations in terms of being stuck with non-randomly selected participants, a.k.a., samples of convenience, which are not likely to be representative of any particular group or geographic area (Konstan and Chen 2007).

3.5

Our recruitment strategy was basically to use a geographically-stratified sample and create three levels of agent abundance (high, medium, low) using a models two forms of political representation in the United States Congress. To recruit the "low" level of approximately 25 from our population, we used a model similar to political representation the U.S. Senate, by selecting one agent from each major subdivision (e.g., county or watershed) beginning with the most populated ZCTA or CSD. To recruit medium and high levels of approximately 100 and 1000 agents, we used a different model more like the congressional districts in the U.S. House of Representatives, selecting agents proportional to the population of each minor subdivision (e.g., zip code tabulation area or Canadian census subdivision).

3.6

As noted, agents use symbols to communicate their mental models about situations in their environment. For human social actor entities, a "situation" is any set of social-ecological entities or relationships to which the social actor has an individual social and geographic orientation. A social actor's orientation with respect to those referents might be perceived in terms of a direct benefit or resource produced by that situation, or it might be perceived as an indirect, parallel, or induced benefit or service derived from a situation. Likewise, a social actor's orientation may be based on their perception of a direct or indirect benefit from a situation, or alternatively, in terms of that social actor's occupation in terms of a public agency's jurisdiction over a situation. Mental models have been of longstanding interest in sustainability science (e.g., see Mathevet et al. 2011). However, less influential in sustainability science are geohistorical social science perspectives that demonstrate the contemporary social and political manifestations stemming from the long-term influence of social and geographic, i.e., maritime-commercial versus territorial-administrative, orientation to everyday flows of goods and materials, people, finance, and information (Fox 1971, 1980; Braudel 1972). Discussion of the geohistorical social science literature is beyond the intent of this article, but it bears mention in terms of calls for reunifying social and behavioral science with social theory in computational cognitive modeling (Conte 2002). Nonetheless, with such general theoretical insights in mind we used GIS to calculate a rudimentary social-geographic orientation or level of affectedness with respect to the greater Puget Sound and Salish Sea region as a product of distance from the coast multiplied by elevation above sea level (see the attribute "ORIENTATION" in Figure 6).

Figure 7. Schematic representation of the relational database used in the simulation, representing some of the key tables and attributes of the agent object class. See Figure 8 for a visualization of the mental model tables.

Conceptual Properties of Agents

3.7

Agents operated with one of three modes with respect to their preconceptions (blank slate, clone, social actor). At the first level, agents operate in "blank slate" mode (Figure 7). In blank slate mode, agents begin with no preconceptions about anything, being neutral with respect to every situation regardless of the mental model. At the second level, agents operate in "clone" mode. In clone mode, agents begin the experiment with different preconceptions depending on the situation, but they all have exactly the same preconceptions to the same situations. At the third and most diverse level of preconceptions, agents operate in full "social actor" mode. In social actor mode each agent object instance carries a different set of preconceptions for each situation. Agent mental models were integrated with the agent-based model using a relational database.

3.8

In his classic study of organizational decision making, Thompson (1967) suggested there are two kinds of uncertainties when people make decisions about changing an existing situation into a preferred one. One kind of uncertainty surrounds "beliefs" about the cause and effect relations that produce the current situation or might produce a preferred situation in the future. The other kind of uncertainty is about "preferences" about which future outcomes are more desirable (see also Lave and Dowlatabadi 1993). Elaborating on Thompson's (1967) two kinds of uncertainty we developed three different kinds of social actor preconceptions involving beliefs, preferences or assessments. We based our choice of three kinds of preconceptions on broad summaries of the decision making literature that typically identify three kinds of social actors with slightly different preconceptions (e.g., NRC 1996, 2005), in addition to occasional case studies about participatory decision making for sustainability management that confirm three social actor mental models (e.g., Delgado et al. 2009).

3.9

Agents carry three kinds of preconceptions. The first kind of preconception is the affected party (AP) or stakeholder public mental model that looks at a situation from the perspective of the desirability of changing some existing situation into a more preferred one (i.e., intolerable, undesirable, acceptable, desirable, and indispensable). Another kind of preconception is the technical specialist (TS) mental model that looks at a situation in terms of beliefs about the plausibility that some set of cause and effect relations produced the currently existing situation or could produce some future situation (i.e., unimaginable, implausible, conceivable, plausible, and certain). Finally, the third kind of preconception was the executive (EX) mental model that looks at changing the existing situation to a future situation from the perspective of feasibility assessment (i.e., unrealistic, infeasible, possible, feasible, and practical).

3.10

Figure 8 illustrates how each social actor mental model was controlled using a distinct color pattern in a raster data structure. In the case of the affected party (AP), preconceptions differed from lower right to upper left, in this case from a low preconception colored red representing an "intolerable" situation, to the highest preconception colored green representing an "indispensable" situation. We built into our assumptions that executives will generally attempt to balance affected party and technical specialist preconceptions when assessing the feasibility of any particular project, program, or plan aimed at changing an existing situation into a preferred one. Thus the executive mental model was calculated using raster mathematics in GIS based on the technical specialist mental model and the average of all preferences of the affected parties within the jurisdictional boundary area the executive is supposed to represent, e.g., a county. As noted, we also created four different levels of expertise for each agent object operating in social actor mode in order to further differentiate within affected party (AP), technical specialist (TS), and executive decision maker (EX) mental models on the basis of their level of affectedness, expertise, and authority, resulting in a total of 12 different mental models (see Figure 7).

3.11

The three sets of social actor preconceptions do not define three different agents. For example, in reality the same human social actor may tend to reason for the most part using their affected party preferences, but at times may switch mental models and consider the same situation based on their beliefs or assessments. The interesting complexity when it comes to the interactions of these mental models is when situations are deemed indispensable by affected parties, but only conceivable by technical specialists, and infeasible by executives. In other words, the same social actor may prefer a certain future situation but may also at the very same time understand that their own preferences are unlikely given the time and resources needed.

3.12

Thus every instance of the agent object class carries all three preconceptions. However, each instance of the agent object class also carries a unique probability or tendency to favor one set of preconceptions over others at any given time similar to a fuzzy set. For example, an agent might have an affected party probability ("AP_PROB" in Figure 7) of 0.75, a technical specialist probability of 0.2 ("TS_PROB" in Figure 7), and an executive probability of 0.05 ("EX_PROB" in Figure 7). Therefore this particular agent will tend to reason about a situation based on their affected party "preferences" on average three out of every four times they encounter a symbol, and respond accordingly when voting, posting, or replying in the online platform.

3.13

In real human subjects, preconceptions are often measured in ordinal levels of measurement from a questionnaire or similar self-report measure asking participants to rank their agreement or disagreement on a Likert-type item scale. Originally, we assigned agent's prior preconceptions in the relational database as integers with permissible values ranging from 1 to 5 corresponding to five Likert-type categories. We then converted them to real numbers, e.g., a normalized real number scale ranging from highly negative (0.00) to highly positive (1.00) similar to personal probabilities (Kahneman et al. 1982), in order to store them as Java collections in AnyLogic; although it becomes questionable whether preconceptions should be stored using real number values more precise than the nearest tenth of a decimal point.

3.14

Similar to other approaches that have attempted to organize the mental models of sometimes very large populations of agents (Vogt and Divina 2005, 2007; Chaoqing and Peuquet 2009; Vogt 2009), we structured mental models as a raster or grid data structure in a GIS. Using the mental model data structure in Figure 8 to visualize agent preconceptions, the balance of green versus red color patterns reflects the balance of influence between affected party preferences, technical specialist beliefs, and executive assessments. For example, in Figure 8 the color pattern in the affected party mental model carried by each agent ranges from most preferred (green) to least preferred (red) in a generally upper left to lower right color gradient, representing different preferences of more coastal versus more interior orientated agents (see also "ORIENTATION" in Figure 7). The color pattern in the technical specialist mental model carried by each agent ranges from least believable (red) to most believable (green) in four distinct hot spots (Figure 8). Finally, in a somewhat more complicated scheme the color pattern in the executive mental model carried by each agent ranges from least feasible (red) to most feasible (green) by balancing on the hand the affected party preconceptions of agents from the executive's political jurisdiction, and on the other hand, the base technical specialist preconceptions. As noted, the executive mental model of what is most feasible is literally a mathematical compromise between what is more preferred by the affected parties within the executive jurisdiction, versus what is more believable according to the technical experts (Figure 8). Ideally, any visual analyst can look at a color pattern and visual detect, possibly supported by simple spatial statistics, if an experimental outcome was influenced more by affected party preferences, technical specialist beliefs, or a balancing of the two by executive assessments.

Figure 8. Social-actor's mental model as visualized in a GIS as a raster data structure.

Changes in the Conceptual Properties of Agents

3.15

Each instance of the agent object class carries a unique capacity to update its preconceptions by "learning" from other agents and experiencing conceptual change. According to Bayesian theories of learning, the degree to which a person believes a proposition is true depends on the prior preconceptions that a person has in the truth of the proposition and the evidence collected to investigate that proposition (Dempster 1968; King and Golledge 1969; Golledge and Stimson 1997; Davies Withers 2002; Catenacci and Giupponi 2010). The Bayesian theory of learning can be mathematically described as a function of existing preconceptions (Heckerman 1996; Robinson 2003); the inherent credibility of a particular element of information (Flach 1999); and the availability or exposure to a piece of information by each participant (Acemoglu et al. 2010). We assumed that the mental model to which the agent was most highly-oriented would be more resistant to updating, i.e., a mental model built up over long periods of exposure to credible information. In other words, if an agent was likely to reason with an affected party mental model then that agent object would carry a proportionally low probability to update their affected party mental model. To calculate conceptual change and learning we used the Laplacian-corrected Bayesian algorithm based on its successful implementation as a SPAM filtering algorithm (see Robinson 2003). The algorithm we used, coded as an action chart in AnyLogic, updated an agent's preconceptions in the same manner that a basic SPAM filter works based on the credibility of the message and repeated exposure to certain elements of a message (Robinson 2003). After all of an agent's preconceptions are updated to new values as specified by our algorithm, by subtracting the differences between the immediately prior and the newly updated values of a mental model we were able to calculate an agent object's "conceptual change." When we sum all individual agent conceptual changes over the course of the entire decision situation, we called that sum a measure of "social learning."

3.16

What determines if a human social actor will actually learn, thereby updating their preconceptions and undergoing a conceptual change, remains a matter of theoretical debate within the cognitive sciences (Chater et al. 2006a, 2006b, 2006c) and agent-based simulations as well (Lempert 2002; Ramanath 2004; Sun 2006; Kim et al. 2010; Barreteau and Le 2011; Kim 2011; Squazzoni 2012). It is already understood that Bayesian theories of learning are very sensitive to the simplifying assumptions researchers make about preconceptions (Davies Withers 2002). Not satisfied that we could provide the answer to these theoretical and methodological questions, we decided that we would conduct a parameter variation experiment that varied the level of change each agent object instance could undergo. A global conceptual change value of 0.0 meant that all agents possessed a rigid mental model that never changed, whereas a value of 1.0 meant that any given agent was allowed to experience conceptual change according to a unique agent-based probability for experiencing conceptual change (e.g., "AP_LEARN" in Figure 7).

Symbolic Properties of Agents

3.17

Socially intelligent agents communicate their preconceptions to one another using a system of symbols (Conte 2002). A number of theoretical and philosophical perspectives about how actors interact and influence one another through communication and language like semiotics, symbolic interactionism, or the philosophy of mind point to the importance of reasoning about symbols that stand for a concept in one's mind as applied to a set of referents in the world (Peirce ND ; Sperber 1985, 1990; Auspitz 1994; Hilpinen 1995; Sowa 2000; Mancini and Shum 2006; Sowa 2006; Hilpinen 2007). Interestingly, at least one assessment suggests that simulation tools are lacking when it comes to viewing or visualizing information exchanges between agents in an agent-based model (Ralambondrainy et al. 2007).

3.18

In our simulation, each agent used the online platform to browse and filter symbols and then "reason" about the situation by matching it to their preconceptions. Alphabetical tokens like "A" and "B" stand for "concepts." Numeric tokens like "1" and "2" stand for entities and relationships of a social-ecological system (i.e., the "referents"). We consider tokens "A" or "B" combined with "1" or "2" as the basic bundle of categories that agents use, like in a language game (Shoham and Brown 2009; Gilbert 2008). Adding insights from geodesign, sustainability science, and resilience thinking (Gallopín 2006; Moser 2008; Gunderson 2009; Cumming 2011), the concept "A" could be an assessment of the state or identity of a social-ecological system (e.g., the concept of "moderately-susceptible to organic waste contamination during peak episodes of storm runoff"). This concept "A" could be applied to any particular set of spatial elements or relationships of interest "1" (e.g., "relationships between organic waste from small dairy farms and aquatic invertebrates in the upper reaches of the Duwamish River watershed in King County, Washington"). A third token was added as a cue about whether the agents were expressing their belief (b), preference (p), or assessment (a) of a concept-referent bundle or message, e.g., "b|A|1" or "p|A|1." We considered but did not implement a fourth set of tokens to indicate their ordinal rank strength of belief, preference, or assessment. In sum, with three basic frames of mind (a, b, or p) x 26 concepts (A to Z) × 26 referents (1 to 26), agents had the capability to reason about 676 different situations using 8112 symbols.

3.19

The simulation was set to unfold in real Pacific Standard Time over exactly the same period as one of our online field experiments in 2007 (Aguirre and Nyerges 2011). Figure 10 is an illustrative example of how an agent, when routed through a deliberative action chart after transitioning to the state of being active in the online platform, used the simulated browsing and filtering tools in the platform to sort symbols as messages about situations by most voted, and then reason about the resulting list and vote to agree with one of the situations being posed. Each agent was randomly assigned a certain number of times per day they would be expected to perform a deliberative action. Agents were expected to be active in the online platform for only a certain time during the day and week, based on the frequency of activity observed in human subjects from previous online field experiments.

3.20

Agents had available to them three different methods of browsing and filtering messages including filtering by the top 10 most recently posted, by the top 10 most voted in terms of number of negative or positive votes (see Figure 10), and finally, by the top 10 most replied. Rules for how agents browse and filter messages are a particularly interesting set of controls to consider since actual human participants in online public participation decision making may generally prefer certain methods over others, which may bias certain kinds of messages. Nonetheless, after filtering a sample of 10 messages using one of three methods following the same preferences observed in human subjects, agents "reasoned" about their subset of messages in terms of how they matched their preconceptions. Agents re-sorted their sample of 10 messages from highest to lowest match with their existing preconceptions and then selected the top result of this re-sorted list to vote on or reply to (Figure 10). If they intended to find the situation that most matched their preconceptions, then they voted to "agree" with the top result. If the agents were replying to a message rather than simply voting on it, they could engage in a somewhat more complex situation where they would be able to change one token in the message, either the concept or the referent token, so that the resulting bundle of tokens in the symbol ranked higher according to their mental model at the time.

Results

4.1

The three factors and three levels included 1) the social and geographic distribution of agents (local, regional, international), 2) the abundance of agents (low, medium, high), and 3) the diversity of preconceptions (blank slate, clone, social actor). Cross-tabulating all three factors and levels meant running 27 simulated field experiments, not including sensitivity analyses or replication experiments to evaluate random effects. However, we were unable to run any treatments at the "high" level of abundance of agents involving roughly 1000 agents because the complexity of the simulation outstripped the power of our desktop computing capabilities. Thus we were only able to examine the first two levels of abundance of agents (low and medium), resulting in a total of 18 treatments instead of the originally planned 27 treatments. In future research, either a simpler model design or higher performing computing systems would be needed.

Figure 9. Event log table from simulated online field experiment.

Figure 10.

4.2

For the 18 simulated field experiments we were able to successfully run, we generated a set of observations resembling a client-server event log (Figure 9). The simulated event log in Figure 9 was designed to be very similar to what was collected from the online platform used in actual field experiments (Nyerges and Aguirre 2011; Aguirre and Nyerges 2011). Several thousand events were logged for each treatment, after which they were exported to a relational database for analysis. Parsing out some of the attribute information in a sample row from the event log table in Figure 9, one can see an example of deliberative activity by an agent object instance with ID "78" operating in social actor mode ("Preconceptions, 0.0, 0.0") during the low abundance international scale treatment ("1423 LOW 101"), referring to the 101 participants recruited from 1423 sub-divisions throughout Washington State and British Columbia. The terms "Updating 0.0, 0.0, 0.2" indicates that the level of conceptual change in the parameter variation sensitivity analysis was at step "0.2" on a possible range of 0.1 to 1.0. The simulated event log recorded a particular interaction event by agent object instance "78," an agent that tends to operate with an executive social actor mental model (0.61), during Step 6 of the simulated experiment at time Friday, November 9, 2007 at 08:03:04 AM PST. At that time agent "78" replied to a situation represented by symbol "a|T|7" with a modified message "a|H|7," which according to their executive mental model represented a slightly more feasible (0.69 versus 0.66) state for the social-ecological system referred to in "7."

Scaling did not affect conceptual change on a per agent basis

4.3

As expected, as an agent's ability to experience conceptual change increased the overall social learning steeply increased. In addition, the greater the diversity of preconceptions the greater the average level of conceptual change on a per agent basis. For example, the results of average level of conceptual change for a "medium" abundance experiment (c. 100 participants) across different local, regional, and international scales indicate much more conceptual change occurs when agents are acting in social actor mode as opposed to blank slate or clone mode. However, not as expected, changing the social and geographic distribution and abundance of agents did not seem to have any significant impact on social learning outcomes measured on an average agent basis. In fact, we found nearly the same levels of conceptual change on a per agent basis for the "low" abundance experiment (between 12 and 37 participants) across all three local, regional, and international social and geographic distributions. This finding might suggest that while a diversity of preconceptions increases social learning, varying social and geographic distribution as well as abundance are not important influences. Why is it that 100 agents from a local geographic area would experience the same level of conceptual change, on average, as 100 agents from a regional or international geographic area if preconceptions are supposed to vary geographically? We felt that this result was a product of our own simplifying assumptions in the model itself, but not a reasonable one. Further model design should focus on the sensitivity of the model to changes in the social and geographic distribution of agents (local, regional, international) and the abundance of agents (low, medium, high).

Scaling may affect the choices agents make

4.4

It was expected that changing the geographic distribution and abundance of agents would have an impact on the most popular situations, in particular showing the influence of affected party preferences visually in terms of a color pattern shifted from upper left to lower right after scaling out from a local (central Puget Sound region or "A") to regional (Salish Sea drainage basins or "B") to an international (Washington and British Columbia or "C") region. To adequately test this hypothesis, ideally we would have preferred to simply iterate each experiment hundreds or thousands of times, possibly using spatial statistics to determine how each raster data structure was different. The AnyLogic simulation platform provided us with a way of managing replication experiments using its OptQuest algorithm.

4.5

As expected, the most important result of the simulation is the finding that when the social and geographic distribution and abundance of agents change, the most popular and least popular choices out of the 676 situations also change (Figure 11). We measured the most and least popular choices by calculating a popularity ratio based on subtracting agree votes from disagree votes and then dividing by total number of votes cast. The highest popularity ratio possible is 1.0, whereas the lowest popularity ratio possible is –1.0. Figure 11 illustrates an example of the most popular choices selected by all agents at the local, regional, and international scale within the "medium" abundance experiment of about 100 agents, visualized in ArcGIS using a raster data structure. We discuss the theoretical implications of these findings in more detail below.

Figure 11. The most popular and least popular situations as voted on by agents in the "medium" abundance experiment (c. 100 participants) across local (A), regional (B), and international (C) scales.

Conclusion

5.1

The goal of the simulation was to model the impact of scaling on how social actors might self-organize through online communication and consensus. Our factorial research design involved socially intelligent agents interacting under different conditions based on three sets of factors involving 27 different treatments. The nine factors included the social and geographic distribution of agents (local, regional, international), abundance of agents (low, medium, high), and diversity of preconceptions (blank slate, clone, social actor). Due to computational limitations, we were not able to run the 9 treatments involving a "high" abundance of agents.

5.2

We expected that social and geographic distribution of agents as well as diversity of agent preconceptions would strongly impact consensus about which situations to change and which ones not to change. However, our expectations were not met by our findings. Firstly, we examined how changes in social and geographic distribution and abundance of agents, as well as mental model diversity, affected conceptual change and social learning on a per agent basis. As expected, increasing an agent's ability to experience conceptual change and increasing the diversity of preconceptions increased the average level of conceptual change on a per agent basis. Somewhat unexpectedly, geographic distribution and abundance had little impact on conceptual change. Secondly, we examined whether changes in social and geographic distribution and abundance of agents might affect the choices agents make. As expected, when we changed the social and geographic distribution and abundance of social actor agents, the most popular choice of situations also changed, as measured using a popularity ratio from 1.0 and -1.0.

5.3

In future simulations, we might more carefully structure affected party, technical specialist, and executive social actor mental models in visual patterns to generate predictable tensions between what is most preferred, most plausible, and most feasible such that we could compute an optimum set of choices and then compare actual simulation results of the most popular choices. For example, we might see the most popular situations in the online platform change as a function of the activity of certain kinds of social actor agents. As another example, by increasing the abundance or the relative importance of certain social actor roles, simulating the influence of compulsion and power, we could calculate spatial statistics based on visual representations like Figure 11 to see how the most popular choices are made to conform to a certain mental model. Another step would be to control the number and complexity of representational signs of meaning from a cognitively fundamental "handful" (5×5 or 25 situations), to a "dozen" (12×12 or 144 situations), and then finally the "alphabet soup" set of conditions (26×26 or 676 situations) we used in our current research design. In terms of a future research design, it would be useful to establish controls over certain agent object parameters or variables now that we have more insight about what to control, e.g., the balance of social actor roles, the variety of situations being considered, or even the online platform tools available for browsing and filtering. In future simulations we might also consider entirely new mental model representations like concept maps rather than the 26×26 raster cell matrices implemented as sortable Java collections. Lastly, unexpected computing issues prevented our being able to run a complete set of 27 controlled conditions. Obviously, a useful next step is to make use of a more powerful computational platform.

5.4

We have yet to take the lessons learned from simulation and turn back to experiments with human subject participants, as in earlier research on face-to-face human computer interaction (Jankowski and Nyerges 2001) and online field experiments (Nyerges and Aguirre 2011; Aguirre and Nyerges 2011). Brinberg and McGrath ( 1985), who we draw upon for our own research in this article, offered warnings about the impact of methodological, theoretical, or substantive preferences in the social sciences. Reflecting on the impact of methodological disputes about the merits of experimentation versus field observation in the history of biological thought, Ernst Mayr (1982) believed that any narrative statement about a relationship between elements could legitimately be tested by experimentation. However, if the narrative statement in question described an actual sequence of occurrences then it could only be reconstructed through substantive observations of the past, in which case harboring a preference for theoretical experimentation at the expense of field observations was misplaced. Mayr felt that a biological researcher's own premature insistence on either experimentation or field observation was what had caused biological research itself to move into unsuitable directions as if stuck between two "false alternatives," something he felt was the cause of nearly every controversy in the history of evolutionary biology (Mayr 1982).

5.5

Research on participatory decision making is susceptible to controversies at an even more impulsive level, since researcher's confidence in "false alternatives" is likely based upon simplifying statements that have never been fully explored either through laboratory experimentation or evaluation in the field (Laurian and Shaw 2009). Investigating a single element of success or failure when it comes to participatory decision making for sustainability management might naturally lead a researcher to make premature conclusions about the "best" way to manage any number of important elements including the best way of recruiting participants, making factual information available, scaffolding reasoning and learning, or creating a forum for deliberation. The ways in which all these elements are related and the sometimes unintended, unanticipated, or unknown spatial and temporal relationships that emerge between them have yet to be understood.

5.6

Though a simulation-based research design is not a substitute for research with human subjects, it is well suited to triangulating findings drawn from field experiments and case studies. However, our results suggested to us more about the theoretical concepts we used to inform our agent-based model design than our substantive area of interest, the greater Puget Sound region. The theory of self-organizing control systems in sustainability science assumes that the more resource users are able to communicate their mental models of the system of which they are a dependent part, combined with the importance of that resource to the users themselves, the more likely they will invest the necessary time and energy to manage the system to maintain its identity and its resilience to disturbance or overuse. Sustainability science provides a conceptual framework of variables predicting self-organizing behavior, but this framework was created for the most part through case studies, not experimentation with human subjects or agent-based models. As a result, when one asks fundamental questions of the theory of self-organizing behavior for the purpose of an agent-based model the answers are not clear.

5.7

We feel that our conceptual modeling efforts were challenged by the current state of sustainability science theory. In terms of geographic space, how is the strength of a social actor's preconceptions about a specific situation in their environment, e.g., the direct harvesting of timber resources, interdependent with their social and geographic orientation to any of the myriad flows of goods, people, finance and information associated with those timber resources? In terms of historical time, can self-organizing behavior among resource users be sparked by no more than a month-long decision making situation hosted in an online platform? How can self-organizing behavior be sustained given short-term political or disturbance events, medium-term economic cycles, or long-term cultural and environmental change? As our findings clearly suggest, experimentation or simulation are especially useful in at least one particular regard, i.e., it forces one to specify the social, geographic, and historical factors predicting when a group of social actors in a certain context will self-organize to avoid deteriorating their own environment, and when the conditions tend to make government compulsion and authority necessary.

Acknowledgements

A portion of this material is based upon work supported by the National Science Foundation under Grant Number OCI-1047916, BCS-0921688, and EIA 0325916, and National Oceanic and Atmospheric Administration Sectoral Applications Research Program Grant NA07OAR4310410. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. Support from the National Science Foundation and National Oceanic and Atmospheric Administration is gratefully acknowledged. We would also like to acknowledge the Department of Geography, the Professional Master's Program in Geographic Information Systems for Sustainability Management, and the Participatory Geographic Information Systems Technologies Group at the University of Washington. The authors are solely responsible for the content. For full documentation of the model, including complete descriptions of all model elements in PDF, DOCX, or HTML format, or for the working version of the AnyLogic model and accompanying relational database please feel free to contact the authors.

Notes

¹ The term "public participation" includes organized processes by elected officials, government agencies, or other public or private-sector organizations to engage affected parties and technical specialists in environmental assessment, planning, decision making, management, monitoring, or evaluation. These processes supplement traditional forms of public participation (voting, forming interest groups, demonstrating, lobbying) by directly involving the public in functions which, when conducted in government, are traditionally delegated to public sector executives.

References

ACEMOGLU, D., Dahleh, M. A., Lobel, I., and Ozdaglar, A. (2010). Bayesian Learning in Social Networks, MIT LIDS Working Paper #2780. http://pages.stern.nyu.edu/~ilobel/socialnetworks_revised.pdf.

AGRAWAL , A. (2001). Common Property Institutions and Sustainable Governance of Resources. World Development, 29(10), 1649-1672. [doi:10.1016/S0305-750X(01)00063-8]

AGUIRRE, R. & Nyerges, T. (2011). Geovisual evaluation of public participation in decision making: The grapevine. Special Issue on Challenging Problems in Geovisual Analytics, Journal of Visual Languages and Computing, 22(4), 305–321. [doi:10.1016/j.jvlc.2010.12.004]

ANYLOGIC (2013). http://www.anylogic.com/

AUSPITZ, J. L. (1994). The wasp leaves the bottle. The American Scholar, 63, 602–6.

BARRETEAU, O. & Le, P. C. (2011). Using social simulation to explore the dynamics at stake in participatory research. Journal of Artificial Societies and Social Simulation, 14, 4.

BEISNER, B. E., Haydon, D. T. and Cuddington, K. (2003). "Alternative Stable States in Ecology". Frontiers in Ecology and the Environment, 1, 7. [doi:10.1890/1540-9295(2003)001[0376:ASSIE]2.0.CO;2]

BENNETT, R. J. & Chorley, R. J. (1978). Environmental systems: Philosophy, analysis and control. London: Metheun & Co Ltd.

BRAUDEL, F. (1972). The Mediterranean and the Mediterranean world in the age of Philip II. New York: Harper & Row.

BRINBERG, D. & McGrath, J. E. (1985). Validity and the research process. Beverly Hills: Sage Publications.

BROWN, D. G., Riolo, R., Robinson, D. T., North, M., and Rand, W. (2005). Spatial process and data models: Toward integration of agent-based models and GIS. Journal of Geographical Systems, 7, 25–47. [doi:10.1007/s10109-005-0148-5]

CAMPAGNA, M. (2006). GIS for sustainable development. Boca Raton: CRC Press.

CARSON, L. and B. Martin, B. (2002). Random Selection of Citizens for Technological Decision Making. Science and Public Policy 29(2), 105-113. [doi:10.3152/147154302781781047]

CATENACCI, M. & Giupponi, C. (2010). Potentials and Limits of Bayesian Networks to Deal with Uncertainty in the Assessment of Climate Change Adaptation Policies. Milano: Fondazione Eni Enrico Mattei.

CHAOQING, Y. & Peuquet, D.J. (2009). A GeoAgent-based framework for knowledge-oriented representation: Embracing social rules in GIS. International Journal of Geographical Information Science, 23(7), 923–960. [doi:10.1080/13658810701602104]

CHATER, N. & Manning, C. D. (2006a). Probabilistic models of language processing and acquisition. Trends in Cognitive Sciences, 10(7), 335–344. [doi:10.1016/j.tics.2006.05.006]

CHATER, N., Tenenbaum, J. B., & Yuille, A. (2006b). Probabilistic models of cognition: Conceptual foundations. Trends in Cognitive Sciences, 10(7), 287–291. [doi:10.1016/j.tics.2006.05.007]

CHATER, N., Tenenbaum, J. B., & Yuille, A. (2006c). Probabilistic models of cognition: where next. Trends in Cognitive Sciences, 10(7), 292–293. [doi:10.1016/j.tics.2006.05.008]

CLARK, W. C. (2007). Sustainability science: A room of its own. Proceedings of the National Academy of Sciences of the United States of America, 104(6), 1737–8. [doi:10.1073/pnas.0611291104]

CONTE, R. (2002). Agent-based modeling for understanding social intelligence. Proceedings of the National Academy of Sciences of the United States of America, 99, 7189-90. [doi:10.1073/pnas.072078999]

CUMMING, G. S. (2011). Spatial Resilience in Social-Ecological Systems. Dordecht: Springer. [doi:10.1007/978-94-007-0307-0]

DAVIES WITHERS, S. (2002). Quantitative methods: Bayesian inference, Bayesian thinking. Progress in Human Geography, 26(4), 553–566. [doi:10.1191/0309132502ph386pr]

DELGADO, L. E., Marín, V. H., Bachmann, P. L. and Torres-Gomez, M. (2009). Conceptual models for ecosystem management through the participation of local social actors: the Río Cruces wetland conflict. Ecology and Society, 14(1), 50. http://www.ecologyandsociety.org/vol14/iss1/art50/

DEMPSTER, A. P. (1968). A Generalization of Bayesian Inference. Journal of the Royal Statistical Society. Series B (Methodological), 30(2), 205–247.

EARLE, C. (1988). The Myth of the Southern Soil Miner: Macrohistory, Agricultural Innovation, and Environmental Change. In Worster, D. The Ends of the earth: Perspectives on modern environmental history. Cambridge, England: Cambridge University Press.

FERGUSON, M. L. (2007). Initiatives, Referenda, and the Problem of Democratic Inclusion: A Reply to John Gastil and Kevin O'Leary. University of Colorado Law Review, 78(4), 1537–49.

FLACH, P., and Lachiche, N. (1999). 1BC: A First-Order Bayesian Classifier. Lecture Notes in Computer Science, 1634, 92. [doi:10.1007/3-540-48751-4_10]

FOX, E. 1971. History in geographic perspective: The other France. New York: W. W. Norton.

FOX, E. 1980. The range of communications and the shape of social organization. Communication, 5, 275–287.

GALLOPÍN, G. C. (2006). Resilience, Vulnerability, and Adaptation: A Cross-Cutting Theme of the International Human Dimensions Programme on Global Environmental Change. Global Environmental Change, 16(3), 293–303. [doi:10.1016/j.gloenvcha.2006.02.004]

GASTIL, J., Reedy, J. and Wells, C. (2007). When Good Voters Make Bad Policies: Assessing and Improving the Deliberative Quality of Initiative Elections. University of Colorado Law Review, 78(4), 1435-1488.

GILBERT, G. N. (2008). Agent-based models. Los Angeles: Sage Publications.

GOLLEDGE, R. G. & Stimson, R. J. (1997). Spatial behavior: a geographic perspective. New York: Guilford Press.

GRIMM V, Berger U, DeAngelis DL, Polhill G, Giske J, Railsback SF. (2010). The ODD protocol: a review and first update. Ecological Modelling 221: 2760–2768. [doi:10.1016/j.ecolmodel.2010.08.019]

GUNDERSON, L. (2009). Comparing ecological and human community resilience. CARRI Research Report 5. Oak Ridge, TN: Community and Regional Resilience Initiative.

HECKERMAN, D. (1996). A tutorial on learning with Bayesian networks. Technical Report MSR-TR-95-06. Redmond: Microsoft Research, Advanced Technology Division, Microsoft Corporation.

HILPINEN, R. (1995). Peirce on language and reference. In Ketner, K. L., Peirce and contemporary thought: philosophical inquiries. New York: Fordham University Press.

HILPINEN, R. (2007). On the Objects and Interpretants of Signs: Comments on T. L. Short's Peirce's Theory of Signs." Transactions of the Charles S. Peirce Society: A Quarterly Journal in American Philosophy, Volume 43, Number 4, Fall 2007, pp. 610–618.

HILTY, L. M., Seifert, E. K., & Treibert, R. (2005). Information systems for sustainable development. Hershey, PA: Idea Group Pub. [doi:10.4018/978-1-59140-342-5]

JANKOWSKI, P., & Nyerges, T. L. (2001). Geographic information systems for group decision making: Towards a participatory, geographic information science. London: Taylor & Francis.

JEFFERSON CENTER. 2009. Citizens Jury Handbook. http://www.jefferson-center.org (last accessed 15 July 2009).

JOHNSTON, Kevin M. 2013. Agent Analyst: Agent-Based Modeling in ArcGIS. Redlands: Esri Press.

KAHNEMAN, D., Slovic, P., and Tversky, A. (Eds). (1982). Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press. [doi:10.1017/cbo9780511809477]

KATES, R. K., Clark, W. C., Corell, R., Hall, J. M., Jaeger, C. C., Lowe, I., McCarthy, J. J., Schellnhuber, H. J., Bolin, B., Dickson, N. M., Faucheux, S., Gallopin, G. C., Grubler, A., Huntley, B., Jager, J., Jodha, N. S., Kasperson, R. E., Mabogunje, A., Matson, P., Mooney, H., Moore III, B., O'Riordan, T., Svedin, U. (2001). Sustainability Science, Science, 292, 641–642. [doi:10.1126/science.1059386]

KERSTEN, G. E., Yeh, A. G. O., Mikolajuk, Z., & International Development Research Centre (Canada). (2000). Decision support for sustainable development: A resource book of methods and applications. Boston: Kluwer.

KIM S.-Y., Taber C.S., and Lodge M. (2010). A computational model of the citizen as motivated reasoner: Modeling the dynamics of the 2000 presidential election. Political Behavior, 32(1), 1–28. [doi:10.1007/s11109-009-9099-8]

KIM S. (2011). A model of political judgment: An agent-based simulation of candidate evaluation. Journal of Artificial Societies and Social Simulation, 14(2).

KING, L. J. and Golledge, R.G. (1969). Bayesian analysis and models in geographic research. In McCarty, H. H., Geographical essays commemorating the retirement of Professor Harold H. McCarty. Iowa City: Dept. of Geography, University of Iowa.

KLINSKY, S., Sieber, R., and Meredith, T. (2010). Connecting Local to Global: Geographic Information Systems and Ecological Footprints as Tools for Sustainability. The Professional Geographer, 62(1), 84–102. [doi:10.1080/00330120903404892]

KONSTAN, J.A. and Chen, Y. (2007). Online Field Experiments: Lessons from CommunityLab. Proceedings of the Third Annual Conference on e-Social Science Conference, Ann Arbor, MI.

LAURIAN, L. & Shaw, M. (2009). Evaluation of Public Participation. Journal of Planning Education and Research, 28(3), 293–309. [doi:10.1177/0739456X08326532]

LAVE, L. B. & Dowlatabadi, H. (1993). Climate change: the effects of personal beliefs and scientific uncertainty. Environmental Science and Technology, 27(10), 1962–72. [doi:10.1021/es00047a001]

LEMPERT, R. (2002). Agent-based modeling as organizational and public policy simulators. Proceedings of the National Academy of Sciences of the United States of America, 99(10), 7195–6. [doi:10.1073/pnas.072079399]

LIU, J., Dietz, T., Carpenter, S. R., Alberti, M., Folke, C., Moran, E., Pell, A. N., ... Taylor, W. W. (2007). Complexity of coupled human and natural systems, Science, 317(5844), 1513–6. [doi:10.1126/science.1144004]

MANCINI, C. & Shum, S. J. B. (2006). Modelling discourse in contested domains: A semiotic and cognitive framework. International Journal of Human-Computer Studies, 64(11), 1154–1171. [doi:10.1016/j.ijhcs.2006.07.002]

MATHEVET Raphael, Etienne, M., Lynam, T. and Calvet, C. (2011). Water Management in the Camargue Biosphere Reserve: Insights from Comparative Mental Models Analysis, Ecology & Society, 16, 1.

MAYR, E. (1982). The growth of biological thought: Diversity, evolution, and inheritance. Cambridge, Mass: Belknap Press.

MOSER, S. (2008). Resilience in the face of global environmental change. CARRI Research Report 2. Oak Ridge, Tenn: Community and Regional Resilience Initiative.

NATIONAL RESEARCH COUNCIL. (1996). Understanding Risk: Informing Decisions in a Democratic Society, National Academy Press, Washington, DC.

NATIONAL RESEARCH COUNCIL. (2005). Decision Making for the Environment: Social and Behavioral Science Research Priorities, National Academy Press, Washington, DC.

NATIONAL RESEARCH COUNCIL. (2012). Computing and sustainability, National Academy Press, Washington, DC.

NYERGES, T. & Aguirre, R. (2011). Public Participation in Analytic-Deliberative Decision Making: Evaluating a Large-Group Online Field Experiment. Annals of the Association of American Geographers, 101(3), 561–586. [doi:10.1080/00045608.2011.563669]

NYERGES, T. L., & Jankowski, P. (2010). Regional and urban GIS: A decision support approach. New York: Guilford Press.

OSTROM, E. (2007). A diagnostic approach for going beyond panaceas. Proceedings of the National Academy of Sciences, 104(39), 15181–15187. [doi:10.1073/pnas.0702288104]

OSTROM, E. (2009). A General Framework for Analyzing Sustainability of Social-Ecological Systems. Science, 325, 5939, 419–422. [doi:10.1126/science.1172133]

PEIRCE, C. S. (NO DATE). What is a Sign? MS 404 http://www.iupui.edu/~peirce/ep/ep2/ep2book/ch02/ep2ch2.htm

POLHILL, J. G, Parker, D., Brown, D. and Grimm, V. (2008) Using the ODD Protocol for Describing Three Agent-Based Social Simulation Models of Land-Use Change. Journal of Artificial Societies and Social Simulation, 11, 2.

RALAMBONDRAINY, T., Médoc, J.-M., Courdier, R. & Guerrin, F. (2007). Tools to Visualize the Structure of Multi-agent Conversations at Various Levels of Analysis. In Oxley, L. and Kulasiri, D. (Eds.) MODSIM 2007 http://www.mssanz.org.au/MODSIM07/papers/56_s43/ToolsToVisualizes43_Ralambondrainy_.pdf

RAMANATH, A. N. and Gilbert, N. (2004). The Design of Participatory Agent-Based Social Simulations. Journal of Artificial Societies and Social Simulation, 7(4).

RAYNER, S. (2003). Democracy in the Age of Assessment: Reflections on the Roles of Expertise and Democracy in Public-Sector Decision Making. Science and Public Policy 30(3),163-170. [doi:10.3152/147154303781780533]

ROBINSON, G. 2003. A STATISTICAL APPROACH TO THE SPAM PROBLEM - Can mathematics tell spam apart from legitimate mail? Find out which approaches work best in real-world tests. Linux Journal. (107), 58.

SIMON, H. A. (1976). Administrative behavior: A study of decision-making processes in administrative organization. New York: Free Press.

SIMON, H. A. (1981). The sciences of the artificial. Cambridge, Mass: MIT Press.

SHOHAM, Y., and Leyton-Brown, K. (2009). Multiagent systems: algorithmic, game-theoretic, and logical foundations. Cambridge: Cambridge University Press.

SOWA, J. F. (2000). Ontology, Metadata, and Semiotics. Lecture Notes in Computer Science, 1867, 55–81. [doi:10.1007/10722280_5]

SOWA, J. (2006). Worlds, Models and Descriptions. Studia Logica, 84(2), 323–360. [doi:10.1007/s11225-006-9012-y]

SPERBER, D. (1985). Anthropology and Psychology: Towards an Epidemiology of Representations. Man. 20(1), 73–89. [doi:10.2307/2802222]

SPERBER, D. (1990). The epidemiology of beliefs. In Fraser, C. & Gaskell, G. The social psychological study of widespread beliefs. Oxford: Clarendon Press.

SQUAZZONI, F. (2012). Agent-based computational sociology. Hoboken, N.J.: Wiley & Sons. [doi:10.1002/9781119954200]

STEINITZ, C. (2011). On Scale and Complexity and the Need for Spatial Analysis. Position paper delivered to the Specialist Meeting on Spatial Concepts in GIS and Design, Santa Barbara, CA, December 15–16, 2008. http://ncgia.ucsb.edu/projects/scdg/docs/position/Steinitz-position-paper.pdf

STEINITZ, C. (2012). A framework for geodesign. Redlands: Esri Press.

SUN, R. (2006). Cognition and multi-agent interaction: From cognitive modeling to social simulation. Cambridge: Cambridge University Press.

THOMPSON, J. D. (1967). Organizations in action; social science bases of administrative theory. New York: McGraw-Hill.

VOGT, P. (2009). Modeling Interactions Between Language Evolution and Demography. Human Biology, 81(2/3), 237–58. [doi:10.3378/027.081.0307]

VOGT, P. & Divina, F. (2005). Language evolution in large populations of autonomous agents: issues in scaling. http://arno.uvt.nl.offcampus.lib.washington.edu/show.cgi?fid=52775

VOGT, P. & Divina, F. (2007). Social symbol grounding and language evolution. Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems, 8(1), 31–52. [doi:10.1075/is.8.1.04vog]

WECD - World Commission on Environment and Development (1987). Our Common Future. http://www.un-documents.net/wced-ocf.htm