Abstract

This is an introduction to the special section of JASSS on the above topic. It argues for the importance of qualitative evidence in social science, and particularly in the specification of agent-based models. It ends by suggesting some criteria for judging methods for using qualitative evidence for this purpose.

Keywords:

Qualitative, Evidence, Narrative, Specification, Quantitative, Formal

Introduction

1.1

Maybe what distinguishes science from other domains is that when theory and evidence clash, that (eventually) it is the theory that is discarded or changed^[1]. In other words, that evidence should not be ignored without a very, very good reason — including both quantitative and qualitative evidence. Qualitative evidence has often been seen as "unscientific", critiqued as: subjective, biased, unreliable and context-specific. These critiques are not without merit — qualitative evidence does have its difficulties — but it also has its own advantages (Yang & Gilbert 2008) and the difficulties are not sufficient to justify ignoring it. All kinds of evidence have their own difficulties — abstracting and simplifying from a complex world in different ways^[2]. Quantitative representation has its own difficulties and may not always be the better method (Edmonds 2004, 2010). All evidence is "bootstrapped" upon previous tools, assumptions, theories and evidence meaning that one has to take care in its use — not pushing it beyond its limits and being as transparent as possible about its derivation. The response to such difficulties and complications should not be to ignore any kind of evidence, but rather find methods and tools to deal with it, to capture it, to analyse it, to check it and assess it^[3]. This special section of JASSS is specifically focussing on ways of productively using qualitative evidence.

1.2

I think there is another reason why qualitative data has been overlooked in the past. Previously the only kinds of formal model available were mathematical in nature^[4]. As I have argued previously (Edmonds 2010) formal models^[5] are essential to science, not because they are more "true" than other kinds of model, but that they facilitate a collective and progressive development of knowledge between researchers. Before the advent of accessible computational power, the available models, (being mathematical) made it hard to use qualitative data, and easy to use quantitative data. Since the development of complex science required formal models, science became associated with the quantitative^[6]. However, with the advent of computational models (which by their nature are formal), especially agent-based modelling (ABM) techniques, this difficulty is no longer a deal-breaker. We no longer have to ignore evidence because it does not fit our models, but rather can choose use the most appropriate kind of model for what is being modelled, and the most appropriate formal model may well not be mathematical. According to this view, the association of the quantitative with "being scientific" is simply mistaken — indeed by the above principle, discarding evidence because it does not fit easily into the (mathematical) form of one's theories is the more unscientific. Furthermore, as others have pointed out (e.g. Yang & Gilbert 2008), ABM is particularly suited for encoding some aspects the qualitative data — particularly those that have a narrative component to them — and that (to a substantial degree) unnecessary numerical representation can be avoided^[7]. This special section looks at some different ways that qualitative data can be used to inform the specification of ABM.

1.3

Of course, although ignored by fields aspiring to be "scientific" for many decades, qualitative evidence has not gone away. There is a vast and flourishing field of qualitative social science, which is now composed of a forest of schools and approaches^[8]. However, for some time they have been largely divorced from those fields that use formal models — even to the extent that people talked about the "Qual-Quant Wars". During this period there were essentially two ways for simulation modellers to cross the "trenches" that divided the two worlds: one could be informally motivated by work based on qualitative social science but justify actual modelling decisions on other grounds^[9], or one could construct and use a computational models but only as an analogy to inform thought (that is without a specific relationship to any evidence). The contributions to this special section move beyond this to a more specific and identifiable link between the qualitative and the formal. This special issue amply demonstrates that there is a well-founded way to combine the qualitative and quantitative — via agent-based simulation.

1.4

Agent-based modellers have always used a variety of sources to inform the design of their simulations. Often this process has been left as informal: unsystematic and undocumented. The contributions to this special section, in their different ways, seek to move beyond this and develop methods that are more systematic (so that others could follow and repeat what they did) and transparent (so that it is easier to see what additional structures and assumptions are used and how). There are two approaches these papers take to bridging the gap between qualitative data and simulation code: (1) by constraining the process of elicitation/analysis or (2) via the use of prior structures/frameworks. All use some of both, but tend to focus on one side or the other.

1.5

Bharwani et al, Dilaver, Neumann, and Le Page et al. concentrate on process whilst Rosales-Carreón & García-Díaz, Edmonds, Ghorbani et al. and Thorngate assume or propose structures which the evidence or responses are fitted into. Thus, this special section covers a number of techniques: Grounded Theory (Dilaver, Neumann), Companion Modelling (Le Page et al), the 'KnETs' decision elicitation process (Bharwani et al),policy capturing, information seeking, and social choice (Thorngate). It also covers the use of several prior structures or formalisations: domain knowledge and systemigrams (Rosales-Carreón & García-Díaz), context/scope/narrative-element distinction (Edmonds), formal ontologies (Neumann), the "MAIA" framework (Ghorbani et al.), preferences, sequences, and features (Thorngate).

1.6

Ideally any such approach should:

• Preserve as much of the meaning from the original data/situation as possible^[10];
• Introduce as few distortions as possible^[11];
• Be as transparent as possible, that is that when assumptions are used, they are clear from the report of the procedure and not hidden;
• Be practical as a process and not demanding of impossible or infeasible steps;
• Be as systematic as possible, so that others can attempt to retrace the steps taken;
• Be as honest as possible, in that it does not fudge results claiming to do more than it can deliver.

1.7

Methods that go some way to meeting the above criteria could have a transformative effect on social science by helping bring together the worlds of qualitative and quantitative evidence in a principled manner using agent-based simulation. Further enriching our simulations with information gained from qualitative evidence might change how agent-based social simulations look, since they might include more of the "mess" and complexity inherent in the social world we observe. The papers in this special section together make a substantial contribution in this direction^[12].

Notes

¹Feyerabend (1975) rightly pointed out that it is hard to find any cast-iron rules that characterize what we call "science". There will be times when counter-evidence to a theory does not cause its immediate downfall, however if counter-evidence does persist then one should keep an element of doubt about the finality of a theory and, more importantly, maintain efforts to resolve the inconsistency. Thus this characterization is not entirely descriptive but also has a normative flavour.

²Similarly, all kinds of evidence have their own, particular, advantages and this is true of qualitative evidence, for example it can reveal a lot about motivation and subjects' ostensive view of the processes that they are embedded in.

³The development of kinds of data production (which in the quantitative case is called "measurement") is a central, and relatively unsung part of the development of science — a side that gets relatively little status in the social sciences.

⁴There were also non-numerical formalisms, such as logic, set theory, and category theory. However it was hard to apply these to complex targets and infer anything useful.

⁵A formal model is one that is unambiguous, that is it can be communicated many times without being distorted, and where different people will make the same inferences from the same model. In this sense both mathematical and computational models are completely formal.

⁶This was not the only reason qualitative evidence was rejected, of course. Many other factors such as a wish to transcend subjective and emotional domains and attain to a purely "rational" ideal played its part.

⁷An example of a formal yet essentially non-numerical simulation is (Edmonds & Hales 2004).

⁸It would be foolish of me to try and survey these here, but if you are interested you could start with Yang and Gilbert (2008) which covers some of its characteristics and dimensions; you could then browse the methods@manchester resources at: http://www.methods.manchester.ac.uk/resources/categories/qualitative/ and after that read Berg (2007), Mason (2002), and Hammersley and Atkinson (1995).

⁹Notoriously "simplicity" — this (I have argued) is either misplaced (a misnomer for something else that is not really about how simple something is) or a proxy label for limitations on personal resources (which are inevitable, but should be acknowledged honestly on that basis) (Edmonds 2007).

¹⁰Although it is unclear that meaning "resides out there" in the situation, it is clear that some techniques do abstract from the original in a way that loses information and hence potential meaning. For example, an average statistic loses the shape of the original data.

¹¹Again, whilst any attribution of meaning to data could be seen as a distortion, some techniques introduce known systematic distortions.

¹²Some of these papers had their first airing at the 2013 ESSA conference in Warsaw, but have been substantially expanded, updated and revised since.

References

BERG, B. L. (2007) Qualitative Research Methods for the Social Sciences. Pearson/Allyn & Bacon.

EDMONDS, B. (2004) Against the inappropriate use of numerical representation in social simulation. CPM Report 04-131, MMU. <http://cfpm.org/cpmrep129.html>.

EDMONDS, B. (2007) Simplicity is Not Truth-Indicative. In Gershenson, C. et al. (2007) Philosophy and Complexity. World Scientific, 65—80. [doi:10.1142/9789812707420_0005]

EDMONDS, B. (2010) Computational Modelling and Social Theory — the dangers of numerical representation. In Mollona, E. (ed.), Computational Analysis of Firm's Organization and Strategic Behaviour. Routledge, 36—68.

EDMONDS, B. & Hales, D. (2004) When and Why Does Haggling Occur? Some suggestions from a qualitative but computational simulation of negotiation. Journal of Artificial Societies and Social Simulation, 7(2) 9: <https://www.jasss.org/7/2/9.html>.

FEYERABEND, P. (1975) Against Method. New Left Books.

HAMMERSELEY, M. and Atkinson, Y. (eds.) (1995) Ethnography, 2nd edition. Routledge.

MASON, J. (2002) Qualitative Researching, 2nd edition. Sage.

YANG L., & Gilbert N. (2008) Getting away from numbers: Using qualitative observation for agent-based modeling. Advances In Complex Systems, 11(2), 175—185. [doi:10.1142/S0219525908001556]