Reviewed by
H. Van Dyke Parunak
Altarum, 3520 Green Court,
Suite 300, Ann Arbor, MI 48105-1570, USA.
In the early years of social simulation, models were evaluated based on comparisons of their behaviour with that of the target system, comparisons that may be characterised as superficial and qualitative. They were superficial because they were based on the most obvious behaviours of the model, rather than characteristics that can only be derived indirectly. They were qualitative because they focused on a narrative characterisation of these superficial characteristics, rather than a quantitative analysis. A simple example is the emergence of segregation in the garbage can model. This segregation is not defined quantitatively, in terms (for example) of the spatial entropy of the population over time. Rather, the observer is expected to recognise the increased clustering of the agents in a visual display of the model as it runs.
As the field has matured, practitioners have pursued ways to characterise their models that draw on deeper relations among model variables and seek to assess these quantitatively. This movement requires researchers to broaden their analytic toolkit. One important facet of the behaviour of a social model is its evolution over time, requiring researchers to learn methods for the analysis of non-linear time series. This monograph presents a coherent, though highly technical, exposition of one set of techniques for addressing this problem. It summarises a research program conducted by the authors at Siemens Corporate Technology, drawing together in an organised fashion a large number of results previous published in various journals. Their approach is to combine concepts of non-linear dynamics with information theory, leading to the two terms in the title. The book itself offers no explicit application to social simulation or artificial societies, though researchers with a quantitative bent will find its methods readily applicable.
The book begins with an overview introduction (chapter 1) and a summary of dynamical systems theory (chapter 2). The remaining eight chapters are organised as a series of pairs. Each pair deals with a single theme, offering a theoretical development in the first chapter of the pair and example applications in the second.
Chapters 3 and 4 discuss parametric formulations for statistical structure extraction. Chapter 3 relates concepts of information theory (derivatives of the Shannon entropy) to the principle of maximum-likelihood estimation for parameters. The authors' model of choice for dynamical systems is the neural network (both feed forward and recurrent), which they expound in these terms. The final section of chapter 3 raises the traditional tension between supervised and unsupervised learning. It argues that from an information theoretic perspective, the two can be viewed as duals of one another: "maximum-likelihood supervised learning is equivalent to minimising the entropy of the error, which is the goal of unsupervised reduction of redundancy" (p. 69). Chapter 4 demonstrates the application of parametric (neural) models to both artificial chaotic data and real-world time series from biomedicine and hydrodynamics.
Parametric models assume that statistical patterns exist and seek to model them. A more fundamental problem is to determine the existence and nature of those patterns in the first place: are the dynamics stationary or non-stationary, deterministic or stochastic, chaotic or non-chaotic? Chapter 5 introduces a set of non-parametric methods based on cumulants (analogues to the moments of a statistical distribution, when one works with the Fourier transform of the distribution rather than the distribution itself). The first half of the chapter tests whether a single point in the time series is independent of its past by applying the cumulant-based measures to compare the original time series with a surrogate constructed by shuffling the data points in time. (This approach to surrogate data is a common technique in non-linear systems analysis; the authors' contribution is in developing the statistics they use to compare the two series.) For cases in which these techniques detect dependencies, the second half of the chapter offers a way to characterise them, based on an entropy-based measure of the system's sensitivity to initial conditions and the related loss of information through time, a hallmark of chaotic systems. Chapter 6 shows how these techniques can be applied to data, including sunspots, financial markets and epilepsy. The benefits include enabling researchers to find stationary time intervals appropriate for training a parametric model, selecting the optimal time delay in modelling time-series data, and distinguishing white and coloured noise in stochastic systems.
The information flow methods in chapter 5 are based on symbolic dynamics using infinitesimal partitions, which are theoretically convenient but cumbersome in application. Chapter 7 addresses this problem using a semi parametric formulation: a non-parametric measure (based on information flow) is used to select the most appropriate parametric model (from a hierarchy of non-linear Markov processes of increasing order). Chapter 8 returns to the sunspot and biomedical data to demonstrate the application of these methods.
The first three pairs of chapters deal with extracting statistical structure in temporal data. The last pair adds a spatial dimension to consider spatio-temporal dynamics, in particular, spiking networks (such as biological neural networks, in which information is conveyed by the timing of firing events rather than the levels of activation potentials). As in the previous pairs of chapters, chapter 9 develops the theory, and chapter 10 applies it, this time to pattern recognition in artificial neural networks.
A series of appendices offers concise summaries of important results in information theory, cumulants, information flow in chaotic systems, and details of the spiking neuron analysis. The book includes a bibliography with over 170 items (dating up to 2000) and a detailed index.
Readers should recognise that the Information Dynamics approach is not the only approach to the problem, though the authors do a reasonable job of comparing their approach with selected alternatives. Traditional approaches (e.g. Kantz and Schreiber's Non-linear Time Series Analysis) are based on the perspective of state space geometry rather than information flow. The availability of coherent expositions of alternative methods will help practitioners better understand the assumptions and limitations of each, and should greatly increase the quality of research and analysis.
Return to Contents
of this issue
© Copyright Journal of Artificial Societies and Social Simulation, 2004