Abstract

The accumulation of gradual changes in scientific landscape and research practice due to the Internet has the potential to enhance the quality of both cognitive and social aspects of science and scientists. New types of research outputs, modes of scientific communication and new circulation mechanisms, as well as enhanced opportunities for scientific re-use and measuring research impact, in combination with new approaches to research assessment and evaluation are all having profound effects on the social system of science. To be sure that these innovations will not break the social sustainability of the science community, it will be valuable to develop a model of science as a tool for computer simulation of social consequences from possible innovations within virtual research environment. Focusing on possible social problems related to these new virtual research environments this short paper provides a brief analysis of the current situation in science (challenges, problems, main actors), general views on model of science (landscape, main agents, important properties, etc.) and on areas where simulation can contribute to better understanding of possible futures for the scientific community.

Keywords:

Virtual Research Environment, Science System Social Sustainability, Agent Based Modeling

Preface

2.1

Research information systems (RIS) offer great opportunities for progress for the community of scientists but may also poze potential hazards for the sustainability of Science as a social system. The development of communication technology, virtual research environments, and the new possibilities in tracking and monitoring research outputs can radically change traditional relations among the main actors of the Science social system including researchers, managers, funders, and publishers.

2.2

One approach to identifying possible social risks for community is to develop simulations of "Science System" behavior under different scenarios representing possible changes within the research environment, including changing circulation mechanisms for research outputs, developments in science communication, changes to the research outputs life-cycle, the monitoring of the impact of research at both personal and organizational levels, changes in research funding policy, and so on.

2.3

The RIS developers' community alongside the social simulation community could build models of the Science social system to analyze the possible social consequences of RIS technical innovations. It will allow the optimization of the efficiency and effectiveness of research virtual environments in effecting desired changes while avoiding the risk of community breakdown.

Current situation in science

3.1

The internet is rapidly changing scientific circulation mechanisms, research life cycles, science communication and is starting to change the fundamentals general research practice and motivation. Information technology creates a space for radical innovation in many areas of the science system. It allows research funders and government to focus more and more on how the outputs of research are being applied beyond the research community (Neylon 2009a). At the same time the Open Research agenda ( http://en.wikipedia.org/wiki/Open_research;Neylon 2009b) with its focus on enabling and supporting the wider re-use of research outputs is gaining momentum on a range of fronts. Researchers in turn are exploring new approaches of research communication with the aim of increasing the efficiency of communication and the ability for these outputs to reach beyond the research community, yet frequently these non-traditional outputs are seen as "not counting" even when they may have greater impact (Neylon 2010).

3.2

These trends have been supported by different kind of activities within research community.

3.3

The "Beyond the PDF" workshops ( https://sites.google.com/site/beyondthepdf/) discussed the evolution of research communication from the traditional "journal article" to a future "research object" ( http://wiki.myexperiment.org/index.php/Research_Objects). Initial efforts to implement aspects of this vision using commodity internet architectures such as WordPress, and the development of possible shared transport formats such as ScholarlyHTML offer views of potentially radically different scientific communication approaches. These innovation, if implemented, will change many of the central relationships in science, including those between researchers and between researchers and publishers.

3.4

The "Beyond Impact" project ( http://beyond-impact.org/) aims to facilitate a conversation between researchers, their funders, and developers about what we mean by the "impact" of research and how we can make its measurement more reliable, more useful, and more accepted by the research community. At the end the project aims to build a technology prototype that will provide new tools to aggregate and measure evidence of scientific impact and to use it to support research policy and funding decisions. This also has the potential to change traditional relations between researchers, their managers and their funders.

3.5

The euroCRIS professional association ( http://www.eurocris.org/) is dedicated to developing a model of Current Research Information System (CRIS) and Common European Research Information Format (CERIF) which will provide a road to the wide-scale interoperability of research information systems and the possibility of new global circulation mechanisms for this information, supporting an advanced research life cycle and communication. Proposed approaches create a new type of virtual research environment, which offers new mechanisms for the re-use of research outputs, measuring scientific impact and many other related improvements in research assessment, evaluation and funding procedures.

3.6

Thus research practices within these virtual research environments will have many new properties, particularly radically more efficient communication. This new stage is sometimes called "Open Science". It is expected that Open Science landscape for researchers' behavior will have a set of important attributes: a) open access to research outputs; b) open access to usage/impact statistics of research outputs; and c) open access to basic research assessment data accumulated for outputs, researchers and organizations (Parinov 2010a;Parinov 2010b).

3.7

This changing scientific landscape is influencing the behavior of science system actors and creates new challenges for them and for the science system at large:

1. Funders today are more and more conscious of the need to both measure and demonstrate the impact of the work that they fund. However moving beyond simple paper counting to measurement of the wide range of downstream uses is a significant challenge.
   Simulation of the science system can help funders to model, understand, and articulate their needs and desires in being able to track, measure, and quantify the use and re-use of the research that they fund. Simulations also can answer where funders can act to make the tracking and measurement of research impact easier for researchers and developers to provide evidence of impact.
2. Researchers and their managers in today's competitive environment need to be able to demonstrate the impact of their research. Research managers in turn are concerned about tracking and valuing an increasingly diverse range of research outputs. Many researchers are concerned that non-traditional research outputs, even where they maximize the potential for their research to be used, are not regarded as important for career progression.
   Simulations can help to re-engage researchers with the impact agenda by helping them to demonstrate potential downstream impacts and providing a common ground for them to demonstrate and validate the impact of their work with their institutional research managers.
3. There are a wide range of publishers, data aggregators and other commercial tool providers who can provide either information or tools that will support the framework that delivers both diverse and reliable measures of the use and re-use of research outputs.
   Simulation can help this group of actors to play different scenarios to identify requirements and necessary properties of the next generation of research assessment systems. Simulation can also provide direct measures that may be useful in research assessment.
4. The world of research data and particularly data about the research process presents great opportunities for developers with experience in visualisation, data aggregation and analysis. The opportunity exists to build a set of tools that will assist researchers, institutions, and research funders in maximising the wider positive impact of their research."
   Simulation can help developers to analyse the possible social consequences of implementing tools that can change the social relations between scientific actors and affect on a sustainability of science system.

A Model

4.1

Science social system is functioning as a system with a specialization and labor division among its scientific agents (researchers). Scientific agents use the labor division system to interact and to produce scientific knowledge. There are also funding agents who collect data about scientific agents' reputation and who distribute limited compensation (money) for scientific labor according to reputation ranks. In a simple case, research managers might be classed as funding agents. In the more general case managers can be another type of agent with their own rules of behavior. The last type of model agents is publishers. They are responsible for producing reputation data by processing the results of scientific agents' operation.

4.2

Scientific agents do not compete for scientific knowledge (SK) resource allocation. That is, knowledge itself is a non-rivalrous good. The agents can take any SK resources essentially for free. They don't need to carry out exchange with other members of their labor division to take SK resources for personal scientific production process.

4.3

In the model scientific agents can select SK resources for their own production process according to different schemes. To produce a scientific output the agent should create scientific inference and usage relationships (see details below) with the selected SK resources. Data about these relationships could be available for model's agents in different modes.

4.4

Possible differences between forms of research outputs (traditional article or research object) can be represented by amount and quality of relations' data which are available for model's agents.

4.5

Scientific circulation mechanisms are responsible for making new research outputs available for selection and re-use by scientific agents. These can work in different modes (like a traditional system of journals and publishers or as a system of direct access without mediators).

4.6

Scientific agents may have different abilities (at individual level or for all of them) to support and promote their research outputs.

4.7

Interactions between scientific agents within SK labor division system do no create backward distribution of compensation for their scientific labor. Distribution of the compensation among the agents is made by funding agents. Funding agents distribute limited compensation resource (money) among scientific agents according their reputation rank or the rank of their outputs. It can be done it the model according different scenarios.

4.8

Scientific agents without funding drop out from the labor division system at some given speed. In the science system there is no natural mechanism of assessment of scientific agents' impact (since no competition for SK resources). So the "natural" scientific impact data is replaced by artificially produced reputational data. A crucial factor for a viable science system is a relationship between scientific impact and reputation.

4.9

The science system is efficient and sustainable if reputation ranks of agents have strong correlation with scientific impact of these agents. The model can represent different kinds of correlation between these factors.

4.10

Publisher agents are providing some reputation resource for allocation among scientific agents. Reputation resources can be limited or not. A process of reputation resources allocation can be represented in the model by different scenarios with more or less involvement of publisher agents.

4.11

Scientific agents are competing for a higher reputation rank, since it is a basis for a distribution of limited compensation resources.

4.12

Reputation rank depends on accumulated relationships produced by the labor division system and/or by publishers.

4.13

The science system makes relationships with a person and produced outputs by: (1) using outputs for producing new scientific knowledge; (2) gradual connecting outputs with current corpus of the science; and (3) reviewing/assessing the output and/or the scientist personally. Publishers can have different participation in this process of making relationships.

4.14

As a result the science system produces research outputs with several types of relationship networks:

1. scientific inference relationships, which mean a necessity of revision of related outputs, if its parent node is proven wrong;
2. usage/impact relationships, which mean that outputs were used in some sense to create a new one, but it is not a scientific inference;
3. professional reviewing and assessment relationships;
4. hierarchical and associative relationships between outputs.

4.15

The relationship types "b" and "c" allow positive and, as well, negative expressions. E.g. usage relation can have characters like "corrects", "critiques", "refutes", etc. Reviewing/assessment relation can be "disagrees", "parodies", "plagiarizes", etc.

4.16

For each scientific agent the science system collects both forward relationships made by the agent in connection with other agents and/or their outputs, and backward relationships made by other agents for the agent's person and outputs.

4.17

Scientific agent's reputation is a generalization of all relationships collected for him/her by the science system with taking into account a reputation of linked agents. If an agent "A" with "bad" reputation creates a negative relationship for agent "B" that should not decrease reputation of the agent "B".

4.18

Scientific agents seek for higher reputation rank by implementing different scenarios. They can actively create relationships all listed type or only some of them. They can provoke other agents by making relationships with negative values, or they can express only positive esteem, and so on.

Challenges of Simulating the Social Processes of Science

5.1

We have a generational opportunity to make our science system better through effective evaluation and evidence based policy making and architecture development. But we will squander this opportunity if we either take a utopian view of what might technically feasible, or fail to act for a fear of a dystopian future. The way to approach this is through a careful, timely, transparent and thoughtful approach to understanding ourselves and the system we work within by modeling and computer simulations.

5.2

Working on developing virtual research environment we need to understand the measures of the science system we might develop, what forms of researchers evaluation they are useful for and how change can be effected where appropriate. This will require significant work on simulating the social processes of science as well as an appreciation of the close coupling of the whole science system.

5.3

We need computer simulations of possible social effects from using different data and approaches on research evaluation.

5.4

We need computer simulations to understand how we can integrate our mechanisms of scientific recognition and attribution into the way the web works through identifying research objects and linking them together in standard ways.

5.5

Finally we must closely study by computer simulations the context in which our data collection and indicator assessment develops. Science as a social system cannot be measured without perturbing it and we can do no good with data or evidence if we do not understand and respect both the systems being measured and the effects of implementing any policy decision.

5.6

To have progress on this way we propose to establish a task group within some professional association (e.g. European Social Simulation Association) which will take a responsibility for managing possible collective efforts of RIS developers' community alongside the social simulation community.

References

NEYLON C., Wu S. (2009a) Article-Level Metrics and the Evolution of Scientific Impact. PLoS Biol 7(11): e1000242. [doi:10.1371/journal.pbio.1000242 ]

NEYLON C. (2009b) Open Research: The personal, the social, and the political. http://cameronneylon.net/blog/open-research-the-personal-the-social-and-the-political/

NEYLON C. (2010) Metrics of use: How to align researcher incentives with outcomes. http://cameronneylon.net/blog/metrics-of-use-how-to-align-researcher-incentives-with-outcomes/

PARINOV S. (2010a) The electronic library: using technology to measure and support Open Science. In Proceedings of the World Library and Information Congress: 76th IFLA General Conference and Assembly, 2010, http://www.ifla.org/files/hq/papers/ifla76/155-parinov-en.pdf

PARINOV S. (2010b) Open Science: a concept of virtual research environment (in Russian), Working paper. http://socionet.ru/publication.xml?h=repec:rus:mqijxk:24