Abstract

Health care improvement efforts often focus on changing the behavior of individuals while the interdependencies among individuals are overlooked. The application of complex adaptive systems approach to studying healthcare delivery changes the focus of improvement efforts from the individual to the relationships and interdependencies among individuals in the system. Sensemaking and improvising are social activities that take place in the context of relationships between individuals. We explore the impact of sensemaking and improvising on patient outcomes in healthcare settings. We conducted an in-depth observational study of relationships in inpatient general medicine teams. Data collection focused on examining associations between team relationship behaviors of sensemaking and improvising and patient outcomes. Data analysis revealed that these activities were positively associated with patient outcomes. Based on these observational data and findings, we developed an agent-based model to further explore and clarify the relationships between sensemaking and improvising on physician teams and patient outcomes. Specifically, we used the agent-based model to simulate the impact of variation in physicians' sensemaking and improvisation abilities on patient length of stay, the likelihood of a patient worsening or developing complications, the need for transfer to a higher level of care, and mortality. Our in-depth study indicates that systematic differences in patient outcomes are associated with differences in the capacity of physician teams to effectively make sense and improvise in the dynamic conditions inherent in healthcare systems. The results of our simulation demonstrate that an agent-based modeling approach is feasible and useful for exploring the impact of physician team behaviors on patient outcomes. This finding suggests the need for new tools and approaches to improve sensemaking and improvisation in physician care teams as strategies to improve patient outcomes.

Keywords:

Healthcare Delivery, Sensemaking, Clinical Systems

Introduction

1.1

Improving health care system performance has tended to focus on the behavior of individuals. Typical strategies include pushing knowledge to providers through continuing education (Iliffe 2010; Bosworth 2009; Schieffercecker 2008), clinical reminders to prompt providers to deliver indicated care (Fung 2008), or decision support tools to help providers deliver guideline-concordant care (Holbrook 2009; Peiris 2009). Even attempts to improve system performance through quality improvement efforts break processes down into parts performed by individuals, and then work on improving those component parts (Tague 2005).

1.2

The application of a complex adaptive systems (CAS) approach to health care settings changes the foci of improvement efforts from the individual to relationships among individuals in the organization (Plsek 2000). As complex systems are characterized by the interconnections between individuals, the way people relate to each other is important (Cilliers 1998; Lanham 2009; Miller 2010). Additionally, the CAS framework introduces the concepts of non-linearity and uncertainty into our understanding of clinical settings (Plsek 2001). Because of the inherent uncertainty of CASs, like the uncertainty in the clinic from day to day, we cannot reliably plan for how events will unfold. The implications of the CAS framework for how we think about improving clinical systems are profound: first, our focus needs to be on interrelationships, not individuals, and second, strategies to improve the collective ability to act effectively in the face of uncertainty are critical to clinical system performance.

1.3

The literature on uncertainty in organizations suggests that sensemaking and improvising are two strategies that can be used to act in unanticipated and ambiguous situations (Weick 2001; Weick 2004; McDaniel 2005; Blatt 2006; Jordan 2009). Sensemaking has been defined as a social act of understanding events as they unfold, in a way that can lead to action: "Sensemaking is a diagnostic process directed at constructing plausible interpretations of ambiguous cues that are sufficient to sustain action (Weick 1995)." Inpatient teaching rounds, the focus of this article, can be considered an act of sensemaking. On rounds, a group of individuals with different levels of experience and perspectives on the patient comes together to assess the progress of their patients and decide on courses of action.

1.4

Sensemaking and improvising are complimentary, interdependent actions. While sensemaking is the process of interpreting cues and determining action, improvising is varying behavior based on the situation at hand. In the context of physician rounds, a team might improvise by varying what they are doing based on how sick their patients are. Teams also improvise when they determine the treatment plan of patients whose symptoms are unclear, because they are building in some way on their previous actions in similar situations. Together, sense-making and improvising allow people to assess an uncertain situation and proceed with action. For sensemaking and improvising to be effective, the relationships between individuals are critical (Lanham 2009; Edmundson 2003).

1.5

The CAS framework highlights the importance of sensemaking and improvising as adaptations to uncertainty in healthcare systems, and has implications for the ways in which we study and improve clinical systems. Randomized controlled trials and plan-do-study-act cycles are inherently reductionist, as they tend to focus on identifying and isolating single elements of health care systems. Instead, methods that allow us to better understand the dynamic relationships among individuals over time and the variable outputs that can emerge on the system level may be better suited for studying clinical systems with a goal to improving overall system performance.

1.6

The ability to investigate the dynamic relationships among individuals and their system-wide outputs makes agent-based modeling a compelling strategy for studying clinical organizations as complex adaptive systems. This technique has already been used to study disease spread (Rubin 2008; Perez 2009,Dancik 2010) and patterns of drug use (Agar 2005). However, to our knowledge, agent-based modeling has not yet been applied to understand the impact of inpatient physician interactions on patient outcomes.

1.7

The purpose of this research is to look at the ways in which the behavior of physician teams impacts patient care in new ways. To accomplish this, we first use qualitative observational data of sensemaking and improvisation on inpatient physician teams to create an agent-based model to simulate these activities and their impact on patient outcomes. We then use this model to understand the potential impacts of changing the ways in which physicians engage in sensemaking and improvisation. This simulation allows us to see the potential variation in patient outcomes that would not be feasible with other methods.

Methods

Observation of physician team behaviors

2.1

One physician and one PhD student observer with experience in ethnographic methods observed seven inpatient general medicine physician teams during seven month-long data collection periods at the two primary teaching hospitals of the University of Texas Health Science Center at San Antonio. These hospitals include the Audie L. Murphy Veterans Affairs Hospital and University Hospital, the county hospital for Bexar county. Observations were conducted from August 2008 through February 2010. Each physician team is comprised of an attending (faculty physician), resident (physician in second or third year of residency training), two interns (physicians in first year of residency training), and up to three third and fourth year medical students. Typically, patients are admitted to specific interns, who are responsible for being the primary point of contact for patients and staff, while the resident and attending help to oversee management of these patients. Medical students' roles may vary from also admitting patients to simply helping other team members. The entire group of students, interns, resident, and attending rounds daily for several hours to discuss each patient and make decisions regarding management.

2.2

During the data collection period, an observer rounded with each team daily to assess aspects of the relationships among the physician group, as well as strategies they used to make sense of what was happening with their patients and their ability to improvise. As data collection progressed, the attitude of the attending regarding the goal of the team to learn, to take care of patients, or do to both became apparent as an important determinant of team behavior. Because we felt that relationships were critical to and manifest in the sensemaking and improvisation of the team, relationships were not incorporated separately into the model.

2.3

Patient outcomes were also collected on daily rounds, and were assessed during the presentation of each patient.

Definition of Observed Behaviors

2.4

Weick's framework for sensemaking was used to frame our observation of sensemaking behaviors (Weick 1995). This framework encompasses seven strategies for sensemaking including solitary versus social sensemaking, the degree of identity definition, backwards-noticing events, continuous versus episodic sensemaking, the use of possibility and probability, and the use of equivocal cues. Improvising was conceptualized in terms of variation in the amounts of time spent on various activities, as well as whether the teams did things like change the order in which they rounded in response to patient needs. Specific behaviors used as measures of sensemaking and improvisation activities are listed in Table 1. During rounds, observers wrote daily field notes to document their observation of sensemaking and improvising behaviors on rounds. They also documented the time spent by the team on the activities related to rounds, including total time spent, time spent talking about each patient, time spent at the bedside, etc.

Table 1: Specific behaviors used as measures of physician team sensemaking and improvisation
Team attribute	Specific behavioral measures
Sensemaking
Social vs. solitary	o Number of people contributing to patient discussions o Whether tasks are done by individuals or the group o The degree to which decision making appears shared o Information from ancillary services or consultants
Degree of identity definition	Degree to which team members perform tasks outside of hierarchical role
Backward-noticing	Discussion of prior patients with similar presentation, or prior history of patients
Continuous vs. episodic & Enactive vs. reactive	Degree to which active management decisions are made in reaction to a change
Use of possibility / probability & Use of equivocal cues	Pre-emptive, anticipatory discussions related to possible complications, contingency plans Degree to which general medical knowledge is considered in patient care discussions
Improvising
Presence of improvising	Degree of change in daily rounds based on needs of patients: Overall time spent on rounds Time spent discussing individual patients Time at the bedside of individual patient Using novel therapeutic approaches for the treatment of individual patients

Outcome Measures

2.5

While rounding with physician teams, observers also assessed clinical outcomes of each individual patient admitted to teams. Because teams discuss each patient every day, these discussions allowed us the opportunity to assess patient outcomes. Outcome data collected are listed and defined in Table 2, and included length of stay, unnecessary length of stay, clinical status or development of complications, need for increased level of care, or mortality.

Table 2: Outcome measures and operational definitions
Outcome	Definition
Length of stay (LOS)	The total number of days a patient is in the hospital
Unnecessary LOS	Days a patient is in the hospital despite being medically stable for discharge
Clinical status	Assessed as better, stable, or worse depending on the assessment of the physician team. The "worse" category includes the development of a new medical problem while a patient is hospitalized.
Need for increased level of care	Need for a patient to be transferred to a higher level of care than can be provided on the inpatient medicine floors, typically an intensive care unit
Death	Death of a hospitalized patient

Observed Behaviors and their Association with Outcomes

2.6

We observed differences between teams with regard to sensemaking and improvising. While all teams exhibited at east one type of behavior indicative of sensemaking, teams varied in the number of categories of sensemaking behaviors we observed on rounds. For example, on some teams, it was rare for the entire team to round on all of the patients; instead, the attending and resident would round with the interns individually. On a minority of teams, the attending and/or resident would help with the "follow-up" work of writing orders or calling consultants, rather than leaving those tasks to the interns and students. Based on our observations and field notes, we rated teams according to the categories of sensemaking behaviors we routinely observed. We then categorized these ratings into tertiles, as we felt these more meaningfully stratified teams according to sensemaking behaviors than scores of one to seven, particularly since we had a relatively small number of teams.

2.7

To understand improvising, we looked at the field notes regarding examples of improvising observed. These examples included times when the team applied knowledge from one context to another in patient care - for example, trying a treatment that was known to work for one disease to a different disease which might have similar features. We also looked at variability in time data within teams. Two patterns emerged: teams whose behavior varied little, and who rounded in the same way each day, and teams for which there was a great deal of variability in how rounds were conducted with times varying daily. Based on these patterns, we stratified teams in terms of "low" or "high" improvising.

2.8

Finally, we observed differences in how attendings oriented the teams to the month in terms of the expectations they set. Three sub-types of expectations emerged. First were attendings who focused on education, saying things like "you are here to learn … treat me as your consultant and ask questions when you don't know the answer … you have to attend lecture every day … " Second were attendings who clearly set out clinical expectations, such as "discharge starts at admission … All consults must be called before noon conference … all discharge summaries must be done before the patient is discharged … " A third group of attendings highlighted both educational and patient care expectations. One said "we're here to learn and take care of patients." These expectations seemed indicative of the attitude of the attending, and the things they focused on during rounds throughout the month. We then looked at trends in patient outcomes (specified in Table 2) for each team. In preliminary analyses, quantifiable differences emerged between teams with regard to these outcomes, up to differences in likelihood of patient worsening ranging from just under 10% to over 30% between teams. We were able to link differences in our preliminary analyses of clinical outcomes to our stratifications of sensemaking and improvising, as well as to attending attitude. As we show in the next section, we used these associations as the basis for the agent-based model.

Construction of Agent-Based Model

3.1

NetLogo 4.1 (Wilensky 1999) was used to create an agent-based model. In this model, the physician team is treated as a single entity interacting with patients. Each team has the following key attributes which can be varied: attending attitude, improvisation, and sensemaking. These key attributes were identified in the observational data, as described in the preceeding section. Depending on the variations in these key attributes, patients will have differential likelihoods of improving, worsening or developing complications, or being discharged. These likelihoods are based on the observed associations between team behaviors and patient outcomes. For example, teams who had low improvising behavior and were in the bottom tertile of sensemaking tended to have worse patient outcomes, and admitted fewer patients during the course of the month. These trends were exacerbated by having an "educationally focused" attending. Opposite patterns were seen with high sensemaking and improvising teams, particularly with attendings focused both on patient care and education. Table 3 lists the specific ways in which these attributes can be varied by team, and shows the impact that this variation has on patient outcomes.

Table 3: Model parameters and impact on outcomes
Parameter	Potential values	Impact on outcomes
Attending attitude / identity	Education	Patients improve more slowly Increased mortality
	Patient care	No change in patient improvement or mortality
	Both	Patients improve more quickly Decreased mortality
Sensemaking	1-3	Patients improve more slowly Increased mortality
	4-5	No change in patient improvement or mortality
	6-7	Patients improve more quickly Decreased mortality
Improvisation	low	No change in patient improvement or mortality
	high	Patients improve more quickly Decreased mortality

3.2

Patients are admitted by inpatient physician general medical teams in a predetermined pattern according to the admissions schedule. This schedule is determined by the Internal Medicine residency training program. The maximum number of patients, or census, that can be cared for by a single team is twenty patients. The number of patients admitted to the team and their severity (or level of illness) is modeled by a random function that can occur between set parameters dictated by the residency program. Finally, individuals on the team will spend time away from the service because of days off or outpatient clinics. The schedules and census are regulated by stringent parameters set by the Accreditation Council for Graduate Medical Education. Therefore, while there may be some variation from teaching hospital to teaching hospital, the general parameters are consistent nationally. We used these general rules to model the pattern of admissions by physician teams in our model.

3.3

Not infrequently, days occur for inpatient medicine teams when the events of the day do not unfold as planned, and may not seem controllable. This tends to occur on days when a high number of patients have been admitted, patients are more ill than is typically the case, or team members have a day off. The likelihood that a patient may become more ill or stay longer than might otherwise occur increases. We captured this type of occurrence in the model, naming it a "crisis." The development of a crisis is also a random function, but will occur more frequently based on the census, average severity, and absence of a team member. In these circumstances, the likelihood that patients will worsen or stay longer than necessary in the hospital increases.

3.4

Average daily census, length of stay, crisis development, proportion of patients worsening on a given day, transfer to ICU, and mortality rates are outputs of the model that depend on the team attributes and the number and severity of admissions. Simulation was used to explore the differences that would occur in these patient outcomes across team types. Team types were created through combinations of the three parameters outlined in Table 3: attending attitude of education, patient care, or mixed; high, medium, or low sensemaking based on the number of sensemaking approaches or strategies used by the team; and the presence or absence of improvisation in the team's behaviors. A total of 18 combinations resulted. Each combination was run one hundred times. These simulation scenarios were produced and compared using a Netlogo tool called Behavior Space that allows for multiple runs of a model under varying parameter values.

3.5

In summary, the factors that influence patient outcomes are the physician team characteristics of attending attitude, sensemaking, and improvisation, which were set and varied by the investigators, and the number and severity of patients admitted to the team, which was a random function.

Results

4.1

The model was developed over a thirteen-month period, with iterative refinements made based on the results of each additional data collection period. As we refined our analyses of physician team behaviors and how they related to outcomes, we updated the dynamics of the model. Appendix 1 details the final simulation schedule, outlining the specific ways in which the model parameters relate to patient outcomes. The NetLogo code used to create the agent-based model is detailed in Appendix 2. The model can be accessed at http://ccl.northwestern.edu/netlogo/models/community/Agent-Based%20Model. A screen shot of the NetLogo set-up is shown in Figure 1.

Figure 1.

4.2

In this figure, the set-up parameters are in the top left. The center of the figure shows the team in the center, with up to 20 patients admitted to the team. The color of each patient reflects that patient's severity. The blue and red boxes represent patients who went to the ICU or died respectively. The boxes on the lower left show the tallies for the parameters written on each box for the month. The graphs on the right demonstrate the daily census and average length of stay for patients on the team as the month progresses.

4.3

Our model output revealed systematic differences in patient outcomes depending on the characteristics of the physician teams. In general, teams with high sensemaking and improvising had better patient outcomes with regard to all outcomes, including the clinical outcomes of development of complications, need for transfer to increased levels of care, and mortality. When sensemaking and improvisation are low, the attending attitude greatly influences team outcomes, with outcomes being best among teams with a mixed education and patient care approach. As sensemaking and improvisation improve, the impact of attending attitude on patient outcomes is mitigated to the point where it seems to have almost no effect. The variation in outcomes between high sensemaking and improvisational teams is much less than in teams without these capabilities. Appendix 3 shows the graphical distribution of results for length of stay and mortality across teams with different initial attending attitude, sensemaking, and improvising parameters.

4.4

Because of the frequency and number of patient admissions, it is unusual for teams to consistently have a low number of patients admitted to them. However, high sensemaking and improvising teams have a lower length of stay and complication rate, so discharge patients more quickly, giving them greater admission capacity and a lower census than lower sensemaking and improvising teams.

4.5

A subset of the range of differences is shown for length of stay (Table 4) and mortality (Table 5). In Table 4, the impact of physician team attributes on the range of outcomes in days of length of stay (LOS) is shown. LOS represents how long patients would stay in the hospital based on the type of team to which the patients were admitted. Table 5 shows the range of mortality rates projected based on team type. Mortality rates were defined as the number of deaths divided by the total number of patients the team cared for that month. Table 4:

Table 4: Impact of different physician team types on days of length of stay (mean and standard deviation)
	Education	Patient Care	Mixed
No improvisation Low sensemaking	8.4 (0.2)	6.3 (0.2)	5.0 (0.3)
No improvisation Moderate sensemaking	7.4 (0.6)	4.0 (0.1)	3.5 (0.1)
Improvisation High sensemaking	2.7 (0.1)	2.6 (0.1)	2.4 (0.1)

Table 5: Impact of different physician team types on mortality (mean and standard deviation)
	Education	Patient Care	Mixed
No improvisation Low sensemaking	9.4% (6.0)	9.5% (3.9)	5.2% (3.2)
No improvisation Moderate sensemaking	4.1% (3.6)	4.1% (2.7)	4.2% (2.9)
Improvisation High sensemaking	2.0% (1.3)	3.2% (2.8)	2.8% (2.5)

The distribution of outcomes is shown graphically in Appendix 3.

Discussion

5.1

In our observational research, we observed that physician teams differ in how they make sense of what is happening with their patients, and their ability to improvise. The attitudes of the team attendings also differ. These differences in turn are associated with differences in patient outcomes. We successfully created an agent-based model that reflects these differences. This model helps us to better understand the dynamic relationship between physician team characteristics and patient outcomes under varying parameters of number of admissions, severity of diseases among those admitted, and absences of clinical team members

5.2

The process of creating the model helped us to clarify our understanding of the relationship between the model components. As we built the model, we were able to compare outcomes with our observations in a generative approach in which we validated the model against ongoing observation in an iterative fashion (Epstein 1997). We believe that this generative approach, strongly grounded in in-depth observations of physician teams, is a strength of our model. Additionally, this approach makes the potential impact of improved sensemaking and improvisation more apparent. This moderation of the influence of attending attitude by the team behaviors of sensemaking and improvisation initially surprised us, but was consistent with our observations, and with our hypothesis that team-level characteristics would have a greater effect than individual characteristics on patient outcomes. Interestingly, in our observations, attendings focused on education or education and patient care in combination seemed to be associated with teams who were most able to engage in sensemaking and improvising. We hypothesize that this is related to their greater likelihood of bringing general medical knowledge to patient discussions and their being more inclusive of all team members in discussion, allowing for different types of knowledge and perspectives to be part of the sensemaking activity.

5.3

Using this model to simulate outcomes over hundreds of teams indicates that these attributes may in turn lead to significant differences in patient outcomes when aggregated across teams. The range in mortality between low and high sensemaking teams of each attending attitude type (Table 5) is 2.4 to 5.7%, averaging 4.05%. If this average difference were spread across the one hundred and fifty-eight medical schools in the United States, assuming that each has an average of 5 inpatient teams, the potential impact of physician team mortality is in the range of twenty-four to twenty-five thousand patients per year. With the numbers of patients cared for by groups of physicians across the 5,815 hospitals registered in the United States, this number is even greater, potentially rivaling the number of deaths attributed to medication errors in hospitalized patients. This finding speaks to the potential value of using simulation to understand clinical systems. Without the ability to use simulation, understanding impacts of physician team attributes across large numbers of settings would not be possible. Thus, our model provides insights regarding the impact of physician team attributes on patient outcomes that would not be possible with other approaches to understanding this phenomenon.

5.4

These results provide a strong rationale for interventions focusing on the interconnections between individuals and on their interactions to improve clinical system performance. Our model suggests that intervening to improve sensemaking and improvisation may overcome potentially problematic individual attributes to lead to improved outcomes. Strategies to accomplish this may include changing the number and types of interactions between individuals, the patterns of information flow between people in clinical systems, improving individuals' ability to cull information from a variety or sources, and their ability to work with provisional diagnoses. Specific examples based on our observations include involving team members not directly involved in the care of the patient in discussions of a patient's care, specifically asking for multiple opinions before making a decision, explicitly acknowledging "I don't know" and looking to other sources to obtain data, and having conversations with consultants or other staff about patient care.

5.5

Using an agent based model to explore the impacts of physician team dynamics on patient outcomes is a novel application of this technique. Many prior applications of agent based models have focused on spread of an outcome of interest throughout a system, such as spread of disease or spread of drug use, or on processes of care, such as wait times or movement through systems. Instead, we focused on the ways physicians make sense of what is happening to their patients and improvise. While focused specifically on inpatient medicine services, we believe that modeling and simulation will also be appropriate for understanding other types of clinical systems, such as primary care clinics, emergency departments, or surgical wards.

5.6

Our work is limited in several regards. First, it included physician teams in a single type of setting, at the teaching hospitals of a single medical school. Despite this, we hypothesize that the principles underlying the model are universal, and even if adjusted based on other local contexts, would still demonstrate differences in outcomes based on these key attributes. Second, we have not yet prospectively validated this model. However, because our model is based on in-depth observational data and was iteratively refined with each additional data collection period in a generative approach, we believe that it does accurately reflect aspects of patient care. Rather than be concerned about the precision of the effect on outcomes, we are interested in using the model to understand potential impacts that could drive future intervention studies.

5.7

Next steps in the development of this model will include further refinements and adjustments based on future observations. We also plan to introduce an evolutionary component in which the performance of physician teams improves over time, and incorporate new elements to the model that reflect the potential influence of other aspects of care on outcomes, such as nursing care. Finally, we would like to use the model to help us think about potential interventions to improve sensemaking and improvising among physician teams and test the potential impact of these interventions.

5.8

In summary, we have developed a novel agent-based model that helps us to understand the impact of physician team behaviors of sensemaking and improvisation on patient outcomes. Simulation using this model demonstrates the magnitude of this impact and provides a rationale for interventions to improve sensemaking and improvisation as a strategy to improve patient outcomes. It also demonstrates the utility of the modeling approach to understanding social phenomena in healthcare systems.

Appendix 1: Simulation Schedule

Setup:

7.1

1 team will be run at a time, for 30 days. Teams will start with a random number of patients, ranging from 10 - 15. Each patient on the team will have a random severity of between .5 and 1. Team strategies include:

1. Education (EC)
2. Patient care (PC)
3. Combination (CO)

The other options for set-up include:

1. Sensemaking
2. Improvisation

Rules for how the patients flow:

How teams get patients:

1. Every 5 ^th day, the team is on call , and can get up to 10 new patients, up to a maximum team cap of 20 (i.e., if they start with 15, they can only get 5 more). This can be randomly generated, but on average, should be 6-8 patients.
2. Every 2 ^nd and 3 ^rd day, the team is on short-call and will receive 1-4 patients, again up to a maximum of 20. This also should be random, but on average should be 2-3 patients.

The severity of these patients should be randomly distributed between 0.7 and 1. Patients leaving the team -3 options:

1. Improvisation and discharge . Patients will get better at the following rate:
   1. ED - by .15
   2. PC - by .175
   3. CO - by .20
   Once a patient has a severity of 0.1, they will be discharged. These rates will be mitigated by the following:
   1. Sensemaking:
      1. For scores of 1-3, subtract 0.05 from above rates
      2. For scores of 4-5, keep as above
      3. For scores of 6-7, add 0.05 to above rates
   2. Improvisation:
      1. For high, add 0.05 to above rates
      2. For not high, leave as above
      On some days, patients will get sicker - this can happen no matter what. It will happen with equal frequency on each type of team theoretically, but less or more depending on the sensemaking and improvisation as follows:
      1. Baseline 5% chance that a patient gets sicker on any given day.
      2. This chance decreases to 4% for 6-7 sensemaking and high improvisation.
      3. This chance increases to 7% for 1-3 sensemaking and low improvisation.
2. Transfer to ICU / higher level of care. Patients with a severity of .8 or above on admission will have a 2% chance of needing to go to the ICU.
   1. Chance will increase by 1% if sensemaking 1-3 and low improvisation.
   2. Chance will decrease by 0.5% if sensemaking 6-7 and high improvisation.
3. Death.There is a 1% chance that a patient will die during their hospitalization. This is mitigated by the following:
   1. Chance will increase by 0.5% if sensemaking 1.3 and low improvisation.
   2. Chance will decrease by 0.5% if sensemaking 6-7 and high improvisation.

Sometimes things start to unravel... this is a crisis.

7.2

A crisis will occur if:

1. Census is >17 with average acuity >0.5
2. Census is > 15 with average acuity is >0.6

7.3

In this crisis, the chances of a patient getting sicker increase are as follows:

1. 20% chance that a patient will go to ICU with 6-7 sensemaking and high improvisation
2. 30% chance that a patient will go to ICU with 4-5 sensemaking and low improvisation
3. 40% chance that a patient will go to ICU with 1-3 sensemaking and low improvisation

7.4

Also in a crisis, the daily improvisation for all patients will decrease by .05 for all until the criteria for a crisis are no longer met. Patients will be discharged when they reach a severity of 0.2 on these days.

Appendix 2: Specific NetLogo programming language used to create the inpatient team model

;The following lines are breed declarations.
;They announce that a certain class of agent (e.g. patients)
;in the model will be in use, and that they are unique from
;other agents in at least one way
breed [doctors doctor]
breed [Nosocomial-Counters Nosocomial-Counter]
breed [discharges discharge]
breed [ICU-Patients ICU-Patient]
breed [Dead-Patients Dead-Patient]
breed [beds bed]
breed [patients patient]
breed [crises crisiss]

;The following lines declare global variables.
;These are variables that can be called from anywhere
;in the program, and they are shared by all the agents
globals [
  Healrate
  Date
  Simulation-Day
  Average-Acuity
  Current-Census
  Day-Report
  Clinic-Day
  Vacation-Day
  Crisis
  postCall
  ]

;The following lines declare breed specific variables
patients-own [severity 
days 
discharge-ready 
extra-day 
stay-extra 
discharge-sequence 
ICU-Chance 
Death-Chance 
Nosocomial-Chance 
ICU-Random 
Death-Random 
Nosocomial-Random 
Infection]
patches-own [empty-bed]

;The following lines are executed in sequence upon clicking "setup"
to setup
  clear-all 
  
;Colors select patches of background shades of white, blue, and red
  ask patches with [pycor > 10 and pycor < 40] [set pcolor white] 
  ask patches with [pycor < 10 and pxcor < 20] [set pcolor 92] 
  ask patches with [pycor < 10 and pxcor > 20] [set pcolor 13] 
  ask patches with [pycor = 20 or pycor = 30 and pxcor > 5 
		and pxcor < 35 and pxcor mod 3 = 0] [set pcolor 99] 
  ask patches with [pycor = 21 or pycor = 31 and pxcor > 5 
		and pxcor < 35 and pxcor mod 3 = 0] [set pcolor 99.1] 
  
;Calls all agents defined a certain type to have a recognizable shape
  set-default-shape beds "square" 
  set-default-shape patients "person" 
  set-default-shape ICU-Patients "person" 
  set-default-shape Dead-Patients "person"
  set-default-shape crises "square"
  set-default-shape Nosocomial-Counters "square"
  set-default-shape doctors "person doctor"
  
;Calls those patches with a specific shade of a blue to produce beds
  ask patches with [pcolor = 99] 
		[sprout-beds 1 [set size 1 set color 99 set heading 0]] 
 
;Creates a doctor in the center of the map
  create-ordered-doctors 1 [
		set size 4 
		fd 25 
		rt 90 
		fd 19.8 
		rt 270 
		set color 102] 
  
;Calls the running of a procedure called "initial"
  initial 
  
;Counts the number of patients and sets this to the variable "Current-Census"
;Takes the mean of each individual patient's "severity" variable,
;this is set to the variable "Average-Acuity"
  ask patients [ 
    set Current-Census (count patients) 
    set Average-Acuity (mean [severity] of patients) 
  ]
end

to initial
;Assigns the variable "x" a random value between 10 and 15
;The following code is repeated the variable "x" number of times
  let x (10 + random 6) 
  repeat x [ 
    let stop-search false
    let i 0
    
;The following code is repeated 20 time;
;its purpose is to find a bed that does not have a patient;
;when an empty bed is found a new patient is created
    repeat 20 [ 
      if stop-search = false [
        ask bed i [
          if not any? turtles-on patch-ahead 1 [
            hatch-patients 1 [
              set size 2.5
              set severity (.500 + random-float .300)
              move-to patch-ahead 1
              set color (20 - (severity * 10))
              set Infection false
              ]
            set stop-search true
            ]
          ]
        ]
      set i (i + 1)
      ] 
    set stop-search true
    ]
end

;This procedure has the same purpose as and shares code with
;the empty bed finding algorithm in the procedure "initial".
;It has been separated so that patients can be admitted
;in the middle of a simulation instead of during the setup
to admit-patient 
  let stop-search false 
  let x 0
  repeat 20 [
    if stop-search = false [
      ask bed x [
        if not any? turtles-on patch-ahead 1 [
          hatch-patients 1 [
            set size 2.5
            set severity (.700 + random-float .300)
            move-to patch-ahead 1
            set color (20 - (severity * 10))
            set Infection false
            ]
          set stop-search true
          ]
        ]
      ]
    set x (x + 1)
  ] 
  set stop-search true
end

to go
;The following lines determine the variable "Healrate" depending
;on which team type has been selected
  if Select-Team = "Education" [set Healrate 0.10]
  if Select-Team = "Patient Care" [set Healrate 0.125]
  if Select-Team = "Combination" [set Healrate 0.15]
  
;Depending on the date, where 6 and 7 represent the weekends,
;the variable "Healrate" can be halved
  if Date = 6 or Date = 7 [set Healrate (Healrate * .5)]
  
;Given the two situations in which there is a patient census of > 17
;and an average-acuity of > 0.7 OR if the patient census is > 15
;and the average-acuity is > 0.8, the variable "crisis" will be set to "true"
  if (count patients > 17) and (mean [severity] of patients > .7) 
[set Crisis true]
  if (count patients > 15) and (mean [severity] of patients > .8) 
[set Crisis true]
  
;Given a low or high sensemaking level, the healrate is further adjusted
;by -0.05 or +0.05, respectively
  if Sensemaking < 4 [set healrate (healrate - .05)]
  if sensemaking > 5 [set Healrate (Healrate + .05)]
  
;Given a state of improvisation or crisis, the healrate is again adjusted
;by +0.05 or -0.05, respectively
  if Improvisation = true [set Healrate (Healrate + .05)]
  if crisis = true [set Healrate (Healrate - .05)]
  
;The following code is repeateley executed until 30 days have elapsed
  if Simulation-Day < 30 [
    
;if the model is in a crisis state, an agent representing the crisis is
;created as a marker for behaviorspace.
;multiple crisis markers can be created over the course of
;the simulation, and these markers are counted by behaviorspace
    if crisis = true [create-ordered-crises 1 [setxy 0 0]]
    
    ask patients [
;Setting default values for variables
        set Nosocomial-Chance .05
        set ICU-Chance 0
        set Death-Chance 0
        
;Given the condition that sensemaking is < 4 and improvisation is off, 
;the variable "nosocomial-chance" is increased by 0.02
        if Sensemaking < 4 [
          if improvisation = false 
                  [set Nosocomial-Chance (Nosocomial-Chance + .02)] 
        ]
        
;Given the condition that sensemaking is > 5 and improvisation is on, 
;the variable "nosocomial-chance" is decreased by 0.01
        if Sensemaking > 5 [
          if Improvisation = true 
                  [set Nosocomial-Chance (Nosocomial-Chance - .01)] 
        ]

;Given the date is a clinic-day or vacation-day, 
;the variable "nosocomial-chance" is increased by 0.02
        if (date = Clinic-Day or date = Vacation-Day) 
                  [set Nosocomial-Chance (Nosocomial-Chance + .02)]
        
;Given the team has an education based strategy or improvisation is off, 
;the variable "nosocomial-chance" is increased by 0.003
        if Select-Team = "Education" or Improvisation = false 
                  [set Nosocomial-Chance (Nosocomial-Chance + .003)]
        
;Given that there is a crisis, the patient with the highest severity level 
;has a large increase in chance of being sent to the ICU. This chance, 
;denoted by the variable "ICU-Chance", is dependent on the 
;sensemaking level and whether improvisation is on or off
        if crisis = true [
          ask max-one-of patients [severity] [
          if Sensemaking > 5 and Improvisation = true 
                  [set ICU-Chance .15]
          if (Sensemaking = 4 or Sensemaking = 5) 
                  and Improvisation = false 
                  [set ICU-Chance .20]
          if Sensemaking < 4 and improvisation = false 
                  [set ICU-Chance .25]
          ]
        ]
        
;Increase the days counter by one
        set days (days + 1)
        
;On a patient's first day, the chances of being transferred to the ICU 
;and the chance for death are determined. These depend on sensemaking 
;levels and whether improvisation is on or off
        if days = 1 [
        if Sensemaking < 4 
              [if improvisation = false 
              	[set ICU-Chance (severity * .05 * 1.1) 
              	set Death-Chance (severity * .03 * 1.1)
              	]
              ]
        if Sensemaking = 4 or Sensemaking = 5 
				[set ICU-Chance (severity * .05) 
				set Death-Chance (severity * .03)
				]
        if Sensemaking > 5 
				[if Improvisation = true 
					[set ICU-Chance (severity * .05 * 0.9) 
					set Death-Chance (severity * .03 * 0.9)
					]
				]
			]				
        
;The color of the patient on screen is a function of its severity: 
;a darker color represents a more severe state
        set color (20 - (severity * 10))
        
;The levels at which a patient is ready for discharge is 
;dependent on whether or not a crisis is in effect. In a crisis, 
;patients are considered to be ready for discharge at a 
;higher severity state (up to 0.2 vs. 0.1 outside of a crisis)
        if crisis = false [if severity = .1 or severity < .1  
					[set discharge-ready true] ]
        			if crisis = true [if severity = .2 or severity < .2 
						[set discharge-ready true] ]
        
;If a patient is ready for discharge, there is a 20% chance 
;that the patient may have to stay for an extra day. The variable 
;"extra-day" can be decreased to 15% if improvisation is on, 
;or it can be decreased to 10% if the team is a patient care team
        if discharge-ready = true [
          set extra-day .20
          if Improvisation = true [set extra-day .15]
          if Select-Team = "Patient-Care" [set extra-day .10]
          if stay-extra = true  [set discharge-sequence true]
          if extra-day < (random-float 1.00) [set stay-extra true]
          if extra-day > (random-float 1.00) [set discharge-sequence true]
        ]
        
;If the patient is ready to be discharged, an agent representing 
;discharges is created to be counted by BehaviorSpace. Multiple 
;discharge counters can be created. After creating the counters,
;the patient is removed from the simulation using the "die" command
        if discharge-sequence = true [
          hatch-discharges 1 [setxy 0 40 set size .01 set color black]
          die
        ]

;The variable "ICU-Random" represents the chance that
;the patient will be sent to the ICU

        set ICU-Random random-float 1.00

;If the patient's severity state is high enough (> 0.8) and 
;the variable "ICU-Random" is less than the variable 
;"ICU-Chance", the patient is transferred to the ICU
        if severity > .8 [
        if ICU-Random < icu-chance [
          hatch-icu-patients 1 [setxy 1 5 set color pcolor]
          die
          ]
        ]
        
;a variable "death-random" is chosen at random, and if this value 
;is less than the value of "death-chance" the patient dies
        set death-random random-float 1.00
        if death-random < death-chance [
          hatch-dead-patients 1 [setxy 22 5 set color pcolor]
          die
        ]
        
;a variable "nosocomial-random" is chosen at random, and if this 
;value is less than the value of "nosocomial-chance" the patient
;is infected. an infected patient has its severity state worsened 
;by a function of the healrate (severity + healrate * 2)
        set nosocomial-random random-float 1.00
        if nosocomial-random < nosocomial-chance 
					and infection = false [
          		set severity (severity + Healrate * 2)
          		hatch-Nosocomial-Counters 1 [
                    setxy 40 40 set size .01 set color black
                    ]
          set Infection true
        ]
        
;Each day a patient's severity state is reduced by the value of the Healrate
        set severity (severity - Healrate)         
    ]
    
;The following are calls to three separate procedures detailed below
    Admission-Check
    Update-Calendar
    Graph-Data

;Increases the counter "Simulation-Day", which is used to admit 
;more patients on certain days
    set Simulation-Day (Simulation-Day + 1)
    
;At the end of a day, the variable crisis is set to false as a crisis 
;only lasts for a single day
    set Crisis false
    
;A tick is a unit of internal time measurement within NetLogo
;This command increases the tick count by one
    tick
  ]
end

;The admission-check procedure allows patients to be added to the service
;Postcall allows 1 to 4 patients to be added if the variable "postCall" is true

to Admission-Check
  if postCall = true [
    let x (1 + random 4)
    repeat x [admit-Patient]
    set postCall false
  ]
  
;On every 5th day, 5 to 15 patients can be added to the service.
;A counter keeps the patient capacity at a maximum of 20
  if Simulation-Day = 5 or Simulation-Day = 10 or 
Simulation-Day = 15 or Simulation-Day = 20 or 
Simulation-Day = 25 or Simulation-Day = 30  [   
    let i (1 + random 10)
    if i < 6 [set i (1 + random 10)]
    if i < 16 [set postcall true]
    while [(count patients + i) > 20] [set i (i - 1)]
    repeat i [admit-Patient]
  ]
  
;On the 2nd or 3rd day after each 5th day, 1 to 4 patients can 
;be added to the service
  if Simulation-Day = 2 or Simulation-Day = 3 or 
Simulation-Day = 7 or Simulation-Day = 8 or 
Simulation-Day = 12 or Simulation-Day = 13 or 
Simulation-Day = 17 or Simulation-Day = 18 or 
Simulation-Day = 22 or Simulation-Day = 23 or 
Simulation-Day = 27 or Simulation-Day = 28 [
    repeat (1 + random 4) 
    [ admit-Patient ]
]
end

;The update-calendar procedure allows the day to be reported 
;where 1 is Monday, 2 is Tuesday, etc.
to Update-Calendar
  set Date (Date + 1)
  if Date = 8 [Update-Schedules set Date 1]
  if Date = 1 [set Day-Report "Monday"]
  if Date = 2 [set Day-Report "Tuesday"]
  if Date = 3 [set Day-Report "Wednesday"]
  if Date = 4 [set Day-Report "Thursday"]
  if Date = 5 [set Day-Report "Friday"]
  if Date = 6 [set Day-Report "Saturday"]
  if Date = 7 [set Day-Report "Sunday"]
end

;The update-schedules procedure selects a random day 
;in the week to be a clinic-day and a separate day to be 
;a vacation-day
to Update-Schedules
  set Clinic-Day (random 8)
  set Vacation-Day (random 8)
  while [Clinic-Day = Vacation-Day] 
[set Vacation-Day (random 8)]
end

;Graphs the data on the plots next to the simulation
to Graph-Data
  set-current-plot "Current Census"
  set-current-plot-pen "Census"
  plot (count patients)
  set-current-plot "Average Length of Stay"
  set-current-plot-pen "Average"
  plot (mean [days] of patients)
end

Appendix 3: Graphical distribution of outcomes based on team attributes

Graphs 1-9: Length of Stay Distributions

Graphs 1 - 3: Attending education strategy, varying sensemaking and improvising

Graph 1

Graph 2

Graph 3

Graphs 4-6: Attending patient care strategy, varying sensemaking and improvising

Graph 4

Graph 5

Graph 6

Graphs 7-9: Attending education and patient care strategy, varying sensemaking and improvising

Graph 7

Graph 8

Graph 9

Graphs 10-18: Mortality Distributions

Graphs 10-12: Attending education strategy, varying sensemaking and improvising

Graph 10

Graph 11

Graph 12

Graphs 13-15: Attending patient care strategy, varying sensemaking and improvising

Graph 13

Graph 14

Graph 15

Graphs 16-18: Attending education and patient care strategy, varying sensemaking and improvising

Graph 16

Graph 17

Graph 18

Acknowledgements

The research reported here was supported by the National Institute of Health (NIDDK Grant R18DK075692) and the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. Investigator salary support is provided through this funding, and through the South Texas Veterans Health Care System. Dr. McDaniel receives support from the IC2 Institute of the University of Texas at Austin.

The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.

References

AGAR, M (2005) Agents in Living Color: Towards Emic Agent-Based Models. Journal of Artificial Societies and Social Simulation 8 (1) 4 https://www.jasss.org/8/1/4.html

BLATT R, Christianson, MK, Sutcliffe, KM, and Rosenthal, MM (2006) A sensemaking lens on reliability. Journal of Organizational Behavior 27 pp.897-917. [doi:10.1002/job.392]

BOSWORTH HB, Olsen, MK, Dudley, T, Orr, M, Goldstein, MK, Datta, SK, MdCant, F, Gentry, P, Simel, DL, and Oddone, EZ (2009) Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. American Heart Journal 157 (3) pp.145-6. [doi:10.1016/j.ahj.2008.11.003]

CILLIERS P (1998) Complexity and Postmodernism: Understanding Complex Systems. New York, NY: Routledge.

DANCIK GM, Jones, DE, ad Dorman KS. Parameter estimation and sensitivity analysis in an agent-based model of Leishmania major infection. Journal of Theoretical Biology 262 (3) pp. 398-412. [doi:10.1016/j.jtbi.2009.10.007]

EDMUNDSON A (2003) Speaking up in the Operating Room: How Team Leaders Promote Learning in Interdisciplinary Action Teams. Journal of Management Studies 40(6) pp.1419-1452. [doi:10.1111/1467-6486.00386]

EPSTEIN J (1997) Nonlinear Dynamics, Mathematical Biology, and Social Science. Boston: Addison-Wesley. FUNG CH, Tsai, JS, Luluejian, Glassman, P, Patterson, E, Doebbeling, BN, and Asch, SM (2008) An evaluation of the Veterans Health Administration's clinical reminder system: a national survey of generalists. Journal of General Internal Medicine 34 (4) pp. 392-8.

HOLBROOK A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, and Gerstein H (2009) Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ 181(1-2) pp.37-44. [doi:10.1503/cmaj.081272]

ILIFFE S, Wilcock, J, Griffin, M, Jain, P, Thune-Boyle, I, Koch, T, and Lefford F (2010) Evidence-based interventions in dementia: a pragmatic cluster-randomised trial of an educational intervention to promote earlier recognition and response to dementia in primary care. Trials 11:13. [doi:10.1186/1745-6215-11-13]

JORDAN ME, Lanham, HJ, Crabtree, BF, Nutting, PA, Miller, WL, Stange, KC, and McDaniel, RR (2009) The role of conversation in health care interventions: enabling sensemaking and learning. Implementation Science 4(15). [doi:10.1186/1748-5908-4-15]

LANHAM HJ, McDaniel, RR, Crabtree, BF, Miller WL, Stange, K, Tallia, A, and Nutting P (2009) How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Safety 35 (9) pp. 457-466.

MCDANIEL RR, and Driebe, D (2005) Uncertainty and Surprise in Complex Systems, Springer Verlag Heidelberg. [doi:10.1007/b13122]

MILLER WL, Crabtree, BF, Nutting PA, Stange KC, and Jaen CR (2010) Primary care practice development: a relationship-centered approach. Annals of Family Medicine 1:S pp. 68-79. [doi:10.1370/afm.1089]

PEIRIS DP, Joshi, R, Webster, RJ, Groenestein, P, Usherwood, TP, Heeley, E, Turnbull, FM, Lipman, A, and Patel, AA (2009) An electronic clinical decision support tool to assist primary care providers in cardiovascular disease risk management: development and mixed methods evaluation. Journal of Medidcal Internet Research 11(4):e51. [doi:10.2196/jmir.1258]

PEREZ L, and Dragicevic, S. An agent-based approach for modeling dynamics of contagious disease spread (2009) International Journal of Health Geographics 8 (50).

PLSEK P (2000) Redesigning Health Care with Insights from the Science of Complex Adaptive Systems. In: Crossing the Quality Chasm: A New Heath System for the 21st Century: National Academy of Sciences p.. 309-22.

PLSEK PE, and Greenhalgh, T (2001) Complexity science: The challenge of complexity in health care. BMJ 323(7313) pp. 625-8. [doi:10.1136/bmj.323.7313.625]

RUBIN MA, Mayer, J, Greene, T, Sauer, BC, Hota, B, Trick, W, Jernigan, JA, and Samore, MH (2008) An agent-based model for evaluating surveillance methods for catheter-related bloodstream infections. AMIA Annual Symposium Proceedings pp. 631-5.

SCHIEFFERCECKER KE, Reed, VA, and Homa, K (2008) A training intervention to improve information management in primary care. Family Medicine 40 (6) pp. 423-32.

TAGUE N (2005) 'The Quality Improvement Process' in The Quality Toolbox, 2nd Ed pp. 35-51.

WEICK KE (1995) Sensemaking in organizations. Sage Publications, Thousand Oaks, CA.

WEICK KE, and Sutcliffe, KM (2001) Managing the unexpected: assuring high performance in an age of complexity. University of Michigan Business School Series.

WEICK KE, and Roberts, KH (2004) Collective mind in organizations: heedful inter-relating on flight decks. Administrative Science Quarterly, pp. 357-81.

WILENSKY U (1999) "NetLogo." Evanston IL: Center for Connected Learning and Computer Based Modeling, Northwestern University, http://ccl.northwestern.edu/netlogo/