Largest immersive Operation Outbreak simulation at Walter Johnson High School!

By Andrés Colubri —

A little more than two weeks ago, the largest Operation Outbreak (OO) simulation to date in a high school took place: on October 14th, nearly 400 students played a fully immersive epidemic scenario for almost 3 hours at Walter Johnson High School (WJHS) in Bethesda, Maryland. They had to keep a functioning “society” running while an unknown and highly contagious virtual pathogen spread through student’s phones via the Bluetooth connection. Like in the original outbreak experiential activity that Dr. Todd Brown developed back in 2015, students adopted roles as members of the general population, government, health workers, epidemiologists, and others. The goal was to administer resources, identify the pathogen causing the disease, and eventually discover and distribute a vaccine to contain the outbreak. But, different from the original activity where stickers were used to simulate transmission, here we were able to deploy the latest version of a sophisticated technology platform that involves not only the OO smartphone app but also cloud infrastructure and web-based tools. This platform makes it possible to simulate a realistic epidemiological model “in real-life,” allowing students to experience the spread of a virtual pathogen in an engaging hands-on educational activity about infectious disease outbreaks.

Students from Walter Johnson High School playing the OO simulation in the sports court area of the school in the morning of October 14th, 2022

As we have described in earlier publications (see the press section of the OO website for more information), we introduced the app-based simulation in Sarasota, Florida in 2018, which complemented the societal game that helped students learn about decision-making during a health emergency. The use of the app greatly increased the realism of the simulation, based on our observations of the previous OO pilots. Ironically, the COVID-19 pandemic severely restricted our ability to keep piloting the immersive in-person version of the simulation due to the school closures and other restrictions introduced to contain the real-world pandemic, which affected educational institutions in the US and all around the world. Even though we were able to conduct a number of large-scale simulations during COVID, most notably at Colorado Mesa University and Brigham Young University, resulting in valuable data for epidemiological research, these simulations took place at college campuses instead of middle- or high-schools and followed a more “passive” data-generation mode. In this mode, participants simply had the OO app running in the background of their phones while they went about with their daily routines. They occasionally also wore (digital) masks or took (virtual) rapid diagnostic tests via QR codes available on campus to simulate the effect of various public health interventions. The main strength of these campus-wide, multi-day simulations is that they can generate anonymous contact data that reveals potential transmission chains and other social network properties that could be useful to implement better strategies to respond to real-world outbreaks. But these simulations put less emphasis on the educational goals of the project, which we are aiming to realize with the immersive role-playing mode of OO, particularly in middle- and high-school settings.

The simulation at WJHS represented a long-awaited return to the fully immersive OO format, which we hope to continue deploying and expanding in the near future. But it was important also because it was the first OO immersive simulation that was organized and run by other people, with us only providing overall guidance. This simulation was possible because Todd and I were invited earlier this year, thanks to Dr. Pardis Sabeti’s referral, to talk about OO at the Short Course in Genomics from the National Human Genome Research Institute (NHGRI), part of the National Institutes of Health. This course offers science educators the opportunity to hear lectures and receive teaching resources from researchers, clinicians, and staff. Among the attendees to our talk was Nichole Kellerman, science teacher at WJHS, who expressed interest in running the fully immersive version of OO at her school. This initial contact led to the possibility of this new pilot, and eventually to the successful simulation on October 14th. It incidentally also became the largest OO simulation to date at a high-school setting, with nearly 400 students from 9th through 12th grades. Todd provided detailed guidance and materials to Nichole and other participating teachers in the weeks leading up to the OO simulation, so they were able to run it by themselves. Todd and I, together with other members of the OO team at UMass Chan and the Broad Institute, attended the simulation in-person simply as spectators to observe and collect field notes that we will use to evaluate and improve the platform in the coming weeks and months.

Student epidemiologists in the epidemiology station (left), members of the general population getting tested at the triage station (center), and students showing their health status in the app (right).

A screenshot of the symptoms presented by a sick player during the simulation (left). Teacher-led student reflection after the simulation, where students from each group shared their experience and answered questions from their peers (right). BTW, the disease was MRSA, and the students eventually figured it out based on the symptoms, which lead to the availability of an “experimental vaccine.”

An immersive OO simulation involves several components that must work together smoothly, as the students carry out different but interrelated tasks meant to represent what can happen in a society when an infectious disease strikes. A key concept is that of managing resources during a health emergency, and in order reflect this, the simulation introduces an economic system where all participants must interact with each other to obtain supplies needed to survive (food) and to defend themselves against the disease (masks, PPEs, rapid tests, vaccines.) Students obtain money tokens by answering quizzes placed around the school, which they can then use to buy food, masks, etc. All these exchanges take place at stations found at specific locations, representing stores and banks. (And of course, these vital interactions also carry a risk of contagion, just like in real life). But this is not where the role-playing ends. In order to further incorporate learning on public health and governance, there are additional stations that represent triage centers where students get tested and report symptoms, epidemiology units that monitor the data collected during the simulation to try to identify the pathogen, and a central government tasked with administering a budget that can be allocated in different ways to produce resources, conduct research, and carry out public health interventions (for example, distribute a vaccine once the epidemiologists discover the disease-causing pathogen.) Some students are assigned to each one of the stations, while the rest play as members of the general population. All of this requires careful preparation to ensure that students and any other people assisting with the execution of the simulation are familiar with the tasks they need to carry out, everyone has all the required materials, and students know how to install and use the OO app. All of these interrelated pieces (quiz taking, economy, health care, government) are put together in a way that they lead to inherent conflict (in-person interaction is needed to survive, but could also spread the disease) and, perhaps most importantly, create an open-ended space for social dynamics and behaviors from the students (as in previous simulations, here we witnessed unscripted events such as claims of corruption in the government, price gauging of protective supplies, even theft of money from the bank station.) The simulation at WJHS allowed us to test all these role-playing elements as they were interpreted and executed by a group of teachers who were new to the platform.

This simulation also represented a large-scale setting to test the latest version of the OO technology, including the app, backend, and supporting tools, which was the result of the hard work of many members in my lab at UMass Chan Medical School. The app itself introduced several improvements and new features, including a set of animated characters that reflect the health status of the player, designed by Mansi Khandpekar, co-op graduate student from the Information Design and Data Visualization program at Northeastern University, building on top of the earlier UI redesign by Yinan Dong, the UX/information visualization designer in the lab. The app was in fact entirely re-implemented from scratch by Andi Grozdani, the app developer in the team, with the help of Shannen Lin, undergraduate CS student from Worcester Polytechnic Institute (WPI), using the Flutter mobile framework that enabled us to unify the previously separate iOS and Android apps into a single codebase. The backend infrastructure was also greatly improved by Hung Hong, the software engineer in the project, to support new simulation parameters and integration with the OO web tools for editing and visualizing the simulations. Last, but not least, the OO app uses the Herald Proximity project, which offers a cross-platform library to build applications that rely on Bluetooth for proximity sensing. The lead developer of Herald, Adam Fowler, provided significant amount of support (as well as around-the-clock fixes), which permitted us to correctly integrate Herald into the app. Over the summer, Ben Kresge, undergraduate CS intern from WPI, implemented custom Herald-based functionality that made it possible to convert raw Bluetooth signal data into estimated distance values. This is a quite hard problem on its own, as shown by the challenges faced by the digital contact tracing apps that were deployed around the world to try to contain the spread of COVID. If you are interested in using Herald from a Flutter-based app, please check the source code of this demo app that is freely available from our lab’s GitHub page :-)

Screen captures from the latest version of the OO app (left). Animations of one character showing the possible health states of the player, and all the different characters currently available in the app in the bottom (right).

The OO web editor and visualization dashboard (left), and a picture of the app design & development team with Mansi, Yinan, Hung, and Andi (right).

We recently published a paper showing how the contact data from OO simulations can be used to characterize contact networks in a school setting. These networks can subsequently inform epidemiological predictions to optimally apply diagnostic testing and quantify the exposure of individuals to cryptic transmission chains through second degree contacts, among other calculations we shown in the paper. For the latest simulation at WJHS, you can see detailed animated plots of the data generated by the OO app, including contacts and transmissions between participants, full transmission networks, as well as number of cases, contacts, and new infections over time, in the following composite video (the code we used to generate these plots is available here):

The plot in the top right area of this video shows the contacts, as detected by the OO app using Bluetooth, between students during the simulation (each circle represents one student with the following color coding: blue=susceptible, orange=infected, gray=dead, green=recovered, violet=vaccinated; coding which applies to the other plots as well), and infections in the moment they take place (with an orange arrow connecting the infecting and susceptible students). Please note that position of the dots does not represent physical location, as the app does not record GPS data, it’s just the way the plot automatically arranges the circles, so they all fit the screen. In the plot in the bottom right, only infected students are shown, and they are added to the plot as they become infected. The lines represent the infections, and all dots remain connected to help understand the growing chains of transmission. The isolated dots are “index cases” that the app assigned randomly since the beginning of the simulation (even though we set only 10 index cases in the parameters of the simulation, it’s clear that there were many more, pointing to a potential bug we need to identify and fix in the app). The three charts on the left side (animated at the same speed so they match the two previous plots) represent, respectively, the number of susceptible, infected, dead, and recovered students over time (top left), number of contacts between pairs of students over time (center left), and number of new cases over time (bottom left).

Although this simulation was much shorter than those used for the epidemiological analyses in the paper (it lasted around 2 hours and half, while the simulations in the paper run for many days), we set the parameters so that the course of the disease and transmissions occurred within the allocated timeframe for the simulation at WJHS. This flexibility in the underlying OO epidemiological models enable us to customize the simulations to very different settings in terms of number of participants, duration of the simulations, properties of the virtual pathogen (e.g.: infectivity, symptoms), and more. In this way, we can create new custom scenarios that match the learning goals of the educators wishing to use OO in their classrooms.

In conclusion, we believe that the simulation at WJHS was an interesting and engaging experience for everybody involved, and it gave us a valuable opportunity to test the latest iterations of the immersive social simulation and the app technology, elements that in our view make OO stand out as an innovative tool for both educational and research on infectious disease. If you are an educator (or scientist) and after reading this post are interested in implementing your own OO simulation, don’t hesitate to get in touch!