The 2019 Emergency Manager Experiment Unwrapped! Insight Into This Year’s Experiment and Product Innovations

The Spring 2019 Emergency Manager (EM) Experiment hosted eight EMs across two weeks in May.  The EMs hailed from a variety of jurisdictions and services–city, county, and state governments, as well as utilities and hospital networks–and represented several different states, including New York, Colorado, Kentucky, Florida, Oklahoma, and Ohio.  

In the experiment, EMs worked archived cases with the help of  experimental forecast products under development at NSSL, CIMMS, and SPC.  The products have been generated as part of the Forecasting a Continuum of Environmental Threats (FACETs) program, which seeks to improve the communication of Probabilistic Hazard Information (PHI).  The new products all represented forecast uncertainty in different ways, offering deeper insight into forecaster thinking about storm likelihood, timing, and location.  EMs first received longer-range forecasting products that were issued days before the event, and worked their way to products issued at the warning time scale, covering a fuller “continuum” of forecast information.  The archived cases encompassed a variety of severe weather threats, e.g., severe thunderstorms, QLCS storms, and supercell tornadoes that occurred across the continental US.  

Each day, the participants began with long range SPC Convective Outlooks–Day 4, Day 3 and Day 2.  Then, depending on the issue time of each product, participants saw Day 1 outlooks, Mesoscale Convective Discussions, and Watches.  Interspersed with these products, participants received an experimental Potential for Severe Timing (PST) product, experimental Warn-on-Forecast (WoF) output, and/or experimental hazard timing graphs from SPC.  Periodically throughout the case, participants completed micro-surveys asking about trends they were noticing, details they were keying in on, and decisions/actions they were taking based on the information received.  There were also mini focus groups at each time step to discuss the same topics in more detail.  As the week progressed, participants received more of the experimental products.  On Tuesday, only the PST was given; on Wednesday, participants saw the PST and WoF; and, on Thursday, participants saw the SPC timing products and WoF.  At the end of each case, at the warning timescale, participants received warning-scale PHI.  Then at the end of the day, a wrap-up survey and focus group evaluated how participants viewed the information and forecast evolution in light of what occurred.  

What are these experimental products I just mentioned? 

Sample Potential for Severe Timing (PST) Product

The PST is a product that specifies the 4-hour window(s) for the areas where severe weather is most likely to occur (see graphic to the left).  Ideally, the PST would be issued with the 11:30 Day 1 Outlook and would be valid until the end of the convective day.  This tool is meant to help provide early and specific timing information to users to help facilitate their planning during severe weather days (e.g., should schools be closed, extra staffing brought in, shift scheduled temporarily modified). 


Sample Warn-on-Forecast (WoF) Product

The WoF output provided to EMs is a timing product that identifies areas where convection is most likely to develop over the next few hours, and the associated probabilities that it will (see graphic to the left).  Further, the output updates every hour.  The SPC Hazard Timing Graph takes the Day 1 Outlook and breaks it into four-hour windows of time, allowing participants to see when hazards are most likely to occur in their area within a 24-hour period.  Ideally, this graphic would automatically update with updates in forecast guidance. This tool would help users know, for example, when a storm is expected to reach the “moderate risk” threshold and for how long.

The last day of the experiment consisted of extensive debriefing and reflecting.  EMs completed post-week surveys and a focus group interview which asked for their deep evaluations of the tools and products they used.  We wanted to know what they liked/did not like, what worked, what was impossible to figure out or use, and their views on how PHI could be implemented in operations.


Ok, so what’s next?

Right now we are in the preliminary stages of analyses.  As a research team, we have met to discuss how to best utilize the wealth of information we gained from the new methodology used this year and rich feedback we received.  Analysis plans have been formed and are underway.  Product development is being informed by observations and early observable trends to continue moving toward operational status.  We are also planning the Fall 2019 Hazard Services PHI experiment for an integrated warning team–forecasters, broadcast meteorologists, and emergency managers working together.  The emergency managers’ portion of the Fall experiment will again feature many of these products, but within a new platform: Hazard Services.

Tags: None

Week 1 is underway!

Week 1 of this year’s Spring EM Experiment kicked off yesterday!  EMs were able to get a first look at some of the new experimental products, and they began to work PHI in the Enhanced Data Display (shown on the screens).  In the coming days we’ll add even more new products.

EM round table discussion
EMs’ first looks at new experimental products.
EMs working PHI in the Enhanced Data Display (EDD).
EMs working PHI in the Enhanced Data Display (EDD).
Tags: None

This year’s experiment is about to kick off!

Spring is buzzing around the building and we are excited to have this year’s EM experiment just around the corner! It’s PHI-nally here!

We have Emergency Managers coming from all over the country–Colorado, Florida, Kentucky, New York, Michigan, Ohio, and Oklahoma–representing a variety of jurisdictions, such as cities, counties, states, hospital networks, utility companies, and more!  We’ll be switching things up a bit this year so stay tuned for pictures and updates of the action.  Week 1 starts next week, May 13, and Week 2 starts right after that on May 20.

Tags: None

What IS the PHI-EM experiment and why is it being run?

Since 2015, an ever growing and evolving team of researchers at CIMMS/NSSL have been collaborating with academic researchers from around the country to host annual springtime Emergency Manager (EM) experiments, including the Probabilistic Hazard Information (PHI)-EM experiment.  These experiments were designed to elicit feedback directly from EMs about new technologies under development at the lab. The PHI-EM experiments have specifically focused on gathering feedback about a new technology called Probabilistic Hazard Information.


The PHI developed at NSSL is meant to help all parties in the communication chain make more informed decisions about impending weather threats.  This new PHI includes the probability of a storm to produce tornadoes, severe thunderstorm hazards (including high wind and hail), and lightning in the next hour.  It also updates rapidly, and is most commonly viewed as a plume of probabilities projected ahead of a storm (see picture below). PHI plumes are created by a forecaster using a suite of tools and algorithms.  The forecaster makes decisions about the probabilistic trend of the storm based on his/her tools (e.g., radar, ensemble model data). After creation, PHI plumes are delivered to EMs and Broadcast Meteorologists through the Enhanced Data Display (EDD).  In addition, the EDD offers other useful information about timing, severity, and the anticipated storm track. EMs and Broadcasters then use the PHI plumes to make decisions for their jurisdictions such as sounding sirens or canceling events (EMs), and whether to run a crawl or cut into on-air programming for live coverage (Broadcasters).  


EM feedback is invaluable to the evaluation process.  While PHI may constitute a huge breakthrough on the forecasting end of the chain by conveying richer and very localized forecast information, if the displays are hard to understand or not providing meaningful and needed information on the users’ end of the chain, then more work needs to be done!  Thus, EMs have been brought in as participants to help assess the viability of this new information and product. Specifically, EMs have been asked for their feedback on the types of PHI that have been created thus far as well as the EDD interface to view them. They have been asked to comment on which aspects of PHI are most/least helpful and easy/difficult to use and understand.  Overall, as researchers, we are trying to ascertain the feasibility of the PHI and EDD and move them toward operations!

Tags: None

2019 Spring EM Experiment Recruitment has started!

Hi everyone!  It’s that time of year again!  That’s right, the call just came out and applications are rolling in for the 2019 Spring EM Experiment!  More details can be found on the Recruitment Page for the project!  Take a look through the details when you get a chance.  We hope all of the Emergency Managers out there will consider applying!

Tags: None

We’re Back!

Hello everyone!  My name is Cassandra Shivers-Williams and I am currently a post-doc researcher at CIMMS/NSSL within the Societal Impacts Group (SIG).  I have been working with Emergency Managers (EMs) in the Hazardous Weather Testbed experiments since 2016!  My wonderful colleague Kodi Berry started this blog as a way to talk about all of the awesome EM research happening here!  Well…I’m picking it up and taking it over! (You may have noticed the site facelift =)  )

This blog will be highlighting lots of fun with EMs and the Probabilistic Hazard Information (PHI) Experiment, sprinkled with some other ongoing social science research within the SIG!  Our sister entity, KPHI TV, also conducts social science research, but primarily with Broadcaster Meteorologists.  That work is blasted on the KPHI TV Blog!   I encourage everyone to check out that blog for more fun with PHI!

This website has been updated with some background information about the work we do and the research team, opportunities for EMs to participate in testbed research, presentations/publications of our EM work, and some alumni!  Take a look around and check back in periodically to see the latest and greatest updates from the SIG!

Tags: None

Week 1 Underway

This week emergency managers from hospital, city, county, and state jurisdictions are evaluating how they might use probabilistic hazard information for decision making during severe convective weather. 

Tags: None

2018 EM Selection

We had an unprecedented response to our call for EM applicants for the 2018 HWT PHI EM project. Over 50 EMs applied for 8 spots! So many strong candidates this year! We are actively pursuing funding for next year so we hope those we had to turn away this year will apply next year.

Thank you to our friends in the National Weather Service for helping to spread the word about our project!

Tags: None


This is the information page for emergency manager (EM) participation in NOAA’s Hazardous Weather Testbed (HWT). EMs in the HWT help evaluate probabilistic hazard information (PHI) for severe weather, the core of the future National Weather Service severe weather warning paradigm.  PHI is part of NSSL’s overall FACETS initiative.  Learn more about FACETS here:

Tags: None