Ensemble Forecasting and Post-Processing

Unfortunately no atmospheric model is perfect. This is why atmospheric scientists use an ensemble of models to create a representative sample of all possible future states. This is advantageous since the variability or spread in model solutions can be used to determine all possible outcomes, rather than just one single deterministic forecast.

To illustrate the benefit of ensemble modeling, lets take a look at Hurricane Sandy. The figure to the left shows an aggregate of many ensemble model forecasts for Hurricane Sandy 60 hours before landfall. At this time, Sandy was off the coast of Florida, and each line denotes a model forecast track for the center of Sandy. As Sandy was forecasted to re-curve westward, the model variability increased greatly. This ensemble suggests that landfall could occur anywhere between Maryland and Maine, with the most likely landfall occurring somewhere in New Jersey. For more information on Hurricane Sandy’s forecasts, click here.

 

An important caveat of using ensembles is that they are frequently biased. Here we will present model biases for near surface temperature. These biases vary by the type of model (figure a), the time of day (figure b), spatially (figure c), and by weather pattern.

The first three types of biases can be removed with some ease for temperature as shown in figures d, e and f below. The bias by model (figure d) and diurnally (figure e) are practically zero, although spatial bias is a little more tricky (figure f).

Biases that depend on weather pattern are the most difficult to remove, since the interpretation of “weather pattern” can be subjective. For instance, we have found that model biases for temperature on fire weather days are considerably cooler than on a typical day. Figure g) shows near-surface temperature bias for the warm season average and on fire weather days. Model biases on fire weather days are considerably cooler (by at least 1 degree). As a result, using the most recent set of days to perform bias correction (i.e. sequential bias correction) does not remove all the bias. The key is to bias correct with similar days, or in this case other fire weather days (conditional bias correction).

In addition to bias in the ensemble mean, ensembles can also have bias in their spread. A common problem is that the range of possible ensemble solutions is too small compared to what frequently happens, and this is known as ensemble underdispersion.

For instance, we would say that the ensemble was underdispersed in the top figure of Sandy if she had struck Florida rather than between Maryland and Maine. Fortunately, the ensemble was not underdispersed in the case of Sandy, but on the average this is a common problem.

To correct for underdispersion, we have been testing Bayesian Model Averaging (BMA) with our ensembles. In short, BMA estimates the probability density function (PDF) for each ensemble member and combines each PDF to determine a posterior estimate for the entire ensemble. This posterior PDF on the average is not underdispersed, allowing for accurate probabilistic predictions to be made.

Detail of COMAP Ensemble Projects

Better Use of Ensembles in the Forecast Process: Scenario Based Tools for Predictability Studies and Hazardous Weather Communication [2017 – Present]https://upload.wikimedia.org/wikipedia/commons/thumb/7/79/NOAA_logo.svg/1024px-NOAA_logo.svg.png
  • Classify ensembles into scenarios, using various statistical methods.
  • Use these tools to understand large scale patterns related to less predictable weather events.
  • Use ensemble scenarios to communicate uncertainty to end users via workshops and presentations.
  • Operational website available here

Project Supported by National Oceanic and Atmospheric Administration Collaborative Science Technology, and Applied Research Program.

Evaluation of Long Range Ensemble Weather Modeling using Ensembles-  [February 2014 – present]
  • Using cyclone track information from different ensembles, assess the predictability of high impact weather events.
  • Use other potentially beneficial forecast variables (wind speed, rainfall, snow) in addition to cyclone track.
  • Determine if post-processing can be used to improve the predictability of long range high impact weather events.
  • Operational website available here.

Project Supported by Con Edison Emergency Management

Employing ensemble data assimilation, parameter estimation, and field data to improve fire weather predictions in mesoscale models. [August 2010 – present]
  • Quantify and correct model biases specific to fire weather days over the Northeast United States.
  • Use cluster analysis as an objective method for separating “weather patterns.”
  • Use data assimilation to elucidate and correct model errors.
  • Operational fire weather website available here.

Project Supported by the United States Forest Service (USFS).

Improving Air Quality Forecasting and Management in New York State through Ensemble and High-Resolution Modeling with Diagnostic Analyses. [June 2008 – July 2011]

  • Quantify and post-process ensemble model specific biases on high pollutant days (i.e. high ozone and high particulate matter.
  • Couple WRF with the Urban Canopy Model (UCM) on days with high surface ozone concentrations and assess potential benefits.

Project Supported by the New York State Energy Research and Development Authority (NYSERDA).

 

Use of Mesoscale Ensemble Weather Predictions to Improve Short-Term Precipitation and Hydrological Forecasts. [Aug 2007 – March 2010]

  • Verify the NCEP SREF and the Stony Brook Ensemble (MM5/WRF) precipitation forecast data over multiple seasons.
  • Use post-processing methods such as bias correction and Bayesian Model averaging to improve bias and dispersion issues with ensembles.
  • Provide real-time post-processed precipitation forecast data for collaborators at the River Forecast Center (RFC) and National Weather Service (NWS) for input in hydrologic models.

Presentations

Erickson, M.J, B.A. Colle and J.J. Charney. “Towards the Usage of Post-processed Operational Ensemble Fire Weather Indices over the Northeast United States” National Weather Service Northeast Regional Operational Workshop. December 2013. Albany, NY. (Virtual presentation).

Erickson, M.J, B.A. Colle and J.J. Charney. “Ensemble Verification and Post-processing of Fire Weather Indices Using the NCEP SREF over the Northeast United States.” 10th Symposium on Fire and Forest Meteorology. October 2013. Bowling Green, KY. (Oral Presentation).

Erickson, M.J, B.A. Colle and J. Pollina.  “A Look at High Fire Threat Risk and Ensemble Modeling Over the Northeast United States.” 4thTri-State Weather Conference. October 2012. Danbury, CT. (Oral presentation).

Colle, B.A, M.J. Erickson and J.J Charney. “Determining Atmospheric Flow Regimes for Particular Ensemble Biases on High Fire Threat Days.”  21st AMS Conference on Weather and Forecasting. May 2012. Montreal, CA (Poster Presentation)

Erickson, M.J, B.A. Colle and J. Pollina. “Northeast U.S. Wildfires: Climatology and Ensemble Post-Processing Using Conditional Training.” Invited Seminar Talk. March 2012. East Lansing, MI (Oral presentation).

Erickson, M.J, B.A. Colle and J.J. Charney. “Similar Day Ensemble Post-Processing as Applied to Wildfire Threat and Ozone Days.” National Weather Service Northeast Regional Operational Workshop. November 2011. Albany, NY (Oral presentation).

Erickson, M.J, B.A. Colle, J. Pollina and J.J. Charney. “Impact of Spatial Bias Correction and Conditional Training on Bayesian Model Averaging Over the Northeast United States.” AMS 20th Conference on Numerical Weather Prediction. January 2011. Seattle, WA (Oral presentation).

Erickson, M.J, and B.A. Colle. “Ensemble Post-Processing and it’s Potential Benefits for the Operational Forecaster.” National Weather Service Northeast Regional Operational Workshop. November 2010. Albany, NY (Oral presentation).

Erickson, M.J, and B.A. Colle. “Post-Processing Approaches that Make Ensembles More Usable to the Forecaster.” 3rd Tri-State Weather Conference. October 2010. Danbury, CT. (Oral presentation).

Erickson, M.J, B.A. Colle, C. Hogrefe, P. Doraiswamy, K. Demerjian, W. Hao, M. Beauharnois, J.Y. Ku, and G. Sistla. “Towards an Ensemble Forecast Air Quality System for New York State.” National Weather Service Northeast Regional Operational Workshop. November 2009. Albany, NY (Oral presentation).

Erickson, M.J, B.A. Colle, J. Tongue, N. Furbush, A. Cope and J. Ostrowski. “Using a Calibrated Mesoscale Ensemble to Improve Precipitation and Hydrological Forecasts over the Northeast U.S.” AMS 19th Conference on Numerical Weather Prediction. June 2009. Omaha, NE (Oral presentation).

Erickson, M.J, and B.A. Colle. “Potential Improvements to Precipitation and Hydrological Forecasts using Mesoscale Ensemble Weather Predictions.” 2nd Tri-State Weather Conference. April 2009. Danbury, CT. (Oral presentation).

Erickson, M.J, B.A. Colle, J. Tongue, N. Furbush, A. Cope and J. Ostrowski. “Use of Mesoscale Ensemble Weather Predictions to Improve Short-Term Precipitation and Hydrological Forecasts.” National Weather Service Northeast Regional Operational Workshop. November 2008. Albany, NY (Oral presentation).

Publications

Erickson, M.J., J. Charney, and B. Colle, 2015: Verificaton and Post-processing of the NCEP-SREF during Fire Weather Days. In Prep.

Erickson, M.J., J. Charney, and B. A. Colle, 2015: Development of a Fire Weather Index Using Meteorological Observations within the Northeast United States.  J. Appl. Meteor. Climatol. In press.

Hodrefe, C, P. Doraiswamy, B. Colle, K. Demerjian, W. Hao, M.J. Erickson, M. Souders, and J-Y Ku, 2014: Meteorology, Emissions, and Grid Resolution: Effects on Discrete and Probabilistic Model Performance. In: D. Steyn, P. Builtjes, R. Timmermans (eds.): Air Pollution and Modeling and Its Application XXII, NARA Science for Peace and Security Series C: Environmental Security, 493-497. Springer Netherlands.

Erickson, M.J., B. A. Colle, and J. Charney, 2012: Impact of bias correction type and conditional training on Bayesian model averaging over the northeast United States. Wea. Forecasting, 27, 1449-1469.

Students Involved

Michael Erickson (2007 – present) – contact michael.erickson@stonybrook.edu
Nathan Korfe (2013 – present)
Matt Souders (2008 – 2010)