Integrated Evaluation System Prototype for Testing Research and Operational Advancements

Forecasts of future streamflow provide important information to guide flood warnings, flood response, water supply operations, hydropower generation and other critical water management activities.  To ensure the forecasts are as accurate as possible, the methods used to generate the forecasts should be constantly reviewed and improved. Past-issued forecasts should also be regularly evaluated to help identify where there might be issues in the operational forecasting process and what types of improvements are most needed. The process to generate streamflow forecasts involves many steps and components.  The major components include 1) weather forecast data (e.g., rainfall, temperature, and other variables) to use as inputs, 2) mathematical models to simulate the amount and timing of water reaching the stream and extent the water reaches in the floodplain, 3) procedures to incorporate real-time monitoring data to improve accuracy of the models, 4) and methods to visualize and communicate the streamflow forecasts to others. Each of these components effects the quality and utility of the streamflow forecast information available to the water managers and the public. The NOAA Office of Water Prediction (OWP) and the Cooperative Institute for Research to Operations in Hydrology (CIROH) are undergoing multiple efforts to evaluate streamflow forecasts and improve the methods of one or more components of the forecast process. This project is building a set of tools to make evaluating streamflow forecasts easier, more interactive and more consistent across a large number of different projects and purposes.

In the past, streamflow forecasts were generated at several thousand locations across the US.  Since the National Water Model (NWM) became operational in 2016, streamflow forecasts are now being generated at several million locations.  The magnitude of data produced by NWM forecasts makes it difficult to efficiently evaluate the quality of forecasts, identify sources of error, and assess the extent to which new developments will create an overall improvement in the forecasts nationwide.  In addition, with many parallel research efforts underway across CIROH and OWP, it is important that the results are evaluated and compared in standardized ways so that valid conclusions can be drawn about which new methods or models will lead to the greatest improvements and should be considered for use in the operational forecast system.  These issues are the motivation for this project. New methods and tools are needed to efficiently execute, synthesize and visualize results across a large number of points, to assess forecast quality in new ways, and to support standardized evaluation across research efforts.

The overall goals of this project are 1) to design and build a flexible and scalable streamflow forecast evaluation system to support research and testing of potential forecast system advancements and 2) establish evaluation standards for common research focus areas and incorporate the standards into the evaluation system. The tools will make exploration and evaluation of large datasets easier and faster, and the standards will enable consistent comparisons of results across projects and external research. The evaluation system resulting from this project will enable CIROH to execute research that leads to operational improvements in the quality and use of hydrologic forecasts for public benefit and decision-making.