NOAA/National Weather Service's Chief Learning Office United States Department of Commerce
Commerce Learning Center

Hydrology Professional Development Series (PDS) 2 : Develop, Implement and Maintain Models and Tools

PDS Statement of Responsibility : Develop and implement hydrological and hydraulic models and tools in support of operations.

Description of the Area of Responsibility : Both RFCs and WFOs are dependent upon effective and well maintained hydrologic and hydraulic models and tools to support operational services.

Prerequisites : Basic fundamental statistics, unit hydrograph theory, general knowledge of water balance accounting, snow hydrology, routing methods, hydraulics, geospatial data manipulation, familiarity with Linux OS and Windows.


PCU 3: Evaluate the hydrological forecasts and services (Ongoing assessment of model performance, forecast validation & verification)


Producers:
Dave Streubel - Alaska-Pacific River Forecast Center, Anchorage, AK
Rick Koehler - Forecast Decision Training Branch, Boulder, CO
Laura Diamond - North Central RFC, Chanhassen, MN

Description of Job Competency to be Achieved :
Performing a periodic comparison between the forecasts and the conditions which occurred and reviewing forecast justification will enable forecasters to continuously improve the services they provide to our customers and partners.

Description of Need :
Accuracy and skill are two performance measures for any hydrologic forecast. To become a better forecaster, it is not enough to simply know that a forecast did not verify based on a specific standard. A forecaster must determine what went into a forecast and identify methods for improvement. This is done through forecaster development that allow assessment of the forecast relative to the inputs and operational environment under which the forecasts were issued and communicated.

In other words, we need to evaluate all aspects of the NWS hydrologic forecast process in order to improve services to the public and other partners.

Abilities/Performance Elements:
  • Perform a periodic/seasonal review
  • Review relevant statistics
  • Identify forecast/verification errors
  • Review forecast justification
  • Create hypothesis to explain why forecast errors occurred
  • Modify the forecasting steps that were involved in the forecast error
  • Develop potential service assessment skills
  • Identify relevant performance standards
Ability 1.
Perform a periodic/seasonal review with input from user community.
    Skill 1.1.
    Review hydrologic forecasts beyond a routine verification calculation. Examine additional factors from a periodic/seasonal review to identify possible hypothesis and actionable items to include in the forecast process.
    • IC Title: Gridded Forecast Verification and Bias Correction Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: To become a better forecaster, it is not enough to simply know that a forecast did not verify. One must determine what happened and identify methods for improvement through forecast verification. In this module, you will be given an example forecast and verifying analysis. Then you will assess what corrective actions are needed based on the mismatches that occurred.
    • Length: 1 hour
    • Vetted: Yes
    Skill 1.2.
    Interact with user community to solicit input from past season regarding product quality and usefulness.

    Other Hydro PDS cross referenced abilities and skills
    • PDS 1, PCU 1 - Provide Hydrologic Services
    • Ability 2 - Build and maintain positive relationships with our partners and users
    • Skill 2.2 - Identify opportunities to enhance and improve NWS hydrologic services
    • Ability 4 - Collaborate with NWS peers for effective forecast and warning services
    • Skill 4.1 - Coordinate hydrologic events, programs, and non-operational activities
    • Ability 5 - Collaborate with partners and users for effective planning, mitigation, forecast and warning services
    • Skill 5.1 - Coordinate hydrologic programs and activities through establishing operational thresholds and event procedures
    • PDS 1, PCU 2 - Monitor and assess the hydrologic and hydrometeorological environment to identify specific threat areas
    • Ability 4 - Monitor data and conditions throughout the operational period to continuously re-evaluate products and services
    • Skill 4.1 - Assess and maintain situational awareness of current and forecast river conditions
    • PDS 1, PCU 4 - Provide hydrologic and hydrometeorological products and services
    • Ability 5 - Prepare and provide hydrologic forecast verification information
    • Skill 5.1 -Utilize available internal and external tools to provide and overview of forecast skill
    • Skill 5.2 - Assess sources of forecast skill and error
    • Skill 5.3 - Follow-up with stakeholders following an event to explain strengths and weaknesses of forecast services provided during the event.


Ability 2.
Review relevant statistics for evaluating forecast accuracy and correct probabilistic values.
    Skill 2.1.
    Describe statistical metrics used in forecast verification and their respective shortfalls.
    • IC Title: Introduction to Verification of Hydrologic Forecasts Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: This module offers a comprehensive description of a set of common verification measures for hydrologic forecasts, both deterministic and probabilistic. In addition to providing a measure of how well a forecast matches observations, verification measures can be used to help forecasters and users learn about the strengths and weaknesses of a forecast.
    • Length: 2 hours
    • Vetted: Yes
    • IC Title: Techniques in Hydrologic Forecast Verification Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: This module demonstrates techniques for developing a hydrologic forecast
    • Length: 2 hours
    • Vetted: Yes


Ability 3.
Identify elements present in forecast errors.
Derive verification data relevant to the forecast event
    Skill 3.1.
    Compare the forecast to the verification data to identify differences present in the forecast.

    Skill 3.2.
    Choose verification statistic in context with performance question being evaluated. Correctly identify differences between forecast and verification metrics as compared to observed values.
    • IC Title: Gridded Forecast Verification and Bias Correction Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: To become a better forecaster, it is not enough to simply know that a forecast did not verify. One must determine what happened and identify methods for improvement through forecast verification. In this module, you will be given an example forecast and verifying analysis. Then you will assess what corrective actions are needed based on the differences that occurred.
    • Length: 1 hour
    • Vetted: Yes
    • Possible training gap - will be considered for further development.


Ability 4.
Review forecast justification
Analyze the parts of forecast reasoning related to the performance statistic of the differences. Review forecast procedures for problematic items.
    Skill 4.1.
    Review forecast justification and analyze the parts of forecaster reasoning related to the difference.

    Possible training gap - will be considered for further development.

    Skill 4.2.
    Critically review the forecast process and examine the logic used in the forecast process that may lead to differences.
    • IC Title: Gridded Forecast Verification and Bias Correction Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: To become a better forecaster, it is not enough to simply know that a forecast did not verify. One must determine what happened and identify methods for improvement through forecast verification. In this module, you will be given an example forecast and verifying analysis. Then you will assess what corrective actions are needed based on the differences that occurred.
    • Length: 1 hour
    • Vetted: Yes
    • Possible training gap - will be considered for further development.


Ability 5.
Determine the model contribution to the forecast/verification difference and how it influenced forecaster reasoning.
    Skill 5.1.
    Generate plausible scenarios and reasons to explain why the forecast/verification difference occurred.
    • IC Title: Gridded Forecast Verification and Bias Correction Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: To become a better forecaster, it is not enough to simply know that a forecast did not verify. One must determine what happened and identify methods for improvement through forecast verification. In this module, you will be given an example forecast and verifying analysis. Then you will assess what corrective actions are needed based on the mismatches that occurred.
    • Length: 1 hour
    • Vetted: Yes
    • Possible training gap - will be considered for further development.


Ability 6.
Modify the forecasting steps that were involved in the forecast error
Use the plausible scenarios and reasons generated earlier to modify: modeling parameters, interpretation of model output, or steps in the forecast process.
    Skill 6.1.
    Identify steps to use in similar future forecast situations and monitor how well the forecast matches observed conditions.

    Possible training gap - will be considered for further development.

    Skill 6.2.
    Evaluate the steps needed to produce a forecast and analyze the process to locate where potential problems could occur. Propose modifications to address the issues identified.
    • IC Title: Gridded Forecast Verification and Bias Correction Present
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: To become a better forecaster, it is not enough to simply know that a forecast did not verify. One must determine what happened and identify methods for improvement through forecast verification. In this module, you will be given an example forecast and verifying analysis. Then you will assess what corrective actions are needed based on the mismatches that occurred.
    • Length: 1 hour
    • Vetted: Yes
    • Possible training gap - will be considered for further development.


Ability 7.
Develop potential service assessment skills
Identify the evaluation skills from hydrologic forecast reviews for use in service assessments.
    Skill 7.1.
    Review Model Calibration

    Reference PDS 2, PCU 1, Ability 3 (Conduct Calibration)

    Skill 7.2.
    Identify elements used in Service Assessments
    • IC Title: National Service Assessment Team Training
    • Type: Formal Training (in LMS)
    • URL/location: https://doc.csod.com
    • Description: This module describes the activities and responsibilities of a National Service Assessment Team. The training covers the following topics: (1) what should you expect as a team member, (2) preliminary activities, (3) duties at the scene of the event, (4) responsibilities after the fieldwork is completed and (5) tips on writing an assessment report.
    • Length: 0.5 hour
    • Vetted: Yes
    • Possible training gap - will be considered for further development.


Ability 8.
Identify relevant performance standards for hydrologic forecasts
    Skill 8.1.
    Identify office and agency standards applicable to forecast verification.

    Coordinate with program office to develop reference materials

    Possible training gap - will be considered for further development.