Traits of a successful Monitoring, Evaluation and Reporting program

Traits of a successful
Monitoring,
Evaluation and
Reporting program
Nick Marsh – Yorb Pty Ltd

Supporting adaptive water planning
and management
----what does that mean
1. Assessing compliance of water plan volumetric specifications
2. Assessing ecological condition
3. Ecological condition in relation to watering regimes
4. Testing hypothesised hydro-ecological associations
5. What about non-flow drivers such as land management and water
quality
6. Short term issues – Drought / flood response
• We often have a legislative requirement to do 1)-volumetric compliance.
But we stretch the budget to achieve all 6.
• So how do you get the best return on MER investment

Traits of MER Programs
1. Data storage, management and dissemination.
2. Reporting: What, Who, How.
3. Risks and Hazards: does my bad = your bad
4. Scalability (both in time and space)
5. Co-ordination with other data sets in space and time
6. Representativeness
7. Statistical power. Is the sample size adequate

1) Data storage, management
and dissemination
• Are the underlying data sets easily accessible for use by public, private
or research projects
• Excellent data sets that are hidden from the public domain severely
limit their value.
• The excuses:-
1. We didn’t budget a good data storage and retrieval system,
2. It might be used for evil if it gets into the wrong hands
3. The Bureau of Meteorology will do it
• Example solution:
• Vic Data warehouse
• EHMP data portal

1) Data Storage continued..
Data mashup …Big Data…
• The opportunities of combining datasets from novel sources is
largely untapped in NRM.
• Consider the finance sector – potential data sources to inform
their decision making includes stock and commodity data,
international long term weather forecasts, community trends
expressed through blogs and tweets…
• By making datasets easy to access allows them to be applied in
unexpected and potentially useful ways. How does the
international cotton price effect wetland health
• Data storage and delivery should focus on providing maximum
volume and ease of access. Let the researchers decide its value.

2) Reporting
• What is the required rigor of the analysis, presenting ‘condition’ scores is
a different level of data collection and analysis to presenting a ‘trend’
or testing an hypothesis.
• Reporting should define the minimum data collection, not the other
way around.
3) Risks and Hazards
• Is my bad = your bad
• For large spatial scales – differences in defining harm can be big – is low
dissolved oxygen in Tasmanian streams the same as low dissolved
oxygen in Northern Territory streams
• the European Union Water Directive development monitoring and
reporting requirements based on ‘equivalency’ panels – so reporting of
adverse conditions has the same meaning.

4) Scalability (both in time and space)
• Can the program be expanded and contracted in line with monitoring
requirements and budgetary limitations
• ….is it OK to report at a catchment level every other year rather than
reach level.
• …..what statistical power is lost if 1/3 the sites are not sampled for 1
year
• Are there well documented protocols and training programs that allow
a wide pool of monitoring expertise so the program is not expertise
limited

5) Co-ordination with other data
sets in space and time
Gauge sites and EPA reaches
• Do the monitoring programs coincide with other data collection
programs and could this add value to the MER programs
• hydrometric measurement stations
• regions of Lidar surveys or
• vegetation mapping studies
• research programs.

6) Representativeness
• What are the approaches for site selection, and does the method of site
selection restrict future use of the data
• Example – non-random site selection, how ‘dry’ sites are recorded
7) Adequate statistical power
• Adequate statistical power to detect target effect size: Is the sample
size adequate The measurement of any parameter is only useful if the
sample size is large enough to detect a change that would trigger
some regulatory response.

Compare a few programs
Trait FARWH EHMP California
Water Board
Chesapeake
Bay
(Estuarine)
Chesapeake
Bay
(Freshwater
)
EU Water
Framework
directive
Purpose (X indicates purpose of program)
Collection X X
Integration X X X X X X
Reporting X X X X X X
Traits (L = low emphasis in program, M= Medium emphasis in program, H= High emphasis in program)
Data storage,
management, and
dissemination
L H M H L L
Reporting M H M H M L
Risks and Hazards L M L M L M
Scalability H M M M M H
Co-ordination with M M L H L L
other data sets
Representativeness N/A H N/A H H N/A
Statistical power N/A H N/A N/A H N/A

Monitoring purpose
Pressure
e.g
Cease to Flow, No. Zero Flow days
Measure:
Early Warning
Response
e.g
Critical NDVI values
• Pressure
• Response
Compliance
Pressure
Response
e.g
e.g
Cease to Flow, ∆ Baseflow Index
Water quality thresholds
• Both
Pressure
e.g
∆ Baseflow Index, ∆ Water quality
Diagnostic
Response
e.g
SIGNAL Score, Trait representation
MER Trait Early Warning Compliance Diagnostic
Data storage, management and
dissemination
Critical Critical Important
Reporting: Critical Critical N/A
Risks and Hazards: Critical Important Important
Scalability Important Critical important
Co-ordination with other data sets in
space and time
important N/A Critical
Representativeness. Important Important Important
Adequate statistical power Important Critical Critical

MER for adaptive water planning
and management
AWPM
• Data storage and dissemination should
be a high priority
• Centres of distribution and agreed
measures of ‘equivalence’ but no need
to centralise the collection and reporting
• Water resource planning and
management is driven by environmental
requirements
• Diagnostic reporting is critical – get long
term biological data sets into the public
domain
Purpose
Collection
Integration
Reporting
Traits
Data storage,
management, and
dissemination
Reporting
Risks and Hazards
Scalability
Co-ordination with
other data sets
Representativeness
Statistical power
X
X
X
H
M
H
H
H
H
H