Robust Decision Making (RDM) - GFDRR

Robust Decision Making
Robert Lempert
Director
RAND Pardee Center on Longer-Range Global Policy
and the Future Human Condition
SDN Panel on Resilience
February 21, 2012

Ensuring Resilience Poses Both Analytic and
Organizational Challenges
• For instance, planning with statistics of future climate based
on projections, rather than just replicating recent history,
requires
• Usefully summarizing incomplete information from new, fastmoving,
and potentially irreducibly uncertain science
• Justifying analytic choices to diverse constituencies, many of
whom may object to implications of some particular choices
• Solution requires rethinking how we use uncertain information in
our planning
• Seek robust strategies
• Employ iterative risk management
• Embed analytic approach in a appropriate process of
stakeholder engagement
2

Traditional Planning Methods Often Fail to
Support Planning Needed for Resilience
Traditional analytic methods characterize uncertainties as a
prelude to assessing alternative decisions
Predict
Act
Many situations confront decisionmakers with deep uncertainty,
where
– They do not know, and/or key parties to the decision do not agree on, the
system model, prior probabilities, and/or “cost” function
Decisions can go awry if decisionmakers assume risks are wellcharacterized
when they are not
– Uncertainties are underestimated
– Competing analyses can contribute to gridlock
– Misplaced concreteness can blind decision-makers to surprise
3

Robust Decision Making (RDM) Helps Inform
Good Decisions Without Reliable Predictions
RDM follows “Deliberation with Analysis” decision support process
Key idea:
Participatory Scoping
1.Define Goals, Uncertainties,
and Strategies
2.Choose Candidate Strategy
• Start with strategy
• Use analytics to identify
scenarios where strategy
fail to meet its goals
• Use these scenarios to
identify and evaluate
responses
Tradeoff Analysis
5.Display and Evaluate
Tradeoffs Among
Strateg(ies)
Scenario Exploration
and Discovery
4.Characterize Strategy’s
Vulnerabilities
Case Generation
3.Estimate Performance of
Strategy in Many Futures
Deliberation
Robust Strategy
Vulnerabilities
Analysis
Deliberation with
Analysis
4

Robust Decision Making (RDM) Helps Inform
Good Decisions Without Reliable Predictions
RDM follows “Deliberation with Analysis” decision support process
Robustness and resilience are
viewed here similarly, but
• Resilience is a property of a
system, viewed externally
• Robustness is property of
choices made by actors in a
system
Tradeoff Analysis
5.Display and Evaluate
Tradeoffs Among
Strateg(ies)
Participatory Scoping
1.Define Goals, Uncertainties,
and Strategies
2.Choose Candidate Strategy
Case Generation
3.Estimate Performance of
Strategy in Many Futures
Scenario Exploration
and Discovery
4.Characterize Strategy’s
Vulnerabilities
Robust Strategy
Vulnerabilities
Deliberation
Analysis
Deliberation with
Analysis
5

Two Examples Illustrate How RDM Approach
Contributes to Integrated Flood Risk Management
1. Evaluating non-structural measures in New Orleans
– Part of extensive RAND support for recently released
Louisiana 2012 Coastal Master Plan
2. Current RDM pilot project in Ho Chi Minh City
6

How Can Non-Structural Policies Help New Orleans
Become More Resilient to Future Flooding?
• New Orleans faces significant risk
from riverine and coastal flooding
• The response to date has focused
on structural protection
• Levees, flood walls, pumps,
gates, etc.
• These existing efforts have
• Provided insufficient protection
• Promoted development in low
lying areas, thus increasing risk
But non-structural measures involve many uncertainties, such as behavioral
responses, so can prove difficult to include in traditional planning
Fischbach, J. (2010). Managing New Orleans Flood Risk in an Uncertain Future
Using Non-Structural Risk Mitigation. Santa Monica, RAND
7

What Non-Structural Policies Should the City Pursue?
Consider 16 alternative
combinations of these policies
Buyout threshold (y)
Elevation
Threshold
(x)
Elev.
only
-5 ft. -4 ft.
-5 ft. X
-2 ft. X X X
+1 ft. X X X
Consider two types of non-structural policies
applied across city’s neighborhoods:
•Incentives to elevate new and existing
homes
•Land use restrictions, including
• Buying out existing homes
• Growth restrictions
+4 ft. X X X
+7 ft. X X X
+10 ft. X X X
8

Difficult-to-Characterize Long-Term Uncertainty
Complicates Any Evaluation of These Options
Uncertainties about physical systems
– Climate change effects on storm intensity or frequency
– Coastal land loss
– Relative sea level rise (Land subsidence + sea level rise)
Uncertainties about socio-economic systems
– Local economic growth and population patterns
– Induced development effects (moral hazard)
– Participation rates in voluntary programs
– Protection system maintenance
Tradeoff
Analysis
Scoping &
Strategy
Scenario
Discovery
Case
Generation
9

Compare How Plans Perform Across 250 Futures that
Sample All Combinations of Uncertainties
Tradeoff
Analysis
Scoping &
Strategy
Scenario
Discovery
Case
Generation
Deviation from optimum for Citywide
Strategies across 250 plausible futures
Blue boxes: elevation only
Red boxes: combined elevation+buyouts
Increasing response cost
10

Use Database to Identify Two Best
Non-Structural Policies
Tradeoff
Analysis
Scoping &
Strategy
Scenario
Discovery
Case
Generation
Deviation from optimum for Citywide
Strategies across 250 plausible futures
Blue boxes: elevation only
Red boxes: combined elevation+buyouts
Best elevation+buyout
strategy performs slightly
better, but costs more, than
best elevation-only strategy
Increasing response cost
11

Use Database to Identify Two Best
Non-Structural Policies
Key question: in which futures
do combined elevation+buyouts
strategies perform better than
elevation standards alone?
Deviation from optimum for Citywide
Strategies across 250 plausible futures
Blue boxes: elevation only
Red boxes: combined elevation+buyouts
Best elevation+buyout
strategy performs slightly
better, but costs more, than
best elevation-only strategy
Increasing response cost
12

Under What Conditions Does the Elevation+Buyouts
Strategy Perform Better Than Elevation-Only Strategy?
Statistical “scenario discovery” analysis asks “what are the
key drivers of vulnerability for elevation-only strategy?”
Active Enforcement,
Increasing Risk Scenario
– Buyout policies 100% enforced
– Levees degrading over time
– Population stable or growing
Active Enforcement, Reduced
Participation Scenario
– Buyout policies 100% enforced
– Buyout costs less than or equal
to current estimates
– Participation rate for elevation
incentives < 76%
Scoping &
Strategy
Tradeoff
Analysis
Case
Generation
Scenario
Discovery
13

Under What Conditions Does the Elevation+Buyouts
Strategy Perform Better Than Elevation-Only Strategy?
Statistical “scenario discovery” analysis asks “what are the
key drivers of vulnerability for elevation-only strategy?”
Active Enforcement,
Increasing Risk Scenario
– Buyout policies 100% enforced
– Levees degrading over time
– Population stable or growing
Active Enforcement, Reduced
Participation Scenario
– Buyout policies 100% enforced
– Buyout costs less than or equal
to current estimates
– Participation rate for elevation
incentives < 76%
Tradeoff
Analysis
Scoping &
Strategy
Scenario
Discovery
Case
Generation
Policy makers should only choose
Elevation+Buyout strategy only if
they ascribe sufficient likelihood to
one of these two scenarios
14

Ho Chi Minh City Developing an Integrated
Flood Risk Management Strategy
HCMC Areas Currently Subject To Flooding
HCMC
•Already experiences extensive routine
flooding
•Ranks on “top ten” lists of places most
likely to be affected by climate change
•Engaged in a multi-billion dollar
infrastructure construction campaign
•Recognizes that non-structural policies
will also be required
Source: Asian Development Bank. Ho Chi Minh City
Adaptation To Climate Change. Mandaluyong City,
Philippines: Asian Development Bank, 2010.
Effective strategy must prove robust
over wide range of future climate and
socio-economic conditions.
15

Our Project Will Demonstrate How HCMC
Can Use RDM to Help Manage Its Flood Risk
Project will use RDM and existing simple
models to
• Identify vulnerabilities of strategy with current
HCMC flood control infrastructure and selected
adaptation and retreat options
• Suggest how to model and evaluate adaptive
decision strategies
• Develop a roadmap describing how SCFC can
incorporate RDM into its future planning efforts
16

“XLRM” Framework Structures Analysis Around Key
Uncertainties, Options, Metrics, and Models
Uncertain Factors (X)
Policy Levers (L)
Relationships (R)
Performance Metrics (M)
The model relates actions (L) to
consequences (M) contingent on
assumptions (X)
17

Factors Considered in
HCMC’s Own Few-Scenario Analysis
X: Exogenous uncertainties L: Policy levers
• Extreme precipitation (X mm/3 hrs)
• Mean sea level
• System described in 2001 JICA Master
Plan
R: Relationships M: Measures of merit
• SWMM Model
• ArcGIS for calculating RI and DI
• RI: Risk exposure (population/housing)
• DI: Damage exposure (economic)
18

Factors Potentially Considered in Our Analysis
X: Exogenous uncertainties L: Policy levers
• Extreme precipitation (X mm/3 hrs)
• Mean sea level
• Subsidence rate
• Infrastructure performance
• Delays in implementing flood control
plans
• Rate and patterns of economic and
population growth
• Effectiveness of policies
• Costs of implementing policies
• System described in 2001 JICA Master
Plan
• Adaptation options include:
• Elevating buildings
• Small scale pumps
• Public awareness
• Retreat options include:
• Restrictions/appropriate land use
• Adaptive decision strategies
• Signposts
• Responses
R: Relationships M: Measures of merit
• SWMM Model
• ArcGIS for calculating RI and DI
• RI: Risk exposure (population/housing)
• DI: Damage exposure (economic)
19

Evaluate Impact of Extreme Climate Events
on Policy Choice Similarly to Recent Work
• Project for Port of Los Angeles considered effect of abrupt sea
level rise on infrastructure investment decisions
• Similarly, in this analysis we will
– Use modeling to identify scenarios where
• Combinations of future extreme event frequencies and future
socio-economic factors
• Cause proposed flood risk management strategy to increase
population (RI) or economic (DI) risk over current levels
– Estimate probability thresholds for those scenarios beyond which
HCMC might choose to alter its flood risk management strategy
– Compare these probability thresholds to the best available
scientific evidence
20

Project Will Follow RDM Process
Oct 2011 workshop and
Nov 2011 XLRM report
Participatory Scoping
1.Define Goals, Uncertainties,
and Strategies
2.Choose Candidate Strategy
Tradeoff Analysis
5.Display and Evaluate
Tradeoffs Among
Strateg(ies)
Spring 2012 workshop
Spring 2012 workshop
Scenario Exploration
and Discovery
4.Characterize Strategy’s
Vulnerabilities
Case Generation
3.Estimate Performance of
Strategy in Many Futures
Current modeling effort
Deliberation
Analysis
Deliberation with
Analysis
Vulnerabilities
Final report will describe vulnerabilities, demonstrate how to evaluate
iterative risk management plans, and provide roadmap for future work
21

Benefits and Challenges
of this RDM Approach
Key idea: Use analysis to identify vulnerabilities of
specific plans and compare robust responses
Provides decision support tools that:
– Reduce information of varying quality from many sources
about many types of physical and socio-economic factors
into a concise set of key tradeoffs
– Allows comparison of dynamic plans that evolve over time
– Provides a means to effectively communicate this
information to diverse stakeholders
However, approach creates risk/vulnerability
assessments focused on specific plans/decisions
22

More Information
Bryant, B. P., and Lempert, R. J., "Thinking inside the box: A participatory, computerassisted
approach to scenario discovery." Technological Forecasting and Social
Change, 77(1), 34-49, 2010.
Fischbach, J., Managing New Orleans Flood Risk in an Uncertain Future Using Non-
Structural Risk Mitigation. Santa Monica, RAND. 2010.
David G. Groves, Robert J. Lempert, Debra Knopman, Sandra H. Berry: Preparing for
an Uncertain Climate Future: Identifying Robust Water Management Strategies,
RAND DB-550-NSF, 2008.
David G. Groves, Debra Knopman, Robert J. Lempert, Sandra H. Berry, and Lynne
Wainfan, Presenting Uncertainty About Climate Change to Water Resource
Managers, RAND TR-505-NSF, 2007.
Lempert, R., and Collins, M. (2007). "Managing the Risk of Uncertain Threshold
Responses: Comparison of Robust, Optimum, and Precautionary Approaches." Risk
Analysis, 27(4), 2007.
Steven W. Popper, Robert J. Lempert, and Steven C. Bankes: "Shaping the Future,"
Scientific American, vol 292, no. 4 pp. 66-71, April 2005
www.rand.org/ise/projects/improvingdecisions/
23

Thank you!
24