Improving the protection of aquatic ecosystems by dynamically constraining reservoir operation via Direct Policy Conditioning

1. IFAC
2014
CAPE
TOWN
-­‐ZA
Matteo Giuliani1, Andrea Castelletti1,2, Patrick M. Reed3
1 Dipartimento di Elettronica, Informazione, e Bioingegneria, Politecnico di Milano, Milano, Italy
2 Institute of Environmental Engineering ETH-Z, Zurich
3 Department of Civil and Environmental Engineering, Cornell University, Ithaca, NY
Modelling
and
Control
of
Water
Systems
Improving the protection of aquatic ecosystems
by dynamically constraining reservoir operation
via Direct Policy Conditioning

2. The fall of the social planner myth?
SOCIAL PLANNER’S
PARETO OPTIMAL
Stakeholder 1’s utility
Stakeholder 2’s utility
utopia
REALITY
dominated
unfeasible

3. A real world example
Anghileri, D. et al. Journal of Water Resources Planning and
Management, 139(5), 492–500, 2013
Hydropower reservoir
Penstock
Power plant
Como city
River Adda
River Adda
Legend
Lario
Lario catchment
River
Irrigated area
Kilometers
0 5 10 20 30 40 50
UNCOORDINATED CENTRALIZED
UNCOORDINATED CENTRALIZED
Lake
Como
Lake
Como
r
s 3
s 1
s 3
u 3
s 2
u 1
u 2
R2
3
(•)
R2
R1
hydropower plant
irrigated area
H2
H3
H2
H1
H3
q 2
q 1
q 3
q 2
q 1
s 1
s 2
s 3
u 2
u 3
u 1
m 1
m 2
m
(•)
(•) m (•)
u 3
(a) (b)
r
3
(•)
FIG. 3. The model scheme under uncoordinated (left) and centralized (right) 23
490’000
480’000
470’000
460’000
800 900 1000 1100 1200 1300 1400 1500 1600
Irrigation deficit [m3/s]2
Hydropower revenue [euro/day]
H
b a
C4
C3
C6
CO2 CO1 UC
C5
C4
C3
C2
C1
UC
Lake
Como
Lake
Como
r
s 1
s 2
u 1
u 2
R2
R1
R2
R1
hydropower plant
irrigated area
H2
H1
H3
H2
H1
H3
q 3
q 2
q 1
q 3
q 2
q 1
s 1
s 2
s 3
u 2
u 3
u 1
m 1
m 2
m
(•)
(•) m (•)
(a) (b)
r
FIG. 3. The model scheme under uncoordinated (left) and centralized (right) man-agement.
23
UNCOORDINATED
CENTRALIZED
(SOCIAL PLANNER)

4. Efficiency vs acceptability: how to trade-off?
Giuliani M. et al., Journal of Water Resources Planning and Management, 2014
acceptability
efficiency
utopia
SOCIAL
PLANNER
INDIVIDUALISM
acceptability
of the
social planner
efficiency of
individualism
coordination
mechanism
design

5. Direct Policy Conditioning
an approach to condition the individualistic control policy and push it
towards a social welfare equilibrium

6. Direct Policy Conditioning
an approach to condition the individualistic control policy and push it
towards a social welfare equilibrium
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

7. Direct Policy Conditioning
COMPUTE THE SOCIAL
PLANNER POLICIES
1
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

8. Direct Policy Conditioning
GET INSIGHTS ON HOW
TO CONDITION THE
PRIMARY’S POLICY 1 2
COMPUTE THE SOCIAL
PLANNER POLICIES
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

9. GET INSIGHTS ON HOW
TO CONDITION THE
PRIMARY’S POLICY 1 2
COMPUTE THE
CONSTRAINED
PRIMARY POLICY
Direct Policy Conditioning
COMPUTE THE SOCIAL
PLANNER POLICIES
3
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

10. GET INSIGHTS ON HOW
TO CONDITION THE
PRIMARY’S POLICY
COMPUTE THE
CONSTRAINED
PRIMARY POLICY
Direct Policy Conditioning
Multi-objective
optimization using the
Direct Policy Search
approach
Policy parameters vectors
Objectives values
1 2
3
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

11. 1 2
COMPUTE THE
CONSTRAINED
PRIMARY POLICY
Direct Policy Conditioning
Multi-objective
optimization using the
Direct Policy Search
approach
Input Variable
Selection of the most
relevant parameters in
explaining the
secondary objectives
Policy parameters vectors
Objectives values
Subset of policy
parameters to
be conditioned
3
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

12. 1 2
Single-objective
optimization of the
primary objective with
restricted constraints on
the sensitive policy
parameters
Direct Policy Conditioning
Multi-objective
optimization using the
Direct Policy Search
approach
Input Variable
Selection of the most
relevant parameters in
explaining the
secondary objectives
Policy parameters vectors
Objectives values
Subset of policy
parameters to
be conditioned
3
PRIMARY obj.
SECONDARY obj.
utopia
SECONDARY’s
OPTIMUM
PRIMARY’s
OPTIMUM

13. CASE STUDY

14. The Susquehanna river system
(a)
(b)
atomic
power plant
Baltimore
Chester
Fishery and
boating
Conowingo
Muddy Run
Marietta facility
station
Pennsylvania
Maryland
lateral
inflow
Susquehanna River
Muddy Run
inflow
Lower
Susquehanna
River
Maryland
New York
Pennsylvania
Conowingo pond
Chester
Baltimore
(b)
Conowingo pond
Marietta facility
station
atomic
power plant
Baltimore
Muddy Run
Chester
Fishery and
boating
FERC environmental
requirements
Conowingo
hydropower plant
Pennsylvania
Maryland
lateral
inflow
Susquehanna River
Muddy Run
inflow
River
Maryland
Chester
Baltimore

15. DPC experimental setting
1
SOCIAL PLANNER POLICIES
• POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters
• OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013]
• NFE = 1,000,000 per replica
• 30 replications to avoid dependence on randomness
1

16. Social Planner policies
Giuliani. M. et al. Water Resources Research, 2014

17. DPC experimental setting
SOCIAL PLANNER POLICIES
• POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters
• OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013]
• NFE = 1,000,000 per replica
• 30 replications to avoid dependence on randomness
INPUT VARIABLE SELECTION
• Tree based iterative input selection [Galelli and Castelletti, 2013]
1
2

18. Input Variable Selection
(a) Selected features and corresponding contribution in explaining
the Environment objective
75
50
25
0
explained variance
2 ct 3
b 3 t 4
t 2
bw4
t 1 bbt 3
w4
4
w4
(b) Decision variables selected on the Pareto-optimal set 1
0.5
variable
x1
x2
x3
u1
Gaussian Radial Basis Function
[Giuliani et al. 2014]
b = Basis radius
c = Basis centre
w = Network weights
60% explained
variance

19. Input Variable Selection
parameter value
Reference p.: the best for the environment
Lower bound p. : current situation

20. DPC experimental setting
SOCIAL PLANNER POLICIES
• POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4
output (release outputs) + 4 basis functions: 32 parameters
• OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013]
• NFE = 1,000,000 per replica
• 30 replications to avoid dependence on randomness
INPUT VARIABLE SELECTION
• Tree based iterative input selection [Galelli and Castelletti, 2013]
CONSTRAINED PRIMARY POLICY
• Baseline policy with constraints on 8 policy parameters
• Default Borg MOEA parameterization [Hadka and Reed 2013]
• NFE = 100,000 per replication
• 30 replications to avoid dependence on randomness
• Historical horizon 1999 (drought)
1
2
3

21. DPC policies’ performance

22. DPC policies’ performance
+18.6 x 106
US$/year
+ 36%
+46%
- 30% but ..

23. Conclusions
§ Direct Policy Conditioning as a coordination mechanism design
§ Preliminary results seem to be interesting in terms of improved
perfomance of current operation in the Susquehanna rb
§ Weakness in the physical interpretation of the parameters: how to
communicate the conditioning to the dam operator?
§ Sensitivity to the conditioning setting

24. THANKS

25. Programmed event supported by the TC
EGU General Assembly, Vienna 12 April—17 April 2015
EGU General Assembly
The EGU General Assembly 2015 will bring together
geoscientists from all over the world into one meet-ing
covering all disciplines of the Earth, Planetary and
Space Sciences. Especially for young scientists the
EGU aims to provide a forum to present their work
and discuss their ideas with experts in all fields of
geosciences.
In the divisions Energy, Resources and the Environ-ment
(ERE) and Hydrological Sciences (HS) the fol-lowing
sessions are proposed:
• Design and Operation of Combined Hydro/Wind/Solar
Power Generation Systems: Computer Based Control
and Optimization;
• Design and Operation of Water Resource Systems:
Computer Based Control and Optimization.
Motivation
Many environmental systems have been modified and
are still being modified by human intervention. This
intervention usually takes the form of the construction
of additions to the system intended to change system
behaviour to better serve the needs of society.
This implies that these systems and their behaviour
are being designed. They are no longer governed by
natural processes alone. Therefore models of both the
natural and the artificial part of the system will be
needed. As the demands placed on water systems by
society increase and are increasingly in conflict with
each other, it will become harder to define goals for
the modification of these systems and their behaviour.
It will also become harder to design systems and oper-ating
rules to satisfy these goals.
The aim of these sessions is to bring together experts
in the fields of water management, hydro-, solar-, and
wind-power, control theory and operations research to
discuss novel methods or novel ways of using tradi-tional
methods to define and implement desired beha-viour
for environmental systems.
Design and Operation of Water Resource
Systems: Computer Based Control and
Optimization
For control theory water systems pose some unique
challenges because of the presence of large delays and
very limited means of control. In fact for some sys-tems
the limits on the size of the change that can be
effected in a given time period necessitate the use of
forecasts to anticipate on system behaviour. For oper-ations
research the special challenge is the presence of
incommensurable and conflicting optimization targets,
the complex network of relations between stakeholders
and the lack of one clear shared motivation amongst
stakeholders. Moreover, a new awareness of more vari-ability
in the climate on longer time scales and rapid
social changes both pose new challenges for the decision
making process. This implies a need for more frequent
reconsideration of decisions and a shorter time scale for
the decision process. This process will therefore need
faster models, for instance simplified dynamic models
of hydrological systems, statistical process emulators,
surrogate models (e.g. linear or nonlinear regression)
based on data to feed faster optimization algorithms.
Currently the following people and institutions are in-volved
in the preparation of this session:
• Niels Schütze, Dresden University of Technology,
Germany;
• Andrea Castelletti, Politecnico di Milano, Italy;
• Francesca Pianosi, University of Bristol, United
Kingdom;
• Renata Romanowicz, Institute of Geophysics,
Polish Academy of Sciences, Warszawa;
• Ronald van Nooijen and Alla Kolechkina, Delft
University of Technology, Netherlands.
Design and Operation of Combined Hy-dro/
Wind/Solar Power Generation Sys-tems:
Computer Based Control and Op-timization
In most locations the yield of wind power or solar power
is uncertain. Hydropower seems an attractive means of
providing backup power and storage of energy for fu-ture
use. Combined schemes seem attractive, but will
need automatic control to optimize their yield. Un-certainty
about yield and future supply and demand
is a key issue for the management of these combined
schemes. They may also need special facilities for in-tegration
in the current energy distribution infrastruc-ture.
Currently the following people and institutions are in-volved
in the preparation of this session:
• Demetris Koutsoyiannis and Andreas Efstra-tiadis,
National Technical University of Athens,
Greece;
• Andrea Castelletti, Politecnico di Milano, Italy;
• Burlando Paolo, ETH Zürich, Zwitzerland;
• Patrick Michael Reed, Cornell University, USA;
• Alla Kolechkina and Ronald van Nooijen, Delft
University of Technology, Netherlands.
Key dates
• Call for papers for EGU 2015: 15 October 2014
• Deadline for receipt of abstracts: 7 January 2015
• Letter of acceptance to key authors: 23 January
2015
• Conference: 12 April to 17 April 2015 in Vienna,
Austria

26. Programmed event supported by the TC
26th IUGG General Assembly 2015, Earth and Environmental Sciences for Future Generations
Prague, Czech Republic June 22 - July 2, 2015
IAHS Workshop Hw07
Announcement
At the 26th IUGG General Assembly in
Prague in 2015 there will be an IAHS work-shop
on Control of Water Resource Systems
Hw07. The workshop is being organized under
the auspices of the International Commission
on Water Resources Systems (ICWRS).
Motivation
Today it is rare to find a water resource sys-tem
where the interaction with society can be
ignored. Most systems consist of both nat-ural
and manmade components and are gov-erned
by both natural processes and processes
within society. The interaction between soci-ety
and the natural system is complex. An
important part of this interaction consists of
our attempts as humans to alter the system
behaviour through the construction and ma-nipulation
of structures such as wells, dams,
pumps, weirs, gates, sluices and locks. In
a changing world it can no longer be taken
for granted that the operational rules for the
manipulation of the manmade components of
the water resource system will be appropriate
over the whole life time of the infrastructure.
This workshop is intended for presentations on
the formulation and adaptation of operational
rules for the automated manipulation of man-made
components of water resource systems
with changing boundary conditions, or, less
formally, for presentations on computer con-trol
of water resource systems in a world in
flux.
Convener team
Currently the following people and institu-tions
are involved in the preparation of this
session.
• Alla Kolechkina, Delft University of
Technology, Netherlands
• Ronald van Nooijen, Delft University of
Technology, Netherlands
• Andrea Castelletti, Politecnico di Mil-ano,
Italy;
26th General Assembly of the Inter-national
Union of Geodesy and Geo-physics
(IUGG)
A better understanding of the way in which
our planet functions and of the effects of our
actions on its behaviour is needed to provide
for the needs of future generations.
This Scientific Assembly to be held in Prague
from 22 June to 2 July 2015 will provide an
opportunity for scientists from all geophysical
disciplines and from all countries to meet and
exchange knowledge and ideas. The Assembly
also will also give the participants the oppor-tunity
to inform the general public and policy
makers.
Key dates for this workshop
• Abstract submission open: September
2014
• Deadline for receipt of workshop ab-stracts:
31 January 2015
• Early bird registration deadline : 10
April 2015
• Standard fee registration deadline : 15
June 2015
• Conference: 22 June to 2 July in Prague,
Czech Republic

27. TC 8.3 meeting – Wed 27 12:00 Dassen Room (Westin)