This 20-minute talk was presented during the Operator Splitting Workshop

1. Progressive Decoupling of Linkages in
Optimization with Elicitable Convexity
Terry Rockafellar
University of Washington, Seattle
Operator Splitting Methods in Data Analysis
SAMSI, Raleigh NC
March 21-23, 2018

2. Linkage Problem
Problem ingredients (in a Hilbert space H, here finite-dim.):
a mapping T : H →→ H, complementary subspaces L, L⊥ ⊂ H
projection mappings: PL, PL⊥ , which satisfy PL + PL⊥ = I
Problem statement:
(L) find ¯x ∈ L and ¯y ∈ L⊥ such that ¯y ∈ T(¯x)
Interpretation: the subspace L stands for “linkage relations”
Monotone case: T maximal monotone (Spingarn, 1983)
−→ solved by his “method of partial inverses”
Optimization case: T = ∂ϕ for a function ϕ : H → (−∞, ∞]
−→ maximal monotone when ϕ is lsc, convex
Challenge: a solution method (local) without monotonicity?

3. Monotonicity: Global and Local
Maximal monotonicity properties of T : H →→ H
global: x1 − x0, y1 − y0 ≥ 0 for all (xi , yi ) ∈ gph T, and
gph T can’t be enlarged with this being maintained
local: around (¯x, ¯y) ∈ gph T if these properties hold relative
to some neighborhood of (¯x, ¯y) (Pennanen, 2003)
Strong monotonicity versions: at a level σ > 0
the condition x1 − x0, y1 − y0 ≥ 0 is replaced
by the condition x1 − x0, y1 − y0 ≥ σ||x1 − x0||2
Subdifferential monotonicity: for lsc ϕ : H → (−∞, ∞], ≡ ∞,
T = ∂ϕ is max monotone globally ⇐⇒ ϕ is a convex function
Previously unexplored issue:
what about ϕ characterizes the local max monotonicity of ∂ϕ?

4. Elicitation of Monotonicity in the Linkage Problem
Recall the problem:
(L) find ¯x ∈ L and ¯y ∈ L⊥ such that ¯y ∈ T(¯x)
Observation: replacing T by T + e PL⊥ doesn’t change (¯x, ¯y)
because T(¯x) + e PL⊥ (¯x) = T(¯x) when ¯x ∈ L
Definition of elicitation in (L) at level e ≥ 0
global: T + e PL⊥ maximally monotone
local: T + e PL⊥ maximally monotone around solution (¯x, ¯y)
elicitation level e = 0 : the global/local monotone cases
Similarly defined: the elicitation of maximal strong monotonicity

5. Proposed Method for Solving the Linkage Problem
Motivator: progressive hedging algorithm Rock./Wets (1991)
for solving problems in convex stochastic programming
Here: widened beyond “stochastic” & global/direct monotonicity
Progressive decoupling algorithm — with parameters r > e ≥ 0
In iteration ν with xν ∈ L and yν ∈ L⊥, get xν from
0 ∈ Tν(xν), where Tν(x) = T(x) − yν + r[x − xν],
then update by xν+1 = PL(xν), yν+1 = yν − (r − e)PL⊥ (xν)
Derivation: from the proximal point algorithm,
• drawing on localization ideas of Pennanen (2002),
• taking partial inverse like Spingarn (2003), but he only treated
fully max monotone T and got the version for e = 0, r = 1
• −→ proximal-point-like convergence properties, maybe local

6. Progressive Decoupling in the Optimization Case
T = ∂ϕ for a lower semicontinuous function ϕ : H → (−∞, ∞]
Linkage problem restated for optimization:
(L) find ¯x ∈ L and ¯y ∈ L⊥ such that ¯y ∈ ∂ϕ(¯x)
(corresponds to minimizing ϕ over the subspace L)
Direct translation of the algorithm’s subproblems:
0 ∈ ∂ϕν(xν), where ϕν(x) = ϕ(x) − yν, x + r
2||x − xν||2
Actual implementation under elicitability:
global: xν = argmin
x∈H
ϕν(x)
local: xν = argmin
x∈U
ϕν(x) for a solution neighborhood U

7. What Eliciting Monotonicity Means in Optimization, 1
Recall global elicitation:
∃ e ≥ 0 such that ∂ϕ + ePL⊥ is maximal monotone (globally)
Facts: ||PL⊥ (x)|| = distL(x) = distance of x from L
∂ϕ + ePL⊥ = ∂ϕe for ϕe = ϕ + e
2 dist2
L
∂ϕ + ePL⊥ is max monotone ⇐⇒ ϕe is convex
Recall local elicitation: at (¯x, ¯y) ∈ (L × L⊥) ∩ gph ∂ϕ
∃ e ≥ 0 and a (convex) neighborhood U × V of (¯x, ¯y)
such that ∂ϕ + ePL⊥ is maximal monotone in U × V
∂ϕ + ePL⊥ is max monotone in U × H ⇐⇒ ϕe is convex on U
Strong elicitation: corresponds in these cases to strong convexity

8. What Eliciting Monotonicity Means in Optimization, 2
Example of elicitation in a smooth setting: ϕ ∈ C2
then (L) solved by (¯x, ¯y) means that ¯x ∈ L, ¯y = ϕ(¯x) ∈ L⊥
2ϕ(¯x) positive-definite relative to L =⇒ strong local elicitability
Broader elicitability criterion: ∃ lsc convex function ψ such that
(x, y) ∈ [U × V ] ∩ gph ∂ψ =⇒ (x, y) ∈ gph ∂ϕe , ψ(x) = ϕe(x)
= ”variational 2nd-order sufficient condition for local optimality”
Theorem: this condition is in fact equivalent to local elicitability
at least under the mild assumption of “variational regularity”
Strong monotonicity version of this optimality condition:
corresponds to a natural tilt stability of the local minimum

9. Application to Problem Decomposition
H = H1 × · · · × Hm, x = (x1, . . . , xm), y = (y1, . . . , ym)
minimize ϕ(x) = ϕ1(x1) + · · · + ϕm(xm) over a subspace L ⊂ H
Corresponding linkage problem:
find (¯x1, . . . , ¯xm) ∈ L and (¯y1, . . . , ¯ym) ∈ L⊥ with ¯yi ∈ ∂ϕi (¯xi ) ∀i
Progressive decoupling algorithm in decomposition (r > e ≥ 0)
In iteration ν with (xν
1 , . . . , xν
m) ∈ L and (yν
1 , . . . , yν
m) ∈ L⊥, get
xν = (xν
1 , . . . , xν
m) ∈ H by
xν
i = [local] argmin
xi ∈Hi
ϕi (xi ) − yν
i , xi + r
2||xi − xν
i ||2 ∀i,
then update by xν+1 = PL(xν), yν+1
i = yν
i − (r − e)PL⊥ (xν
i )
Special case: the progressive hedging algorithm in stoch. prog.
but now nonconvexity is O.K. via 2nd-order local optimality

10. Application to Splitting in Nonconvex Optimization
H = H0 × · · · × H0, x = (x1, . . . , xm), y = (y1, . . . , ym)
minimize ϕ(x) = ϕ1(x1) + · · · + ϕm(xm) over subspace L ⊂ H
taking L = x ∃ w : x1 = · · · = xm = w ,
L⊥ = y y1 + · · · + ym = 0
=⇒ minimize ϕ1(w) + · · · + ϕm(w) over w ∈ H0
Progressive decoupling algorithm in splitting (r > e ≥ 0)
Having wν ∈ H0 and yν
i ∈ H0 with yν
1 + · · · yν
m = 0, determine
xν
i = [local] argmin
xi ∈H0
ϕi (xi ) − yν
i , xi + r
2||xi − wν||2 ∀i,
then update by
wν+1 = 1
m
m
i=1 xν
i , yν+1
i = yν
i − (r − e)[ xν
i − wν+1 ]
Elicitability at level e: reflects second-order local optimality

11. References
[1] J.E. Spingarn (1983) “Partial inverse of a monotone
operator,” Applied Mathematics and Optimization 10, 247–265.
[2] T. Pennanen (2002) “Local convergence of the proximal
point algorithm and multiplier methods without monotonicity,”
Mathematics of Operations Research 27, 170–191.
[3] R.T. Rockafellar, R.J-B Wets (1991) “Scenarios and policy
aggregation in optimization under uncertainty,” Mathematics of
Operations Research 16, 119–147.
[4] R.T. Rockafellar (2017) “Progressive decoupling of linkages
in monotone variational inequalities and convex optimization,”
preprint available.
website: www.math.washington.edu/∼rtr/mypage.html