SlideShare una empresa de Scribd logo
1 de 49
Descargar para leer sin conexión
EE 553
Introduction to Optimization
J. McCalley
1
Real-time
Electricity markets and tools
Day-ahead
SCUC and SCED SCED
Minimize f(x)
subject to
h(x)=c
g(x)< b
BOTH LOOK LIKE THIS
SCUC:
x contains
discrete &
continuous
variables.
SCED:
x contains only
continuous
variables.
2
Optimization Terminology
Minimize f(x)
subject to
h(x)=c
g(x)> b
f(x): Objective function
x: Decision variables
h(x)=c: Equality constraint
g(x)> b: Inequality constraint
An optimization problem or a mathematical program
or a mathematical programming problem.
x*: solution 3
Classification of Optimization Problems
http://www.neos-guide.org/NEOS/index.php/Optimization_Tree
Continuous Optimization
Unconstrained Optimization
Bound Constrained Optimization
Derivative-Free Optimization
Global Optimization
Linear Programming
Network Flow Problems
Nondifferentiable Optimization
Nonlinear Programming
Optimization of Dynamic Systems
Quadratic Constrained Quadratic Programming
Quadratic Programming
Second Order Cone Programming
Semidefinite Programming
Semiinfinite Programming
Discrete and Integer Optimization
Combinatorial Optimization
Traveling Salesman Problem
Integer Programming
Mixed Integer Linear Programming
Mixed Integer Nonlinear Programming
Optimization Under Uncertainty
Robust Optimization
Stochastic Programming
Simulation/Noisy Optimization
Stochastic Algorithms
Complementarity Constraints and
Variational Inequalities
Complementarity Constraints
Game Theory
Linear Complementarity Problems
Mathematical Programs with
Complementarity Constraints
Nonlinear Complementarity
Problems
Systems of Equations
Data Fitting/Robust Estimation
Nonlinear Equations
Nonlinear Least Squares
Systems of Inequalities
Multiobjective Optimization
4
Convex functions
Definition #1: A function f(x) is convex in an interval if its
second derivative is positive on that interval.
Example: f(x)=x2 is convex since f’(x)=2x, f’’(x)=2>0
5
Convex functions
The second derivative test is sufficient but not necessary.
www.ebyte.it/library/docs/math09/AConvexInequality.html
Definition #2: A function f(x) is convex if a line drawn
between any two points on the function remains on or
above the function in the interval between the two points.
6
Convex functions
Definition #2: A function f(x) is convex if a line drawn
between any two points on the function remains on or
above the function in the interval between the two points.
Is a linear function convex?
Answer is “yes” since a line drawn between any two
points on the function remains on the function. 7
Convex Sets
Definition #3: A set C is convex if a line segment between
any two points in C lies in C.
Ex: Which of the below are convex sets?
The set on the left is convex. The set on the right is not.
8
Convex Sets
Definition #3: A set C is convex if a line segment between
any two points in C lies in C.
S. Boyd and L. Vandenberghe, “Convex optimization,” Cambridge University Press, 2004.
9
Global vs. local optima
Example: Solve the following:
Minimize f(x)=x2
Solution: f’(x)=2x=0x*=0.
This solution is a local optimum.
It is also the global optimum.
Example: Solve the following:
Minimize f(x)=x3-17x2+80x-100
Solution: f’(x)=3x2-34x+80=0
Solving the above results in x=3.33 and x=8.
Issue#1: Which is the best solution?
Issue#2: Is the best solution the global solution?
10
Global vs. local optima
Example: Solve the following:
Minimize f(x)=x3-17x2+80x-100
Solution: f’(x)=3x2-34x+80=0. Solving results in x=3.33, x=8.
Issue#1: Which is the
best solution?
Issue#2: Is the best
solution the global
solution?
x=8
No! It is unbounded.
11
Convexity & global vs. local optima
When minimizing a function, if we want to be sure that we
can get a global solution via differentiation, we need to impose
some requirements on our objective function.
We will also need to impose some requirements on the
feasible set S (set of possible values the solution x* may take).
Min f(x)
subject to
h(x)=c
g(x)> b S
x
x
f

subject to
)
(
min
Definition: If f(x) is a convex function, and if S is a convex set, then
the above problem is a convex programming problem.
Definition: If f(x) is not a convex function, or if S is not a convex set,
then the above problem is a non-convex programming problem. 12
Feasible set
Convex vs. nonconvex programming problems
The desirable quality of a convex
programming problem is that any locally
optimal solution is also a globally optimal
solution. If we have a method of
finding a locally optimal solution, that
method also finds for us the globally
optimum solution.
13
The undesirable quality of a non-convex
programming problem is that any method
which finds a locally optimal solution
does not necessarily find the globally
optimum solution.
MATHEMATICAL
PROGRAMMING
Convex
Non-convex
We address convex
programming
problems in
addressing linear
programming.
We will also, later,
address a special form of
non-convex programming
problems called integer
programs.
A convex programming problem
14
c
x
x
h
s.t.
x
x
f

)
,
(
)
,
(
min
2
1
2
1
c
)
x
h(
x
f

s.t.
)
(
min
c
)
x
(
h
x
f

s.t.
)
(
min
Two variables with one
equality-constraint
Multi-variable with one
equality-constraint.
Multi-variable with multiple
equality-constraints.
We focus on this one, but
conclusions we derive will also
apply to the other two. The
benefit of focusing on this one is
that we can visualize it.
Contour maps
15
Definition: A contour map is a 2-dimensional plane, i.e., a
coordinate system in 2 variables, say, x1, x2, that illustrates
curves (contours) of constant functional value f(x1, x2).
Example: Draw the contour map for
2
2
2
1
2
1 )
,
( x
x
x
x
f 

.
[X,Y] = meshgrid(-
2.0:.2:2.0,-2.0:.2:2.0);
Z = X.^2+Y.^2;
[c,h]=contour(X,Y,Z);
clabel(c,h);
grid;
xlabel('x1');
ylabel('x2');
Contour maps and 3-D illustrations
16
Example: Draw the 3-D surface for
2
2
2
1
2
1 )
,
( x
x
x
x
f 

.
[X,Y] = meshgrid(-
2.0:.2:2.0,-2.0:.2:2.0);
Z = X.^2+Y.^2;
surfc(X,Y,Z)
xlabel('x1')
ylabel('x2')
zlabel('f(x1,x2)')
Height is f(x)
Contours
Each contour of fixed value f is
the projection onto the x1-x2
plane of a horizontal slice
made of the 3-D figure at a
value f above the x1-x2 plane.
Solving a convex program: graphical analysis
17
Example: Solve this convex program:
.
6
)
,
(
min
2
1
2
1
2
2
2
1
2
1





x
x
)
,x
h(x
s.t.
x
x
x
x
f
6
6 1
2
2 




 x
x
x
x1
Superimpose this
relation on top of the
contour plot for f(x1,x2).
1. f(x1,x2) must be minimized, and so we would like the solution
to be as close to the origin as possible;
2. The solution must be on the thick line in the right-hand corner
of the plot, since this line represents the equality constraint.
A straight line is a convex set because a line
segment between any two points on it remain on it.
Solving a convex program: graphical analysis
18
.
)
25
.
1
,
25
.
1
(
)*
,
(
* 2
1 
 x
x
x
3
)*
,
( 2
1 
x
x
f
Solution:
Any contour f<3 does not
intersect the equality
constraint;
Any contour f>3 intersects the
equality constraint at two
points.
The contour f=3 and the
equality constraint just touch
each other at the point x*.
“Just touch”:
The two curves are tangent to one another at the solution point.
Solving a convex program: graphical analysis
19
.
The two curves are tangent to one another at the solution point.
 The normal (gradient)
vectors of the two curves, at
the solution (tangent) point,
are parallel.
“Parallel” means that the two vectors have the same direction. We do not know that
they have the same magnitude. To account for this, we equate with a “multiplier” λ:
 
c
)
,x
h(x
x
x
f 


 *
2
1
*
2
1 )
,
( 
 
 
*
*
2
2
1
*
2
1
*
2
2
2
1
2
1
1
1
6
}
6
*
)
,
(
{
2
2
)*}
,
(
{
























x
x
x
x
h
x
x
x
x
x
x
f
1
This means the following
two vectors are parallel:
Solving a convex program: graphical analysis
20
.
 
c
)
,x
h(x
x
x
f 


 *
2
1
*
2
1 )
,
( 
  0
)
,
( *
2
1
*
2
1 



 c
)
,x
h(x
x
x
f 
Moving everything to the left:
  0
)
,
( *
2
1
*
2
1 



 )
,x
h(x
c
x
x
f 
Alternately:
 
 
 
 



























0
0
)
,
(
)
,
(
*
2
1
2
1
2
2
1
2
1
1
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


Performing the gradient operation (taking
derivatives with respect to x1 and x2) :
In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?
Solving a convex program: analytical analysis
21
 
 
 
 



























0
0
)
,
(
)
,
(
*
2
1
2
1
2
2
1
2
1
1
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


In this problem, we already know the solution, but what if we did not?
Then could we use the above equations to find the solution?
NO! Because we only have 2 equations, yet 3 unknowns: x1, x2, λ.
So we need another equation. Where do we get that equation?
Recall our equality constraint: h(x1, x2)-c=0 . This must be satisfied!
Therefore:
 
 
 
 




































0
0
0
)
,
(
)
,
(
)
,
(
*
2
1
2
1
2
1
2
2
1
2
1
1
c
x
x
h
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


Three equations,
three unknowns,
we can solve.
Solving a convex program: analytical analysis
22
Observation: The three equations are simply partial derivatives of the
function
This is obviously true for the first two equations , but it is not so
obviously true for the last one. But to see it, observe
 
 
 




































0
0
0
)
,
(
)
,
(
)
,
(
*
2
1
2
1
2
1
2
2
1
2
1
1
c
x
x
h
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


 
c
)
,x
h(x
x
x
f 
 2
1
2
1 )
,
( 
 
 
c
)
,x
h(x
c
)
,x
h(x
c
)
,x
h(x
x
x
f











2
1
2
1
2
1
2
1
0
0
)
,
( 

Formal approach to solving our problem
23
Define the Lagrangian function:
In a convex programming problem, the “first-order conditions” for
finding the solution is given by
 
c
)
,x
h(x
x
x
f
x
x 

 2
1
2
1
2
1 )
,
(
)
,
,
( 

L
0
)
,
,
( 2
1 
 
x
x
L
0
)
,
,
(
0
)
,
,
(
0
)
,
,
(
2
1
2
1
2
2
1
1













x
x
x
x
x
x
x
x
L
L
L
OR
Or more
compactly
0
)
,
(
0
)
,
(









x
x
x
L
L
where we have
used x=(x1, x2)
Applying to our example
24
Define the Lagrangian function:
 
 
6
)
,
(
)
,
,
(
2
1
2
2
2
1
2
1
2
1
2
1








x
x
x
x
c
)
,x
h(x
x
x
f
x
x



L
0
)
,
,
( 2
1 
 
x
x
L
0
)
,
,
(
0
)
,
,
(
0
)
,
,
(
2
1
2
1
2
2
1
1













x
x
x
x
x
x
x
x
L
L
L
OR
 
x
x
x
x
x
x
x
x
x
x
x
x
0
6
)
,
,
(
0
2
)
,
,
(
0
2
)
,
,
(
2
1
2
1
2
2
1
2
1
2
1
1























L
L
L
A set of 3 linear equations and 3 unknowns;
we can write in the form of Ax=b.
Applying to our example
25















































































4495
.
2
2247
.
1
2247
.
1
6
0
0
0
1
1
1
2
0
1
0
2
6
0
0
0
1
1
1
2
0
1
0
2
1
2
1
2
1


x
x
x
x
Now, let’s go back to our example with a
nonlinear equality constraint.
Example with nonlinear equality
27
Non-convex because a line connecting two points in the
set do not remain in the set. (see “notes” of this slide)
.
3
2
)
,
(
min
2
1
2
1
2
2
2
1
2
1




x
x
)
,x
h(x
s.t.
x
x
x
x
f
1
2
2
1
2
3
3
2
x
x
x
x 


Superimpose this
relation on top of the
contour plot for f(x1,x2).
1. f(x1,x2) must be minimized, and so we would like the solution
to be as close to the origin as possible;
2. The solution must be on the thick line in the right-hand corner
of the plot, since this line represents the equality constraint.
Example with nonlinear equality
28
.
)
25
.
1
,
25
.
1
(
)*
,
(
* 2
1 
 x
x
x
3
)*
,
( 2
1 
x
x
f
Solution:
Any contour f<3 does not
intersect the equality
constraint;
Any contour f>3 intersects the
equality constraint at two
points.
The contour f=3 and the
equality constraint just touch
each other at the point x*.
“Just touch”:
The two curves are tangent to one another at the solution point.
Example with nonlinear equality
29
.
The two curves are tangent to one another at the solution point.
 The normal (gradient)
vectors of the two curves, at
the solution (tangent) point,
are parallel.
“Parallel” means that the two vectors have the same direction. We do not know that
they have the same magnitude. To account for this, we equate with a “multiplier” λ:
 
c
)
,x
h(x
x
x
f 


 *
2
1
*
2
1 )
,
( 
 
 
*
1
2
*
2
2
1
*
2
1
*
2
2
2
1
2
1
2
2
3
2
}
3
*
)
,
(
{
2
2
)*}
,
(
{























x
x
x
x
x
x
h
x
x
x
x
x
x
f
1
This means the following
two vectors are parallel:
Example with nonlinear equality
30
 
 
 
 



























0
0
)
,
(
)
,
(
*
2
1
2
1
2
2
1
2
1
1
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


This gives us the following two equations.
And we add the equality constraint to give 3 equations, 3 unknowns:
 
 
 
 




































0
0
0
)
,
(
)
,
(
)
,
(
*
2
1
2
1
2
1
2
2
1
2
1
1
c
x
x
h
c
)
,x
h(x
x
x
f
x
c
)
,x
h(x
x
x
f
x


Three equations,
three unknowns,
we can solve.
Example with nonlinear equality
31
Define the Lagrangian function:
 
 
3
2
)
,
(
)
,
,
(
2
1
2
2
2
1
2
1
2
1
2
1







x
x
x
x
c
)
,x
h(x
x
x
f
x
x



L
0
)
,
,
( 2
1 
 
x
x
L
0
)
,
,
(
0
)
,
,
(
0
)
,
,
(
2
1
2
1
2
2
1
1













x
x
x
x
x
x
x
x
L
L
L
OR
  0
3
2
)
,
,
(
0
2
2
)
,
,
(
0
2
2
)
,
,
(
2
1
2
1
1
2
2
1
2
2
1
2
1
1
















x
x
x
x
x
x
x
x
x
x
x
x
x
x






L
L
L
You can solve this algebraically to obtain
2247
.
1
2
3
2
1 

 x
x
2247
.
1
2
3
2
1 



 x
x
and f=3 in
both cases
Example with nonlinear equality
Our approach worked in this case, i.e., we
found a local optimal point that was also a
global optimal point, but because it was
not a convex programming problem, we
had no guarantee that this would happen.
The conditions we established, below, we call first order conditions.
For convex programming problems, they are first order sufficient conditions to provide
the global optimal point.
For nonconvex programming problems, they are first order necessary conditions to
provide the global optimal point.
0
)
,
(
0
)
,
(









x
x
x
L
L
Multiple equality constraints
33
We assume that f and
h are continuously
differentiable.
c
x
h
s.t.
x
f

)
(
)
(
min
   
 
m
m
m c
)
x
(
h
c
)
x
(
h
c
)
x
(
h
x
f
x












...
)
(
)
,
( 2
2
2
1
1
1
L
First order necessary conditions that (x*, λ*) solves the above:
0
)
,
(
0
)
,
(
*
*
*
*









x
x
x
L
L
Multiple equality & 1 inequality constraint
34
We assume that f, h, and
g are continuously
differentiable.
Solution approach:
• Ignore the inequality constraint and solve the problem.
(this is just a problem with multiple equality constraints).
• If inequality constraint is satisfied, then problem is solved.
• If inequality constraint is violated, then the inequality constraint
must be binding  inequality constraint enforced with equality:
b
x
g
c
)
x
(
h
s.t.
x
f


)
(
)
(
min
b
x
g 
)
(
Let’s look at this new problem where the inequality is binding.
Multiple equality & 1 inequality constraint
35
We assume that f, h, and
g are continuously
differentiable.
b
x
g
c
)
x
(
h
s.t.
x
f


)
(
)
(
min
   
   
b
x
g
c
)
x
(
h
c
)
x
(
h
c
)
x
(
h
x
f
x
m
m
m 









)
(
...
)
(
)
,
,
( 2
2
2
1
1
1






L
First order necessary conditions that (x*, λ*, μ*) solves the above:
0
*)
,
,
(
0
*)
,
,
(
0
)
,
,
(
*
*
*
*
*
*
*

















x
x
x
x
L
L
L
We were able to write down this
solution only after we knew the
inequality constraint was binding.
Can we generalize this approach?
Multiple equality & 1 inequality constraint
36
   
   
b
x
g
c
)
x
(
h
c
)
x
(
h
c
)
x
(
h
x
f
x
m
m
m 









)
(
...
)
(
)
,
,
( 2
2
2
1
1
1






L
If inequality is not binding, then apply first order necessary
conditions by ignoring it:
μ=0
g(x)-b≠0 (since it is not binding!)
If inequality is binding, then apply first order necessary
conditions treating inequality constraint as an equality constraint
μ≠0
g(x)-b≠0 (since it is binding!)
Either way:
μ(g(x)-b)=0
This relation encodes our
solution procedure!
It can be used to generalize our
necessary conditions
Multiple equality & multiple inequality constraints
37
We assume that f, h, and
g are continuously
differentiable.
b
x
g
c
)
x
(
h
s.t.
x
f


)
(
)
(
min
     
     
1
1
1
1
2
1
1
1
2
2
2
1
1
1
)
(
...
)
(
)
(
...
)
(
)
,
,
(
b
x
g
b
x
g
b
x
g
c
)
x
(
h
-
c
)
x
(
h
c
)
x
(
h
x
f
x
n
m
m
m






















L
First order necessary conditions that (x*, λ*, μ*) solves the above:
k
k
b
x
g
x
x
x
x
k
k
k














0
0
)
)
(
(
0
*)
,
,
(
0
*)
,
,
(
0
)
,
,
(
*
*
*
*
*
*
*
*
*
*










L
L
L
Complementarity condition:
Inactive constraints have a
zero multiplier.
Nonnegativity
on inequality
multipliers.
These conditions also referred
to as the Kurash-Kuhn-
Tucker (KKT) conditions
An additional requirement
38
We assume that f, h, and
g are continuously
differentiable.
b
x
g
c
)
x
(
h
s.t.
x
f


)
(
)
(
min
For KKT to guarantee finds a local optimum, we need the Kuhn-
Tucker Constraint Qualification (even under convexity).
This condition imposes a certain restriction on the constraint functions .
Its purpose is to rule out certain irregularities on the boundary of the
feasible set, that would invalidate the Kuhn-Tucker conditions should the
optimal solution occur there.
We will not try to tackle this idea, but know this:
If the feasible region is a convex set formed by linear
constraints only, then the constraint qualification will be met, and
the Kuhn-Tucker conditions will always hold at an optimal
solution.
Generator unit cost function:
COSTi =
where
COSTi = production cost
Pi = production power
Economic dispatch calculation (EDC)
    i
i
i
i
i
i
i c
P
b
P
a
P
C 


2
Unit capacity limits
i
i P
P i
i P
P 
level
generation
P
P
level
generation
P
P
where
i
i
max
min
:
max
min




Notation: double underline means lower bound.
Double overline means upper bound.
T
tie
LOSS
D
n
i
i P
P
P
P
P 




1
Power balance
(no transmission
representation)
 



n
i
i
i
i P
C
P
f
1
)
(
min
Subject to
i
i
i
i
i
i
T
n
i
i
P
P
P
P
P
P
P
P








1
General EDC problem statement.
     
 
   
   
2
2
2
2
2
2
1
1
1
1
1
1
2
1
2
2
1
1
2
2
1
1
2
1 ,
,
,
,
,
,
P
P
P
P
P
P
P
P
P
P
P
P
C
P
C
P
P
T

























L
Two unit system, Lagrangian function:
 
 
 
     
   
2
,
1
,
0
2
,
1
,
0
0
,
0
0
,
0
0
0
0
0
0
0
0
2
2
2
2
2
2
1
1
1
1
1
1
2
1
2
2
2
2
2
2
1
1
1
1
1
1













































k
k
P
P
P
P
P
P
P
P
c
x
h
P
P
P
P
P
C
P
P
P
C
P
k
k
T














L
L
L
Two unit system, KKT conditions:
Unit 1 Unit 2
Generation
Specifications:
Minimum Generation 200 MW 100 MW
Maximum Generation 380 MW 200 MW
Cost Curve Coefficients:
Quadratic Term 0.016 0.019
Linear Term 2.187 2.407
Constant Term 120.312 74.074
   
   
100
,
200
200
100
200
,
380
380
200
400
)
,
(
074
.
74
407
.
2
019
.
0
)
(
312
.
120
187
.
2
016
.
0
)
(
2
2
2
1
1
1
2
1
2
1
2
2
2
2
2
1
2
1
1
1























P
P
P
P
P
P
P
P
P
P
h
P
P
P
C
P
P
P
C
     
   
 
   
   
200
100
380
200
400
074
.
74
407
.
2
019
.
0
312
.
120
187
.
2
016
.
0
,
,
,
,
,
,
2
2
2
2
1
1
1
1
2
1
2
2
2
1
2
1
2
2
1
1
2
1



















P
P
P
P
P
P
P
P
P
P
P
P










L
LaGrangian function
 
 
 
     
    0
200
,
0
100
0
380
,
0
200
0
0
400
0
0
407
.
2
038
.
0
0
0
187
.
2
032
.
0
0
2
2
2
2
1
1
1
1
2
1
2
2
2
2
1
1
1
1







































P
P
P
P
c
x
g
P
P
P
P
P
P












L
L
L
KKT conditions
Assume all inequality constraints are non-binding.
This means that
n
i
and i
i ,
1
0
0 


 

 
 
0
400
0
407
.
2
038
.
0
0
187
.
2
032
.
0
2
1
2
1









P
P
P
P


And KKT conditions become
   
   
400
407
.
2
038
.
0
187
.
2
0
032
.
0
2
1
2
1
2
1










P
P
P
P
P
P


Rewrite them as:
And it is easy to see
how to put them
into matrix form for
solution in matlab.




































400
407
.
2
187
.
2
0
1
1
1
038
.
0
0
1
0
032
.
0
2
1

P
P
Solution yields:





















24
.
9
71
.
179
29
.
220
2
1

P
P
What is = $9.24/MW-hr ???
It is the system “incremental cost.”
It is the cost if the system provides an
additional MW over the next hour.
It is the cost of “increasing” the RHS of the
equality constraint by 1 MW for an hour.
We can verify this.
Verification for meaning of lambda.
• Compute total costs/hr for Pd=400 MW
• Compute total costs/hr for Pd=401 MW
• Find the difference in total costs/hr for
the two demands.
If our interpretation of lambda is correct,
this difference should be $9.24.
     
 
     
  hr
P
C
P
C
hr
P
C
P
C
/
$
25
.
1120
074
.
74
71
.
179
407
.
2
71
.
179
019
.
0
/
$
53
.
1378
312
.
120
29
.
220
187
.
2
29
.
220
016
.
0
2
2
2
2
2
1
1
2
1
1








    78
.
2498
25
.
1120
53
.
1378
2
2
1
1 



 P
C
P
C
CT
Total cost/hr are C1+C2
Get cost/hr for each unit.
Now solve EDC for Pd=401 MW to get P1,P2




































401
407
.
2
187
.
2
0
1
1
1
038
.
0
0
1
0
032
.
0
2
1

P
P





















25
.
9
17
.
180
83
.
220
2
1

P
P
     
 
     
  hr
P
C
P
C
hr
P
C
P
C
/
$
51
.
1124
074
.
74
17
.
180
407
.
2
17
.
180
019
.
0
/
$
52
.
1383
312
.
120
83
.
220
187
.
2
83
.
220
016
.
0
2
2
2
2
2
1
1
2
1
1








Total cost/hr are C1+C2
Get cost/hr for each unit.
    03
.
2508
51
.
1124
52
.
1383
2
2
1
1 



 P
C
P
C
CT
Total cost/hr changed by 2508.03-2498.78 = 9.25 $/hr,
which is in agreement with our interpretation of lambda.

Más contenido relacionado

Similar a EE 553 Introduction to Optimization Key Concepts

9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx
9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx
9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptxdnbtraniemyu
 
Simplex method - Maximisation Case
Simplex method - Maximisation CaseSimplex method - Maximisation Case
Simplex method - Maximisation CaseJoseph Konnully
 
Limit 140929031133-phpapp01
Limit 140929031133-phpapp01Limit 140929031133-phpapp01
Limit 140929031133-phpapp01rakambantah
 
2012 mdsp pr13 support vector machine
2012 mdsp pr13 support vector machine2012 mdsp pr13 support vector machine
2012 mdsp pr13 support vector machinenozomuhamada
 
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...IJERA Editor
 
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...IJERA Editor
 
LP special cases and Duality.pptx
LP special cases and Duality.pptxLP special cases and Duality.pptx
LP special cases and Duality.pptxSnehal Athawale
 
4optmizationtechniques-150308051251-conversion-gate01.pdf
4optmizationtechniques-150308051251-conversion-gate01.pdf4optmizationtechniques-150308051251-conversion-gate01.pdf
4optmizationtechniques-150308051251-conversion-gate01.pdfBechanYadav4
 
Basics of Integration and Derivatives
Basics of Integration and DerivativesBasics of Integration and Derivatives
Basics of Integration and DerivativesFaisal Waqar
 
Optimization Techniques.pdf
Optimization Techniques.pdfOptimization Techniques.pdf
Optimization Techniques.pdfanandsimple
 

Similar a EE 553 Introduction to Optimization Key Concepts (20)

Dynamic pgmming
Dynamic pgmmingDynamic pgmming
Dynamic pgmming
 
Duality
DualityDuality
Duality
 
Lp (2)
Lp (2)Lp (2)
Lp (2)
 
Signals and Systems Homework Help.pptx
Signals and Systems Homework Help.pptxSignals and Systems Homework Help.pptx
Signals and Systems Homework Help.pptx
 
9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx
9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx
9 Multi criteria Operation Decision Making - Nov 16 2020. pptx (ver2).pptx
 
Simplex method - Maximisation Case
Simplex method - Maximisation CaseSimplex method - Maximisation Case
Simplex method - Maximisation Case
 
Limit 140929031133-phpapp01
Limit 140929031133-phpapp01Limit 140929031133-phpapp01
Limit 140929031133-phpapp01
 
Graphical method
Graphical methodGraphical method
Graphical method
 
2012 mdsp pr13 support vector machine
2012 mdsp pr13 support vector machine2012 mdsp pr13 support vector machine
2012 mdsp pr13 support vector machine
 
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
 
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
Determination of Optimal Product Mix for Profit Maximization using Linear Pro...
 
Limit
LimitLimit
Limit
 
Linear programing problem
Linear programing problemLinear programing problem
Linear programing problem
 
LP special cases and Duality.pptx
LP special cases and Duality.pptxLP special cases and Duality.pptx
LP special cases and Duality.pptx
 
4optmizationtechniques-150308051251-conversion-gate01.pdf
4optmizationtechniques-150308051251-conversion-gate01.pdf4optmizationtechniques-150308051251-conversion-gate01.pdf
4optmizationtechniques-150308051251-conversion-gate01.pdf
 
Optmization techniques
Optmization techniquesOptmization techniques
Optmization techniques
 
optmizationtechniques.pdf
optmizationtechniques.pdfoptmizationtechniques.pdf
optmizationtechniques.pdf
 
Basics of Integration and Derivatives
Basics of Integration and DerivativesBasics of Integration and Derivatives
Basics of Integration and Derivatives
 
Optimization Techniques.pdf
Optimization Techniques.pdfOptimization Techniques.pdf
Optimization Techniques.pdf
 
Optimization tutorial
Optimization tutorialOptimization tutorial
Optimization tutorial
 

Más de raj20072

Genet algo.ppt
Genet algo.pptGenet algo.ppt
Genet algo.pptraj20072
 
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...raj20072
 

Más de raj20072 (7)

PS.pptx
PS.pptxPS.pptx
PS.pptx
 
CN.ppt
CN.pptCN.ppt
CN.ppt
 
pt.pptx
pt.pptxpt.pptx
pt.pptx
 
pso.ppt
pso.pptpso.ppt
pso.ppt
 
nsga.ppt
nsga.pptnsga.ppt
nsga.ppt
 
Genet algo.ppt
Genet algo.pptGenet algo.ppt
Genet algo.ppt
 
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...
A Decomposition Aggregation Method for Solving Electrical Power Dispatch Prob...
 

Último

Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfDrew Moseley
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSneha Padhiar
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfalene1
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfShreyas Pandit
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...Erbil Polytechnic University
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionMebane Rash
 
TEST CASE GENERATION GENERATION BLOCK BOX APPROACH
TEST CASE GENERATION GENERATION BLOCK BOX APPROACHTEST CASE GENERATION GENERATION BLOCK BOX APPROACH
TEST CASE GENERATION GENERATION BLOCK BOX APPROACHSneha Padhiar
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxRomil Mishra
 
Detection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackingDetection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackinghadarpinhas1
 
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...gerogepatton
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESkarthi keyan
 
Katarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School CourseKatarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School Coursebim.edu.pl
 
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...KrishnaveniKrishnara1
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxRomil Mishra
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdfsahilsajad201
 
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmComputer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmDeepika Walanjkar
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdfCaalaaAbdulkerim
 
Turn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxTurn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxStephen Sitton
 
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSHigh Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSsandhya757531
 

Último (20)

Immutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdfImmutable Image-Based Operating Systems - EW2024.pdf
Immutable Image-Based Operating Systems - EW2024.pdf
 
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATIONSOFTWARE ESTIMATION COCOMO AND FP CALCULATION
SOFTWARE ESTIMATION COCOMO AND FP CALCULATION
 
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdfComprehensive energy systems.pdf Comprehensive energy systems.pdf
Comprehensive energy systems.pdf Comprehensive energy systems.pdf
 
Theory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdfTheory of Machine Notes / Lecture Material .pdf
Theory of Machine Notes / Lecture Material .pdf
 
"Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ..."Exploring the Essential Functions and Design Considerations of Spillways in ...
"Exploring the Essential Functions and Design Considerations of Spillways in ...
 
Versatile Engineering Construction Firms
Versatile Engineering Construction FirmsVersatile Engineering Construction Firms
Versatile Engineering Construction Firms
 
US Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of ActionUS Department of Education FAFSA Week of Action
US Department of Education FAFSA Week of Action
 
TEST CASE GENERATION GENERATION BLOCK BOX APPROACH
TEST CASE GENERATION GENERATION BLOCK BOX APPROACHTEST CASE GENERATION GENERATION BLOCK BOX APPROACH
TEST CASE GENERATION GENERATION BLOCK BOX APPROACH
 
Mine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptxMine Environment II Lab_MI10448MI__________.pptx
Mine Environment II Lab_MI10448MI__________.pptx
 
Detection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and trackingDetection&Tracking - Thermal imaging object detection and tracking
Detection&Tracking - Thermal imaging object detection and tracking
 
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
March 2024 - Top 10 Read Articles in Artificial Intelligence and Applications...
 
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTESCME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
CME 397 - SURFACE ENGINEERING - UNIT 1 FULL NOTES
 
Katarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School CourseKatarzyna Lipka-Sidor - BIM School Course
Katarzyna Lipka-Sidor - BIM School Course
 
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...
22CYT12 & Chemistry for Computer Systems_Unit-II-Corrosion & its Control Meth...
 
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptxTriangulation survey (Basic Mine Surveying)_MI10412MI.pptx
Triangulation survey (Basic Mine Surveying)_MI10412MI.pptx
 
Robotics Group 10 (Control Schemes) cse.pdf
Robotics Group 10  (Control Schemes) cse.pdfRobotics Group 10  (Control Schemes) cse.pdf
Robotics Group 10 (Control Schemes) cse.pdf
 
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithmComputer Graphics Introduction, Open GL, Line and Circle drawing algorithm
Computer Graphics Introduction, Open GL, Line and Circle drawing algorithm
 
Research Methodology for Engineering pdf
Research Methodology for Engineering pdfResearch Methodology for Engineering pdf
Research Methodology for Engineering pdf
 
Turn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptxTurn leadership mistakes into a better future.pptx
Turn leadership mistakes into a better future.pptx
 
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMSHigh Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
High Voltage Engineering- OVER VOLTAGES IN ELECTRICAL POWER SYSTEMS
 

EE 553 Introduction to Optimization Key Concepts

  • 1. EE 553 Introduction to Optimization J. McCalley 1
  • 2. Real-time Electricity markets and tools Day-ahead SCUC and SCED SCED Minimize f(x) subject to h(x)=c g(x)< b BOTH LOOK LIKE THIS SCUC: x contains discrete & continuous variables. SCED: x contains only continuous variables. 2
  • 3. Optimization Terminology Minimize f(x) subject to h(x)=c g(x)> b f(x): Objective function x: Decision variables h(x)=c: Equality constraint g(x)> b: Inequality constraint An optimization problem or a mathematical program or a mathematical programming problem. x*: solution 3
  • 4. Classification of Optimization Problems http://www.neos-guide.org/NEOS/index.php/Optimization_Tree Continuous Optimization Unconstrained Optimization Bound Constrained Optimization Derivative-Free Optimization Global Optimization Linear Programming Network Flow Problems Nondifferentiable Optimization Nonlinear Programming Optimization of Dynamic Systems Quadratic Constrained Quadratic Programming Quadratic Programming Second Order Cone Programming Semidefinite Programming Semiinfinite Programming Discrete and Integer Optimization Combinatorial Optimization Traveling Salesman Problem Integer Programming Mixed Integer Linear Programming Mixed Integer Nonlinear Programming Optimization Under Uncertainty Robust Optimization Stochastic Programming Simulation/Noisy Optimization Stochastic Algorithms Complementarity Constraints and Variational Inequalities Complementarity Constraints Game Theory Linear Complementarity Problems Mathematical Programs with Complementarity Constraints Nonlinear Complementarity Problems Systems of Equations Data Fitting/Robust Estimation Nonlinear Equations Nonlinear Least Squares Systems of Inequalities Multiobjective Optimization 4
  • 5. Convex functions Definition #1: A function f(x) is convex in an interval if its second derivative is positive on that interval. Example: f(x)=x2 is convex since f’(x)=2x, f’’(x)=2>0 5
  • 6. Convex functions The second derivative test is sufficient but not necessary. www.ebyte.it/library/docs/math09/AConvexInequality.html Definition #2: A function f(x) is convex if a line drawn between any two points on the function remains on or above the function in the interval between the two points. 6
  • 7. Convex functions Definition #2: A function f(x) is convex if a line drawn between any two points on the function remains on or above the function in the interval between the two points. Is a linear function convex? Answer is “yes” since a line drawn between any two points on the function remains on the function. 7
  • 8. Convex Sets Definition #3: A set C is convex if a line segment between any two points in C lies in C. Ex: Which of the below are convex sets? The set on the left is convex. The set on the right is not. 8
  • 9. Convex Sets Definition #3: A set C is convex if a line segment between any two points in C lies in C. S. Boyd and L. Vandenberghe, “Convex optimization,” Cambridge University Press, 2004. 9
  • 10. Global vs. local optima Example: Solve the following: Minimize f(x)=x2 Solution: f’(x)=2x=0x*=0. This solution is a local optimum. It is also the global optimum. Example: Solve the following: Minimize f(x)=x3-17x2+80x-100 Solution: f’(x)=3x2-34x+80=0 Solving the above results in x=3.33 and x=8. Issue#1: Which is the best solution? Issue#2: Is the best solution the global solution? 10
  • 11. Global vs. local optima Example: Solve the following: Minimize f(x)=x3-17x2+80x-100 Solution: f’(x)=3x2-34x+80=0. Solving results in x=3.33, x=8. Issue#1: Which is the best solution? Issue#2: Is the best solution the global solution? x=8 No! It is unbounded. 11
  • 12. Convexity & global vs. local optima When minimizing a function, if we want to be sure that we can get a global solution via differentiation, we need to impose some requirements on our objective function. We will also need to impose some requirements on the feasible set S (set of possible values the solution x* may take). Min f(x) subject to h(x)=c g(x)> b S x x f  subject to ) ( min Definition: If f(x) is a convex function, and if S is a convex set, then the above problem is a convex programming problem. Definition: If f(x) is not a convex function, or if S is not a convex set, then the above problem is a non-convex programming problem. 12 Feasible set
  • 13. Convex vs. nonconvex programming problems The desirable quality of a convex programming problem is that any locally optimal solution is also a globally optimal solution. If we have a method of finding a locally optimal solution, that method also finds for us the globally optimum solution. 13 The undesirable quality of a non-convex programming problem is that any method which finds a locally optimal solution does not necessarily find the globally optimum solution. MATHEMATICAL PROGRAMMING Convex Non-convex We address convex programming problems in addressing linear programming. We will also, later, address a special form of non-convex programming problems called integer programs.
  • 14. A convex programming problem 14 c x x h s.t. x x f  ) , ( ) , ( min 2 1 2 1 c ) x h( x f  s.t. ) ( min c ) x ( h x f  s.t. ) ( min Two variables with one equality-constraint Multi-variable with one equality-constraint. Multi-variable with multiple equality-constraints. We focus on this one, but conclusions we derive will also apply to the other two. The benefit of focusing on this one is that we can visualize it.
  • 15. Contour maps 15 Definition: A contour map is a 2-dimensional plane, i.e., a coordinate system in 2 variables, say, x1, x2, that illustrates curves (contours) of constant functional value f(x1, x2). Example: Draw the contour map for 2 2 2 1 2 1 ) , ( x x x x f   . [X,Y] = meshgrid(- 2.0:.2:2.0,-2.0:.2:2.0); Z = X.^2+Y.^2; [c,h]=contour(X,Y,Z); clabel(c,h); grid; xlabel('x1'); ylabel('x2');
  • 16. Contour maps and 3-D illustrations 16 Example: Draw the 3-D surface for 2 2 2 1 2 1 ) , ( x x x x f   . [X,Y] = meshgrid(- 2.0:.2:2.0,-2.0:.2:2.0); Z = X.^2+Y.^2; surfc(X,Y,Z) xlabel('x1') ylabel('x2') zlabel('f(x1,x2)') Height is f(x) Contours Each contour of fixed value f is the projection onto the x1-x2 plane of a horizontal slice made of the 3-D figure at a value f above the x1-x2 plane.
  • 17. Solving a convex program: graphical analysis 17 Example: Solve this convex program: . 6 ) , ( min 2 1 2 1 2 2 2 1 2 1      x x ) ,x h(x s.t. x x x x f 6 6 1 2 2       x x x x1 Superimpose this relation on top of the contour plot for f(x1,x2). 1. f(x1,x2) must be minimized, and so we would like the solution to be as close to the origin as possible; 2. The solution must be on the thick line in the right-hand corner of the plot, since this line represents the equality constraint. A straight line is a convex set because a line segment between any two points on it remain on it.
  • 18. Solving a convex program: graphical analysis 18 . ) 25 . 1 , 25 . 1 ( )* , ( * 2 1   x x x 3 )* , ( 2 1  x x f Solution: Any contour f<3 does not intersect the equality constraint; Any contour f>3 intersects the equality constraint at two points. The contour f=3 and the equality constraint just touch each other at the point x*. “Just touch”: The two curves are tangent to one another at the solution point.
  • 19. Solving a convex program: graphical analysis 19 . The two curves are tangent to one another at the solution point.  The normal (gradient) vectors of the two curves, at the solution (tangent) point, are parallel. “Parallel” means that the two vectors have the same direction. We do not know that they have the same magnitude. To account for this, we equate with a “multiplier” λ:   c ) ,x h(x x x f     * 2 1 * 2 1 ) , (      * * 2 2 1 * 2 1 * 2 2 2 1 2 1 1 1 6 } 6 * ) , ( { 2 2 )*} , ( {                         x x x x h x x x x x x f 1 This means the following two vectors are parallel:
  • 20. Solving a convex program: graphical analysis 20 .   c ) ,x h(x x x f     * 2 1 * 2 1 ) , (    0 ) , ( * 2 1 * 2 1      c ) ,x h(x x x f  Moving everything to the left:   0 ) , ( * 2 1 * 2 1      ) ,x h(x c x x f  Alternately:                                    0 0 ) , ( ) , ( * 2 1 2 1 2 2 1 2 1 1 c ) ,x h(x x x f x c ) ,x h(x x x f x   Performing the gradient operation (taking derivatives with respect to x1 and x2) : In this problem, we already know the solution, but what if we did not? Then could we use the above equations to find the solution?
  • 21. Solving a convex program: analytical analysis 21                                    0 0 ) , ( ) , ( * 2 1 2 1 2 2 1 2 1 1 c ) ,x h(x x x f x c ) ,x h(x x x f x   In this problem, we already know the solution, but what if we did not? Then could we use the above equations to find the solution? NO! Because we only have 2 equations, yet 3 unknowns: x1, x2, λ. So we need another equation. Where do we get that equation? Recall our equality constraint: h(x1, x2)-c=0 . This must be satisfied! Therefore:                                             0 0 0 ) , ( ) , ( ) , ( * 2 1 2 1 2 1 2 2 1 2 1 1 c x x h c ) ,x h(x x x f x c ) ,x h(x x x f x   Three equations, three unknowns, we can solve.
  • 22. Solving a convex program: analytical analysis 22 Observation: The three equations are simply partial derivatives of the function This is obviously true for the first two equations , but it is not so obviously true for the last one. But to see it, observe                                           0 0 0 ) , ( ) , ( ) , ( * 2 1 2 1 2 1 2 2 1 2 1 1 c x x h c ) ,x h(x x x f x c ) ,x h(x x x f x     c ) ,x h(x x x f   2 1 2 1 ) , (      c ) ,x h(x c ) ,x h(x c ) ,x h(x x x f            2 1 2 1 2 1 2 1 0 0 ) , (  
  • 23. Formal approach to solving our problem 23 Define the Lagrangian function: In a convex programming problem, the “first-order conditions” for finding the solution is given by   c ) ,x h(x x x f x x    2 1 2 1 2 1 ) , ( ) , , (   L 0 ) , , ( 2 1    x x L 0 ) , , ( 0 ) , , ( 0 ) , , ( 2 1 2 1 2 2 1 1              x x x x x x x x L L L OR Or more compactly 0 ) , ( 0 ) , (          x x x L L where we have used x=(x1, x2)
  • 24. Applying to our example 24 Define the Lagrangian function:     6 ) , ( ) , , ( 2 1 2 2 2 1 2 1 2 1 2 1         x x x x c ) ,x h(x x x f x x    L 0 ) , , ( 2 1    x x L 0 ) , , ( 0 ) , , ( 0 ) , , ( 2 1 2 1 2 2 1 1              x x x x x x x x L L L OR   x x x x x x x x x x x x 0 6 ) , , ( 0 2 ) , , ( 0 2 ) , , ( 2 1 2 1 2 2 1 2 1 2 1 1                        L L L A set of 3 linear equations and 3 unknowns; we can write in the form of Ax=b.
  • 25. Applying to our example 25                                                                                4495 . 2 2247 . 1 2247 . 1 6 0 0 0 1 1 1 2 0 1 0 2 6 0 0 0 1 1 1 2 0 1 0 2 1 2 1 2 1   x x x x
  • 26. Now, let’s go back to our example with a nonlinear equality constraint.
  • 27. Example with nonlinear equality 27 Non-convex because a line connecting two points in the set do not remain in the set. (see “notes” of this slide) . 3 2 ) , ( min 2 1 2 1 2 2 2 1 2 1     x x ) ,x h(x s.t. x x x x f 1 2 2 1 2 3 3 2 x x x x    Superimpose this relation on top of the contour plot for f(x1,x2). 1. f(x1,x2) must be minimized, and so we would like the solution to be as close to the origin as possible; 2. The solution must be on the thick line in the right-hand corner of the plot, since this line represents the equality constraint.
  • 28. Example with nonlinear equality 28 . ) 25 . 1 , 25 . 1 ( )* , ( * 2 1   x x x 3 )* , ( 2 1  x x f Solution: Any contour f<3 does not intersect the equality constraint; Any contour f>3 intersects the equality constraint at two points. The contour f=3 and the equality constraint just touch each other at the point x*. “Just touch”: The two curves are tangent to one another at the solution point.
  • 29. Example with nonlinear equality 29 . The two curves are tangent to one another at the solution point.  The normal (gradient) vectors of the two curves, at the solution (tangent) point, are parallel. “Parallel” means that the two vectors have the same direction. We do not know that they have the same magnitude. To account for this, we equate with a “multiplier” λ:   c ) ,x h(x x x f     * 2 1 * 2 1 ) , (      * 1 2 * 2 2 1 * 2 1 * 2 2 2 1 2 1 2 2 3 2 } 3 * ) , ( { 2 2 )*} , ( {                        x x x x x x h x x x x x x f 1 This means the following two vectors are parallel:
  • 30. Example with nonlinear equality 30                                    0 0 ) , ( ) , ( * 2 1 2 1 2 2 1 2 1 1 c ) ,x h(x x x f x c ) ,x h(x x x f x   This gives us the following two equations. And we add the equality constraint to give 3 equations, 3 unknowns:                                             0 0 0 ) , ( ) , ( ) , ( * 2 1 2 1 2 1 2 2 1 2 1 1 c x x h c ) ,x h(x x x f x c ) ,x h(x x x f x   Three equations, three unknowns, we can solve.
  • 31. Example with nonlinear equality 31 Define the Lagrangian function:     3 2 ) , ( ) , , ( 2 1 2 2 2 1 2 1 2 1 2 1        x x x x c ) ,x h(x x x f x x    L 0 ) , , ( 2 1    x x L 0 ) , , ( 0 ) , , ( 0 ) , , ( 2 1 2 1 2 2 1 1              x x x x x x x x L L L OR   0 3 2 ) , , ( 0 2 2 ) , , ( 0 2 2 ) , , ( 2 1 2 1 1 2 2 1 2 2 1 2 1 1                 x x x x x x x x x x x x x x       L L L You can solve this algebraically to obtain 2247 . 1 2 3 2 1    x x 2247 . 1 2 3 2 1      x x and f=3 in both cases
  • 32. Example with nonlinear equality Our approach worked in this case, i.e., we found a local optimal point that was also a global optimal point, but because it was not a convex programming problem, we had no guarantee that this would happen. The conditions we established, below, we call first order conditions. For convex programming problems, they are first order sufficient conditions to provide the global optimal point. For nonconvex programming problems, they are first order necessary conditions to provide the global optimal point. 0 ) , ( 0 ) , (          x x x L L
  • 33. Multiple equality constraints 33 We assume that f and h are continuously differentiable. c x h s.t. x f  ) ( ) ( min       m m m c ) x ( h c ) x ( h c ) x ( h x f x             ... ) ( ) , ( 2 2 2 1 1 1 L First order necessary conditions that (x*, λ*) solves the above: 0 ) , ( 0 ) , ( * * * *          x x x L L
  • 34. Multiple equality & 1 inequality constraint 34 We assume that f, h, and g are continuously differentiable. Solution approach: • Ignore the inequality constraint and solve the problem. (this is just a problem with multiple equality constraints). • If inequality constraint is satisfied, then problem is solved. • If inequality constraint is violated, then the inequality constraint must be binding  inequality constraint enforced with equality: b x g c ) x ( h s.t. x f   ) ( ) ( min b x g  ) ( Let’s look at this new problem where the inequality is binding.
  • 35. Multiple equality & 1 inequality constraint 35 We assume that f, h, and g are continuously differentiable. b x g c ) x ( h s.t. x f   ) ( ) ( min         b x g c ) x ( h c ) x ( h c ) x ( h x f x m m m           ) ( ... ) ( ) , , ( 2 2 2 1 1 1       L First order necessary conditions that (x*, λ*, μ*) solves the above: 0 *) , , ( 0 *) , , ( 0 ) , , ( * * * * * * *                  x x x x L L L We were able to write down this solution only after we knew the inequality constraint was binding. Can we generalize this approach?
  • 36. Multiple equality & 1 inequality constraint 36         b x g c ) x ( h c ) x ( h c ) x ( h x f x m m m           ) ( ... ) ( ) , , ( 2 2 2 1 1 1       L If inequality is not binding, then apply first order necessary conditions by ignoring it: μ=0 g(x)-b≠0 (since it is not binding!) If inequality is binding, then apply first order necessary conditions treating inequality constraint as an equality constraint μ≠0 g(x)-b≠0 (since it is binding!) Either way: μ(g(x)-b)=0 This relation encodes our solution procedure! It can be used to generalize our necessary conditions
  • 37. Multiple equality & multiple inequality constraints 37 We assume that f, h, and g are continuously differentiable. b x g c ) x ( h s.t. x f   ) ( ) ( min             1 1 1 1 2 1 1 1 2 2 2 1 1 1 ) ( ... ) ( ) ( ... ) ( ) , , ( b x g b x g b x g c ) x ( h - c ) x ( h c ) x ( h x f x n m m m                       L First order necessary conditions that (x*, λ*, μ*) solves the above: k k b x g x x x x k k k               0 0 ) ) ( ( 0 *) , , ( 0 *) , , ( 0 ) , , ( * * * * * * * * * *           L L L Complementarity condition: Inactive constraints have a zero multiplier. Nonnegativity on inequality multipliers. These conditions also referred to as the Kurash-Kuhn- Tucker (KKT) conditions
  • 38. An additional requirement 38 We assume that f, h, and g are continuously differentiable. b x g c ) x ( h s.t. x f   ) ( ) ( min For KKT to guarantee finds a local optimum, we need the Kuhn- Tucker Constraint Qualification (even under convexity). This condition imposes a certain restriction on the constraint functions . Its purpose is to rule out certain irregularities on the boundary of the feasible set, that would invalidate the Kuhn-Tucker conditions should the optimal solution occur there. We will not try to tackle this idea, but know this: If the feasible region is a convex set formed by linear constraints only, then the constraint qualification will be met, and the Kuhn-Tucker conditions will always hold at an optimal solution.
  • 39. Generator unit cost function: COSTi = where COSTi = production cost Pi = production power Economic dispatch calculation (EDC)     i i i i i i i c P b P a P C    2 Unit capacity limits i i P P i i P P  level generation P P level generation P P where i i max min : max min     Notation: double underline means lower bound. Double overline means upper bound. T tie LOSS D n i i P P P P P      1 Power balance (no transmission representation)
  • 40.      n i i i i P C P f 1 ) ( min Subject to i i i i i i T n i i P P P P P P P P         1 General EDC problem statement.                 2 2 2 2 2 2 1 1 1 1 1 1 2 1 2 2 1 1 2 2 1 1 2 1 , , , , , , P P P P P P P P P P P P C P C P P T                          L Two unit system, Lagrangian function:                 2 , 1 , 0 2 , 1 , 0 0 , 0 0 , 0 0 0 0 0 0 0 0 2 2 2 2 2 2 1 1 1 1 1 1 2 1 2 2 2 2 2 2 1 1 1 1 1 1                                              k k P P P P P P P P c x h P P P P P C P P P C P k k T               L L L Two unit system, KKT conditions:
  • 41. Unit 1 Unit 2 Generation Specifications: Minimum Generation 200 MW 100 MW Maximum Generation 380 MW 200 MW Cost Curve Coefficients: Quadratic Term 0.016 0.019 Linear Term 2.187 2.407 Constant Term 120.312 74.074         100 , 200 200 100 200 , 380 380 200 400 ) , ( 074 . 74 407 . 2 019 . 0 ) ( 312 . 120 187 . 2 016 . 0 ) ( 2 2 2 1 1 1 2 1 2 1 2 2 2 2 2 1 2 1 1 1                        P P P P P P P P P P h P P P C P P P C
  • 42.                     200 100 380 200 400 074 . 74 407 . 2 019 . 0 312 . 120 187 . 2 016 . 0 , , , , , , 2 2 2 2 1 1 1 1 2 1 2 2 2 1 2 1 2 2 1 1 2 1                    P P P P P P P P P P P P           L LaGrangian function                 0 200 , 0 100 0 380 , 0 200 0 0 400 0 0 407 . 2 038 . 0 0 0 187 . 2 032 . 0 0 2 2 2 2 1 1 1 1 2 1 2 2 2 2 1 1 1 1                                        P P P P c x g P P P P P P             L L L KKT conditions
  • 43. Assume all inequality constraints are non-binding. This means that n i and i i , 1 0 0           0 400 0 407 . 2 038 . 0 0 187 . 2 032 . 0 2 1 2 1          P P P P   And KKT conditions become         400 407 . 2 038 . 0 187 . 2 0 032 . 0 2 1 2 1 2 1           P P P P P P   Rewrite them as: And it is easy to see how to put them into matrix form for solution in matlab.
  • 45. What is = $9.24/MW-hr ??? It is the system “incremental cost.” It is the cost if the system provides an additional MW over the next hour. It is the cost of “increasing” the RHS of the equality constraint by 1 MW for an hour. We can verify this.
  • 46. Verification for meaning of lambda. • Compute total costs/hr for Pd=400 MW • Compute total costs/hr for Pd=401 MW • Find the difference in total costs/hr for the two demands. If our interpretation of lambda is correct, this difference should be $9.24.
  • 47.                 hr P C P C hr P C P C / $ 25 . 1120 074 . 74 71 . 179 407 . 2 71 . 179 019 . 0 / $ 53 . 1378 312 . 120 29 . 220 187 . 2 29 . 220 016 . 0 2 2 2 2 2 1 1 2 1 1             78 . 2498 25 . 1120 53 . 1378 2 2 1 1      P C P C CT Total cost/hr are C1+C2 Get cost/hr for each unit.
  • 48. Now solve EDC for Pd=401 MW to get P1,P2                                     401 407 . 2 187 . 2 0 1 1 1 038 . 0 0 1 0 032 . 0 2 1  P P                      25 . 9 17 . 180 83 . 220 2 1  P P
  • 49.                 hr P C P C hr P C P C / $ 51 . 1124 074 . 74 17 . 180 407 . 2 17 . 180 019 . 0 / $ 52 . 1383 312 . 120 83 . 220 187 . 2 83 . 220 016 . 0 2 2 2 2 2 1 1 2 1 1         Total cost/hr are C1+C2 Get cost/hr for each unit.     03 . 2508 51 . 1124 52 . 1383 2 2 1 1      P C P C CT Total cost/hr changed by 2508.03-2498.78 = 9.25 $/hr, which is in agreement with our interpretation of lambda.

Notas del editor

  1. Note that every equality constraint h(x) = 0 can be equivalently replaced by a pair of inequality constraints hi(x) <= 0 and -hi(x) <= 0 . Therefore, for theoretical purposes, equality constraints are redundant; however, it can be beneficial to treat them specially in practice. Following from this fact, it is easy to understand why hi(x) = 0 has to be affine as opposed to merely being convex. If hi(x) is convex, hi(x) <= 0 is convex, but -hi(x) <= 0 is concave. Therefore, the only way for hi(x) = 0 to be convex is for hi(x) to be affine.