交大OCW 江蕙如演算法投影片 https://www.youtube.com/watch?v=7a28lIQIHJQ&list=PLj6E8qlqmkFtoRpLn6IXnH_eboef-3QvZ&index=1

1. Iris Hui-Ru Jiang Fall 2014
Linear Programming
IRIS H.-R. JIANG
Linear Programming
¨ Course contents:
¤ Linear programming
¤ Formulation
¤ Duality
¤ The simplex method
¨ Reading:
¤ Chapter 7 (Dasgupta)
¤ Chapter 29 (Cormen)
Linear programming
2
IRIS H.-R. JIANG
Linear Programming
¨ Linear programming describes a broad class of optimization
tasks in which both the optimization criterion and the
constraints are linear functions.
¨ Linear programming consists of three parts:
¤ A set of decision variables
¤ An objective function:
n maximize or minimize a given linear objective function
¤ A set of constraints:
n satisfy a set of linear inequalities involving these variables
Linear programming
3
IRIS H.-R. JIANG
Example: Profit Maximization (1/4)
¨ A boutique chocolatier has two products:
¤ A (Pyramide): profit $1 per box
¤ B (Nuit): profit $6 per box
¨ Constraints:
¤ The daily demand for these exclusive chocolates is limited to at
most 200 boxes of A and 300 boxes of B
¤ The current workforce can produce a total of at most 400 boxes
of chocolate per day
¨ Decision variables:
¤ x1 = Boxes of A
¤ x2 = Boxes of B
¨ Objective Function:
¤ Maximize profit
Linear programming
4

2. Iris Hui-Ru Jiang Fall 2014
Linear Programming
IRIS H.-R. JIANG
Linear Programming
¨ Course contents:
¤ Linear programming
¤ Formulation
¤ Duality
¤ The simplex method
¨ Reading:
¤ Chapter 7 (Dasgupta)
¤ Chapter 29 (Cormen)
Linear programming
2
IRIS H.-R. JIANG
Linear Programming
¨ Linear programming describes a broad class of optimization
tasks in which both the optimization criterion and the
constraints are linear functions.
¨ Linear programming consists of three parts:
¤ A set of decision variables
¤ An objective function:
n maximize or minimize a given linear objective function
¤ A set of constraints:
n satisfy a set of linear inequalities involving these variables
Linear programming
3
IRIS H.-R. JIANG
Example: Profit Maximization (1/4)
¨ A boutique chocolatier has two products:
¤ A (Pyramide): profit $1 per box
¤ B (Nuit): profit $6 per box
¨ Constraints:
¤ The daily demand for these exclusive chocolates is limited to at
most 200 boxes of A and 300 boxes of B
¤ The current workforce can produce a total of at most 400 boxes
of chocolate per day
¨ Decision variables:
¤ x1 = Boxes of A
¤ x2 = Boxes of B
¨ Objective Function:
¤ Maximize profit
Linear programming
4

3. Iris Hui-Ru Jiang Fall 2014
Linear Programming
IRIS H.-R. JIANG
Linear Programming
¨ Course contents:
¤ Linear programming
¤ Formulation
¤ Duality
¤ The simplex method
¨ Reading:
¤ Chapter 7 (Dasgupta)
¤ Chapter 29 (Cormen)
Linear programming
2
IRIS H.-R. JIANG
Linear Programming
¨ Linear programming describes a broad class of optimization
tasks in which both the optimization criterion and the
constraints are linear functions.
¨ Linear programming consists of three parts:
¤ A set of decision variables
¤ An objective function:
n maximize or minimize a given linear objective function
¤ A set of constraints:
n satisfy a set of linear inequalities involving these variables
Linear programming
3
IRIS H.-R. JIANG
Example: Profit Maximization (1/4)
¨ A boutique chocolatier has two products:
¤ A (Pyramide): profit $1 per box
¤ B (Nuit): profit $6 per box
¨ Constraints:
¤ The daily demand for these exclusive chocolates is limited to at
most 200 boxes of A and 300 boxes of B
¤ The current workforce can produce a total of at most 400 boxes
of chocolate per day
¨ Decision variables:
¤ x1 = Boxes of A
¤ x2 = Boxes of B
¨ Objective Function:
¤ Maximize profit
Linear programming
4

4. Iris Hui-Ru Jiang Fall 2014
Linear Programming
IRIS H.-R. JIANG
Linear Programming
¨ Course contents:
¤ Linear programming
¤ Formulation
¤ Duality
¤ The simplex method
¨ Reading:
¤ Chapter 7 (Dasgupta)
¤ Chapter 29 (Cormen)
Linear programming
2
IRIS H.-R. JIANG
Linear Programming
¨ Linear programming describes a broad class of optimization
tasks in which both the optimization criterion and the
constraints are linear functions.
¨ Linear programming consists of three parts:
¤ A set of decision variables
¤ An objective function:
n maximize or minimize a given linear objective function
¤ A set of constraints:
n satisfy a set of linear inequalities involving these variables
Linear programming
3
IRIS H.-R. JIANG
Example: Profit Maximization (1/4)
¨ A boutique chocolatier has two products:
¤ A (Pyramide): profit $1 per box
¤ B (Nuit): profit $6 per box
¨ Constraints:
¤ The daily demand for these exclusive chocolates is limited to at
most 200 boxes of A and 300 boxes of B
¤ The current workforce can produce a total of at most 400 boxes
of chocolate per day
¨ Decision variables:
¤ x1 = Boxes of A
¤ x2 = Boxes of B
¨ Objective Function:
¤ Maximize profit
Linear programming
4

5. IRIS H.-R. JIANG
Example: Profit Maximization (2/4)
¨ A linear equation in x1 and x2 defines a line in the 2D plane
¨ A linear inequality designates a half-space
¨ The set of all feasible solutions of this linear program is the
intersection of five half-spaces. It is a convex polygon
Linear programming
5
IRIS H.-R. JIANG
Example: Profit Maximization (3/4)
¨ Search for the optimal solution
¤ It is a general rule of linear programs that the optimum is
achieved at a vertex of the feasible region.
Linear programming
6
(100, 300)
IRIS H.-R. JIANG
Example: Profit Maximization (4/4)
¨ The Simplex method: hill climbing
¤ George Dantzig, 1947
¤ Starts at a vertex, say (0, 0)
¤ Repeatedly looks for an adjacent vertex (connected by an edge
of the feasible region) of better objective value
¤ Upon reaching a vertex that has no better neighbor, simplex
declares it to be optimal and halts
Linear programming
7
(100, 300)
IRIS H.-R. JIANG
Multipliers?
¨ Optimal: (x1, x2) = (100, 300); objective value = 1900
¨ Can this answer be checked somehow?
¤ (1) + 6*(2):
x1 + 6x2 <= 2000
¤ 0*(1) + 5*(2) + (3):
x1 + 6x2 <= 1900
¤ The multipliers (0, 5, 1) constitute a certificate of optimality
¤ How would we systematically find the magic multipliers?
Linear programming
8
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.

6. IRIS H.-R. JIANG
Example: Profit Maximization (2/4)
¨ A linear equation in x1 and x2 defines a line in the 2D plane
¨ A linear inequality designates a half-space
¨ The set of all feasible solutions of this linear program is the
intersection of five half-spaces. It is a convex polygon
Linear programming
5
IRIS H.-R. JIANG
Example: Profit Maximization (3/4)
¨ Search for the optimal solution
¤ It is a general rule of linear programs that the optimum is
achieved at a vertex of the feasible region.
Linear programming
6
(100, 300)
IRIS H.-R. JIANG
Example: Profit Maximization (4/4)
¨ The Simplex method: hill climbing
¤ George Dantzig, 1947
¤ Starts at a vertex, say (0, 0)
¤ Repeatedly looks for an adjacent vertex (connected by an edge
of the feasible region) of better objective value
¤ Upon reaching a vertex that has no better neighbor, simplex
declares it to be optimal and halts
Linear programming
7
(100, 300)
IRIS H.-R. JIANG
Multipliers?
¨ Optimal: (x1, x2) = (100, 300); objective value = 1900
¨ Can this answer be checked somehow?
¤ (1) + 6*(2):
x1 + 6x2 <= 2000
¤ 0*(1) + 5*(2) + (3):
x1 + 6x2 <= 1900
¤ The multipliers (0, 5, 1) constitute a certificate of optimality
¤ How would we systematically find the magic multipliers?
Linear programming
8
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.

7. IRIS H.-R. JIANG
Example: Profit Maximization (2/4)
¨ A linear equation in x1 and x2 defines a line in the 2D plane
¨ A linear inequality designates a half-space
¨ The set of all feasible solutions of this linear program is the
intersection of five half-spaces. It is a convex polygon
Linear programming
5
IRIS H.-R. JIANG
Example: Profit Maximization (3/4)
¨ Search for the optimal solution
¤ It is a general rule of linear programs that the optimum is
achieved at a vertex of the feasible region.
Linear programming
6
(100, 300)
IRIS H.-R. JIANG
Example: Profit Maximization (4/4)
¨ The Simplex method: hill climbing
¤ George Dantzig, 1947
¤ Starts at a vertex, say (0, 0)
¤ Repeatedly looks for an adjacent vertex (connected by an edge
of the feasible region) of better objective value
¤ Upon reaching a vertex that has no better neighbor, simplex
declares it to be optimal and halts
Linear programming
7
(100, 300)
IRIS H.-R. JIANG
Multipliers?
¨ Optimal: (x1, x2) = (100, 300); objective value = 1900
¨ Can this answer be checked somehow?
¤ (1) + 6*(2):
x1 + 6x2 <= 2000
¤ 0*(1) + 5*(2) + (3):
x1 + 6x2 <= 1900
¤ The multipliers (0, 5, 1) constitute a certificate of optimality
¤ How would we systematically find the magic multipliers?
Linear programming
8
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.

8. IRIS H.-R. JIANG
Example: Profit Maximization (2/4)
¨ A linear equation in x1 and x2 defines a line in the 2D plane
¨ A linear inequality designates a half-space
¨ The set of all feasible solutions of this linear program is the
intersection of five half-spaces. It is a convex polygon
Linear programming
5
IRIS H.-R. JIANG
Example: Profit Maximization (3/4)
¨ Search for the optimal solution
¤ It is a general rule of linear programs that the optimum is
achieved at a vertex of the feasible region.
Linear programming
6
(100, 300)
IRIS H.-R. JIANG
Example: Profit Maximization (4/4)
¨ The Simplex method: hill climbing
¤ George Dantzig, 1947
¤ Starts at a vertex, say (0, 0)
¤ Repeatedly looks for an adjacent vertex (connected by an edge
of the feasible region) of better objective value
¤ Upon reaching a vertex that has no better neighbor, simplex
declares it to be optimal and halts
Linear programming
7
(100, 300)
IRIS H.-R. JIANG
Multipliers?
¨ Optimal: (x1, x2) = (100, 300); objective value = 1900
¨ Can this answer be checked somehow?
¤ (1) + 6*(2):
x1 + 6x2 <= 2000
¤ 0*(1) + 5*(2) + (3):
x1 + 6x2 <= 1900
¤ The multipliers (0, 5, 1) constitute a certificate of optimality
¤ How would we systematically find the magic multipliers?
Linear programming
8
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.

9. IRIS H.-R. JIANG
Duality (1/3)
¨ Multipliers yi’s must be nonnegative
¨ If the left-hand side to look like our objective function, the right-
hand side is an upper bound on the optimum solution
¨ We want a tight bound!
Linear programming
9
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
IRIS H.-R. JIANG
Duality (2/3)
¨ A new LP: finding multipliers that gives the best upper bound
on our original LP
¤ Primal LP
¤ Any feasible value of dual LP is an upper bound on primal LP
¤ If we find a pair of primal and dual feasible values that are equal,
they must be both optimal.
Linear programming
10
¤ Dual LP
IRIS H.-R. JIANG
Duality (3/3)
¨ Generic form:
¨ Dual theorem: If a linear program has a bounded optimum, then
so does its dual, and the new optimum values coincide.
¤ Max-flow min-cut
Linear programming
11
IRIS H.-R. JIANG
The Simplex Algorithm
Linear programming
12
Every constraint specifies an
n-dimensional half-space
Travel along “edges” until no
improvement can be made

10. IRIS H.-R. JIANG
Duality (1/3)
¨ Multipliers yi’s must be nonnegative
¨ If the left-hand side to look like our objective function, the right-
hand side is an upper bound on the optimum solution
¨ We want a tight bound!
Linear programming
9
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
IRIS H.-R. JIANG
Duality (2/3)
¨ A new LP: finding multipliers that gives the best upper bound
on our original LP
¤ Primal LP
¤ Any feasible value of dual LP is an upper bound on primal LP
¤ If we find a pair of primal and dual feasible values that are equal,
they must be both optimal.
Linear programming
10
¤ Dual LP
IRIS H.-R. JIANG
Duality (3/3)
¨ Generic form:
¨ Dual theorem: If a linear program has a bounded optimum, then
so does its dual, and the new optimum values coincide.
¤ Max-flow min-cut
Linear programming
11
IRIS H.-R. JIANG
The Simplex Algorithm
Linear programming
12
Every constraint specifies an
n-dimensional half-space
Travel along “edges” until no
improvement can be made

11. IRIS H.-R. JIANG
Duality (1/3)
¨ Multipliers yi’s must be nonnegative
¨ If the left-hand side to look like our objective function, the right-
hand side is an upper bound on the optimum solution
¨ We want a tight bound!
Linear programming
9
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
IRIS H.-R. JIANG
Duality (2/3)
¨ A new LP: finding multipliers that gives the best upper bound
on our original LP
¤ Primal LP
¤ Any feasible value of dual LP is an upper bound on primal LP
¤ If we find a pair of primal and dual feasible values that are equal,
they must be both optimal.
Linear programming
10
¤ Dual LP
IRIS H.-R. JIANG
Duality (3/3)
¨ Generic form:
¨ Dual theorem: If a linear program has a bounded optimum, then
so does its dual, and the new optimum values coincide.
¤ Max-flow min-cut
Linear programming
11
IRIS H.-R. JIANG
The Simplex Algorithm
Linear programming
12
Every constraint specifies an
n-dimensional half-space
Travel along “edges” until no
improvement can be made

12. IRIS H.-R. JIANG
Duality (1/3)
¨ Multipliers yi’s must be nonnegative
¨ If the left-hand side to look like our objective function, the right-
hand side is an upper bound on the optimum solution
¨ We want a tight bound!
Linear programming
9
max x1 + 6x2
x1 <= 200 (1)
x2 <= 300 (2)
x1 + x2 <= 400 (3)
x1 , x2 >= 0.
IRIS H.-R. JIANG
Duality (2/3)
¨ A new LP: finding multipliers that gives the best upper bound
on our original LP
¤ Primal LP
¤ Any feasible value of dual LP is an upper bound on primal LP
¤ If we find a pair of primal and dual feasible values that are equal,
they must be both optimal.
Linear programming
10
¤ Dual LP
IRIS H.-R. JIANG
Duality (3/3)
¨ Generic form:
¨ Dual theorem: If a linear program has a bounded optimum, then
so does its dual, and the new optimum values coincide.
¤ Max-flow min-cut
Linear programming
11
IRIS H.-R. JIANG
The Simplex Algorithm
Linear programming
12
Every constraint specifies an
n-dimensional half-space
Travel along “edges” until no
improvement can be made

13. IRIS H.-R. JIANG
Vertex and Neighbors
¨ Pick a subset of the inequalities. If there is
a unique point that satisfies them with equality,
and this point happens to be feasible,
then it is a vertex
¤ {2, 3, 7} ® A
¤ {4, 6} ® no vertex
¨ Two vertices are neighbors if they have
n - 1 defining inequalities in common
¤ {2, 3, 7} ® A
¤ {1, 3, 7} ® C
13
Linear programming
IRIS H.-R. JIANG
The Simplex Algorithm
¨ On each iteration, simplex has two tasks:
¤ Task 1: Check whether the current vertex is optimal
¤ Task 2: Determine where to move next
¨ Both tasks are easy if the vertex happens to be at the origin
¤ Transform the coordinate system to move vertex u to the origin
¨ Task 1:
¤ The origin is optimal if and only if all ci <= 0
¨ Task 2:
¤ We can move by increasing some xi for which ci > 0
¤ Until we hit some other constraint
14
Linear programming
IRIS H.-R. JIANG
Example (1/3)
Linear programming
15
IRIS H.-R. JIANG
Example (2/3)
Linear programming
16

14. IRIS H.-R. JIANG
Vertex and Neighbors
¨ Pick a subset of the inequalities. If there is
a unique point that satisfies them with equality,
and this point happens to be feasible,
then it is a vertex
¤ {2, 3, 7} ® A
¤ {4, 6} ® no vertex
¨ Two vertices are neighbors if they have
n - 1 defining inequalities in common
¤ {2, 3, 7} ® A
¤ {1, 3, 7} ® C
13
Linear programming
IRIS H.-R. JIANG
The Simplex Algorithm
¨ On each iteration, simplex has two tasks:
¤ Task 1: Check whether the current vertex is optimal
¤ Task 2: Determine where to move next
¨ Both tasks are easy if the vertex happens to be at the origin
¤ Transform the coordinate system to move vertex u to the origin
¨ Task 1:
¤ The origin is optimal if and only if all ci <= 0
¨ Task 2:
¤ We can move by increasing some xi for which ci > 0
¤ Until we hit some other constraint
14
Linear programming
IRIS H.-R. JIANG
Example (1/3)
Linear programming
15
IRIS H.-R. JIANG
Example (2/3)
Linear programming
16

15. IRIS H.-R. JIANG
Vertex and Neighbors
¨ Pick a subset of the inequalities. If there is
a unique point that satisfies them with equality,
and this point happens to be feasible,
then it is a vertex
¤ {2, 3, 7} ® A
¤ {4, 6} ® no vertex
¨ Two vertices are neighbors if they have
n - 1 defining inequalities in common
¤ {2, 3, 7} ® A
¤ {1, 3, 7} ® C
13
Linear programming
IRIS H.-R. JIANG
The Simplex Algorithm
¨ On each iteration, simplex has two tasks:
¤ Task 1: Check whether the current vertex is optimal
¤ Task 2: Determine where to move next
¨ Both tasks are easy if the vertex happens to be at the origin
¤ Transform the coordinate system to move vertex u to the origin
¨ Task 1:
¤ The origin is optimal if and only if all ci <= 0
¨ Task 2:
¤ We can move by increasing some xi for which ci > 0
¤ Until we hit some other constraint
14
Linear programming
IRIS H.-R. JIANG
Example (1/3)
Linear programming
15
IRIS H.-R. JIANG
Example (2/3)
Linear programming
16

16. IRIS H.-R. JIANG
Vertex and Neighbors
¨ Pick a subset of the inequalities. If there is
a unique point that satisfies them with equality,
and this point happens to be feasible,
then it is a vertex
¤ {2, 3, 7} ® A
¤ {4, 6} ® no vertex
¨ Two vertices are neighbors if they have
n - 1 defining inequalities in common
¤ {2, 3, 7} ® A
¤ {1, 3, 7} ® C
13
Linear programming
IRIS H.-R. JIANG
The Simplex Algorithm
¨ On each iteration, simplex has two tasks:
¤ Task 1: Check whether the current vertex is optimal
¤ Task 2: Determine where to move next
¨ Both tasks are easy if the vertex happens to be at the origin
¤ Transform the coordinate system to move vertex u to the origin
¨ Task 1:
¤ The origin is optimal if and only if all ci <= 0
¨ Task 2:
¤ We can move by increasing some xi for which ci > 0
¤ Until we hit some other constraint
14
Linear programming
IRIS H.-R. JIANG
Example (1/3)
Linear programming
15
IRIS H.-R. JIANG
Example (2/3)
Linear programming
16

17. IRIS H.-R. JIANG
Example (3/3)
17
Linear programming
IRIS H.-R. JIANG
Standard Form
¨ Variants
¤ Either a maximization or a minimization problem
¤ Constraints can be equations and/or inequalities
¤ Variables are restricted to be nonnegative or unrestricted in sign
¨ Standard form
¤ Objective function: minimization
¤ Constraints: equations
¤ Variables: nonnegative
Linear programming
18
Slack variables

18. IRIS H.-R. JIANG
Example (3/3)
17
Linear programming
IRIS H.-R. JIANG
Standard Form
¨ Variants
¤ Either a maximization or a minimization problem
¤ Constraints can be equations and/or inequalities
¤ Variables are restricted to be nonnegative or unrestricted in sign
¨ Standard form
¤ Objective function: minimization
¤ Constraints: equations
¤ Variables: nonnegative
Linear programming
18
Slack variables