Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Próxima SlideShare
Cargando en…5
×

# Main

402 visualizaciones

Lion 11 Conference

• Full Name
Comment goes here.

Are you sure you want to Yes No
• Sé el primero en comentar

• Sé el primero en recomendar esto

### Main

1. 1. LION11 1 / 40 The use of grossone in optimization: a survey and some recent results R. De Leone School of Science and Technology Universit`a di Camerino June 2017
2. 2. Outline of the talk Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results LION11 2 / 40 Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results
3. 3. Single and Multi Objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 3 / 40
4. 4. Linear Programming and the Simplex Method Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 4 / 40 min x cT x subject to Ax = b x ≥ 0 The simplex method proposed by George Dantzig in 1947 ■ starts at a corner point (a Basic Feasible Solution, BFS) ■ veriﬁes if the current point is optimal ■ if not, moves along an edge to a new corner point until the optimal corner point is identiﬁed or it discovers that the problem has no solution.
5. 5. Preliminary results and notations Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 5 / 40 Let X = {x ∈ IRn : Ax = b, x ≥ 0} where A ∈ IRm×n, b ∈ IRm, m ≤ n. A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of A corresponding to positive components of ¯x are linearly independent.
6. 6. Preliminary results and notations Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 5 / 40 Let X = {x ∈ IRn : Ax = b, x ≥ 0} where A ∈ IRm×n, b ∈ IRm, m ≤ n. A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of A corresponding to positive components of ¯x are linearly independent. Let ¯x be a BFS and deﬁne ¯I = I(¯x) := {j : ¯xj > 0} then rank(A.¯I) = |¯I|. Note: |¯I| ≤ m
7. 7. Preliminary results and notations Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 5 / 40 Let X = {x ∈ IRn : Ax = b, x ≥ 0} where A ∈ IRm×n, b ∈ IRm, m ≤ n. A point ¯x ∈ X is a Basic Feasible Solution (BFS) iff the columns of A corresponding to positive components of ¯x are linearly independent. Let ¯x be a BFS and deﬁne ¯I = I(¯x) := {j : ¯xj > 0} then rank(A.¯I) = |¯I|. Note: |¯I| ≤ m Vertex Point, Extreme Points and Basic Feasible Solution Point coin- cide BFS ≡ Vertex ≡ Extreme Point
8. 8. BFS and associated basis Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 6 / 40 Assume now rank(A) = m ≤ n A base B is a subset of m linearly independent columns of A. B ⊆ {1, . . . , n} , det(A.B) = 0 N = {1, . . . , n} − B Let ¯x be a BFS. .
9. 9. BFS and associated basis Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 6 / 40 Assume now rank(A) = m ≤ n A base B is a subset of m linearly independent columns of A. B ⊆ {1, . . . , n} , det(A.B) = 0 N = {1, . . . , n} − B Let ¯x be a BFS. . If |{j : ¯xj > 0}| = m the BFS is said to be non–degenerate and there is only a single base B := {j : ¯xj > 0} associated to ¯x Non-degenerate BFS
10. 10. BFS and associated basis Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 6 / 40 Assume now rank(A) = m ≤ n A base B is a subset of m linearly independent columns of A. B ⊆ {1, . . . , n} , det(A.B) = 0 N = {1, . . . , n} − B Let ¯x be a BFS. . If |{j : ¯xj > 0}| < m the BFS is said to be degenerate and there are more than one base B1, B2, . . . , Bl associated to ¯x with {j : ¯xj > 0} ⊆ Bi Degenerate BFS
11. 11. BFS and associated basis Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 6 / 40 Assume now rank(A) = m ≤ n A base B is a subset of m linearly independent columns of A. B ⊆ {1, . . . , n} , det(A.B) = 0 N = {1, . . . , n} − B Let ¯x be a BFS. Let B a base associated to ¯x. Then ¯xN = 0, ¯xB = A−1 .B b ≥ 0
12. 12. BFS and associated basis Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 6 / 40 Assume now rank(A) = m ≤ n A base B is a subset of m linearly independent columns of A. B ⊆ {1, . . . , n} , det(A.B) = 0 N = {1, . . . , n} − B Let ¯x be a BFS. Let B a base associated to ¯x.
13. 13. Convergence of the Simplex Method Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 7 / 40 Convergence of the simplex method is ensured if all basis visited by the method are nondegenerate
14. 14. Convergence of the Simplex Method Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 7 / 40 Convergence of the simplex method is ensured if all basis visited by the method are nondegenerate In presence of degenerate BFS, the Simplex method may not terminate (cycling)
15. 15. Convergence of the Simplex Method Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 7 / 40 Convergence of the simplex method is ensured if all basis visited by the method are nondegenerate In presence of degenerate BFS, the Simplex method may not terminate (cycling) ⇓ Hence, speciﬁc anti-cycling procedures must be implemented (Bland’s rule, lexicographic order)
16. 16. Lexicographic Rule Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 8 / 40 At each iteration of the simplex method we choose the leaving variable using the lexicographic rule
17. 17. Lexicographic Rule Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 8 / 40 Let B0 be the initial base and N0 = {1, . . . , n} − B0. We can always assume, after columns reordering, that A has the form A = A.Bo ... A.No
18. 18. Lexicographic Rule Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 8 / 40 Let ¯ρ = min i: ¯Aijr >0 (A.−1 B b)i ¯Aijr if such minimum value is reached in only one index this is the leaving variable. OTHERWISE
19. 19. Lexicographic Rule Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 8 / 40 Among the indices i for which min i: ¯Aijr >0 (A.−1 B b)i ¯Aijr = ¯ρ we choose the index for which min i: ¯Aijr >0 (A.−1 B A.Bo)i1 ¯Aijr If the minimum is reached by only one index this is the leaving variable. OTHERWISE
20. 20. Lexicographic Rule Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 8 / 40 Among the indices reaching the minimum value, choose the index for which min i: ¯Aijr >0 (A.−1 B A.Bo)i2 ¯Aijr Proceed in the same way. This procedure will terminate providing a single index since the rows of the matrix (A.−1 B A.Bo) are linearly independent.
21. 21. Lexicographic rule and RHS perturbation Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 9 / 40 The procedure outlined in the previous slides is equivalent to perturb each component of the RHS vector b by a very small quantity. If this perturbation is small enough, the new linear programming problem is nondegerate and the simplex method produces exactly the same pivot sequence as the lexicographic pivot rule However, is very difﬁcult to determine how small this perturbation must be. More often a symbolic perturbation is used (with higher computational costs)
22. 22. Lexicographic rule and RHS perturbation and ① Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 10 / 40 Replace bi with bi with bi + j∈Bo Aij①−j .
23. 23. Lexicographic rule and RHS perturbation and ① Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 10 / 40 Replace bi with bi with bi + j∈Bo Aij①−j . Let e =      ①−1 ①−2 ... ①−m      and b = A.−1 B (b + A.Boe) = A.−1 B b + A.−1 B A.Boe.
24. 24. Lexicographic rule and RHS perturbation and ① Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 10 / 40 Replace bi with bi with bi + j∈Bo Aij①−j . Therefore b = (A.−1 B b)i + m k=1 (A.−1 B A.Bo)ik ①−k and min i: ¯Aijr >0 (A.−1 B b)i + m k=1 (A.−1 B A.Bo)ik ①−k ¯Aijr = min i: ¯Aijr >0 (A.−1 B b)i ¯Aijr + (A.−1 B A.Bo)i1 ¯Aijr ①−1 + . . . + (A.−1 B A.Bo)im ¯Aijr ①−m
25. 25. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 The set S := {Ax = b, x ≥ 0} is bounded and non-empty.
26. 26. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 Preemptive Scheme Starts considering the ﬁrst objective function alone: max x c1x subject to Ax = b x ≥ 0 Let x∗1 be an optimal solution and β1 = c1T x∗1.
27. 27. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 Preemptive Scheme Then solve max x c2T x subject to Ax = b c1T x = c1T x∗1 x ≥ 0
28. 28. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 Preemptive Scheme Repeat above schema until either the last problem is solved or an unique solution has been determined.
29. 29. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 Non–Preemptive Scheme
30. 30. Lexicographic multi-objective Linear Programming Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 11 / 40 lexmax x c1T x, c2T x, . . . , crT x subject to Ax = b x ≥ 0 Non–Preemptive Scheme There always exists a ﬁnite scalar MIR such that the solution of the above problem can be obtained by solving the one single-objective LP problem max x ˜cT x subject to Ax = b x ≥ 0 where ˜c = r i=1 M−i+1 ci .
31. 31. Non–Preemptive grossone-based scheme Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 12 / 40 Solve the LP max x ˜cT x subject to Ax = b x ≥ 0 where ˜c = r i=1 ①−i+1 ci Note that ˜cT x = c1T x ①0 + c2T x ①−1 + . . . crT x ①r−1
32. 32. Non–Preemptive grossone-based scheme Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 12 / 40 Solve the LP max x ˜cT x subject to Ax = b x ≥ 0 where ˜c = r i=1 ①−i+1 ci Note that ˜cT x = c1T x ①0 + c2T x ①−1 + . . . crT x ①r−1 The main advantage of this scheme is that it does not require the speciﬁcation of a real scalar value M
33. 33. Non–Preemptive grossone-based scheme Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 12 / 40 Solve the LP max x ˜cT x subject to Ax = b x ≥ 0 where ˜c = r i=1 ①−i+1 ci Note that ˜cT x = c1T x ①0 + c2T x ①−1 + . . . crT x ①r−1 M. Cococcioni, M. Pappalardo, Y.D. Sergeyev
34. 34. Theoretical results Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 13 / 40 max x ˜cT x subject to Ax = b x ≥ 0
35. 35. Theoretical results Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 13 / 40 max x ˜cT x subject to Ax = b x ≥ 0 If the LP above has solution, there is always a solution that a vertex. All optimal solutions of the lexicographic problem are feasible for the above problem and have the objective value. Any optimal solutions of the lexicographic problem is optimal for the above problem, and viceversa.
36. 36. Theoretical results Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 13 / 40 max x ˜cT x subject to Ax = b x ≥ 0 The dual problem is min π bT π subject to AT π ≤ ˜c If ¯x is feasible for the primal problem and ¯π feasible for the dual problem ˜cT ¯x ≤ bT ¯π
37. 37. Theoretical results Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 13 / 40 max x ˜cT x subject to Ax = b x ≥ 0 The dual problem is min π bT π subject to AT π ≤ ˜c If x∗ is feasible for the primal problem and π∗ feasible for the dual problem and ˜cT x∗ = bT π∗ x∗ is primal optimal and π∗ dual optimal.
38. 38. The gross-simplex Algorithm Outline of the talk Single and Multi Objective Linear Programming Linear Programming and the Simplex Method Preliminary results and notations BFS and associated basis Convergence of the Simplex Method Lexicographic Rule Lexicographic rule and RHS perturbation Lexicographic rule and RHS perturbation and ① Lexicographic multi-objective Linear Programming Non–Preemptive grossone-based scheme Theoretical results The gross-simplex Algorithm Nonlinear Optimization Some recent resultsLION11 14 / 40 Main issues: 1) Solve AT .Bπ = ˜cB Use LU decomposition of A.B. Note: no divisions by gross-number are required. 2) Calculate reduced cost vector ¯˜cN = ˜cN − AT .N π Also in this case only multiplications and additions of gross-numbers are required. ¯˜cN = 7.331 ①−1 + 0.331 ①−2 4 0 − 3.331 ①−1 − 0.33 ①−2 ¯˜cN = 3.67 ①−1 0.17 ①−2 4 ①0 + 0.33 ①−1 − 0.17 ①−2
39. 39. Nonlinear Optimization Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 15 / 40
40. 40. The case of Equality Constraints Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 16 / 40 min x f(x) subject to h(x) = 0 where f : IRn → IR and h : IRn → IRk L(x, π) := f(x) + k j=1 πjhj(x) = f(x) + πT h(x)
41. 41. Penalty Functions Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 17 / 40 A penalty function P : IRn → IR satisﬁes the following condition P(x) = 0 if x belongs to the feasible region > 0 otherwise
42. 42. Penalty Functions Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 17 / 40 A penalty function P : IRn → IR satisﬁes the following condition P(x) = 0 if x belongs to the feasible region > 0 otherwise P(x) = k j=1 |hj(x)| P(x) = k j=1 h2 j (x)
43. 43. Exactness of a Penalty Function Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 18 / 40 The optimal solution of the constrained problem min x f(x) subject to h(x) = 0 can be obtained by solving the following unconstrained minimization problem min f(x) + 1 σ P(x) for sufﬁciently small but ﬁxed σ > 0.
44. 44. Exactness of a Penalty Function Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 18 / 40 The optimal solution of the constrained problem min x f(x) subject to h(x) = 0 can be obtained by solving the following unconstrained minimization problem min f(x) + 1 σ P(x) for sufﬁciently small but ﬁxed σ > 0. P(x) = k j=1 |hj(x)|
45. 45. Exactness of a Penalty Function Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 18 / 40 The optimal solution of the constrained problem min x f(x) subject to h(x) = 0 can be obtained by solving the following unconstrained minimization problem min f(x) + 1 σ P(x) for sufﬁciently small but ﬁxed σ > 0. P(x) = k j=1 |hj(x)| Non–smooth function!
46. 46. Introducing ① Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 19 / 40 Let P(x) = k j=1 h2 j (x) Solve min f(x) + ①P(x) =: φ (x, ①)
47. 47. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 20 / 40 min x f(x) subject to h(x) = 0 (1)
48. 48. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 20 / 40 min x f(x) subject to h(x) = 0 (1) min x f(x) + 1 2 ① h(x) 2 (2)
49. 49. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 20 / 40 min x f(x) subject to h(x) = 0 (1) min x f(x) + 1 2 ① h(x) 2 (2) Let x∗ = x∗0 + ①−1 x∗1 + ①−2 x∗2 + . . . be a stationary point for (2) and assume that the LICQ condition holds at x∗0 then
50. 50. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 20 / 40 min x f(x) subject to h(x) = 0 (1) min x f(x) + 1 2 ① h(x) 2 (2) Let x∗ = x∗0 + ①−1 x∗1 + ①−2 x∗2 + . . . be a stationary point for (2) and assume that the LICQ condition holds at x∗0 then the pair x∗0, π∗ = h(1)(x∗) is a KKT point of (1).
51. 51. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 min x 1 2x2 1 + 1 6 x2 2 subject to x1 + x2 = 1 The pair (x∗, π∗) with x∗ =   1 4 3 4  , π∗ = −1 4 is a KKT point.
52. 52. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 min x 1 2x2 1 + 1 6 x2 2 subject to x1 + x2 = 1 The pair (x∗, π∗) with x∗ =   1 4 3 4  , π∗ = −1 4 is a KKT point. f(x) + ①P(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2
53. 53. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 f(x) + ①P(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 First Order Optimality Condition x1 + ①(x1 + x2 − 1) = 0 1 3 x2 + ①(x1 + x2 − 1) = 0
54. 54. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 f(x) + ①P(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 x∗ 1 = 1① 1 + 4① , x∗ 2 = 3① 1 + 4①
55. 55. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 f(x) + ①P(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 x∗ 1 = 1① 1 + 4① , x∗ 2 = 3① 1 + 4① x∗ 1 = 1 4 − ①−1 ( 1 16 − 1 64 ①−1 . . .) x∗ 2 = 3 4 − ①−1 ( 3 16 − 3 64 ①−1 . . .)
56. 56. Example 1 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 21 / 40 f(x) + ①P(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 x∗ 1 = 1① 1 + 4① , x∗ 2 = 3① 1 + 4① x∗ 1 + x∗ 2 − 1 = 1 4 − 1 16 ①−1 + 1 64 ①−2 . . . + 3 4 − 3 16 ①−1 + 3 64 ①−2 . . . − 1 = − 4 16 ①−1 − 3 16 ①−1 + 4 64 ①−2 . . . and h(1)(x∗) = −1 4 = π∗
57. 57. Example 2 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 22 / 40 min x1 + x2 subject to x2 1 + x2 2 − 2 = 0 L(x, π) = x1 + x2 + π x2 1 + x2 2 − 2 The optimal solution is x∗ = −1 −1 and the pair x∗, π∗ = 1 2 satisﬁes the KKT conditions.
58. 58. Example 2 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 22 / 40 φ (x, ①) = x1 + x2 + ① 2 x2 1 + x2 2 − 2 2 First–Order Optimality Conditions    x1 + 2①x1 x2 1 + x2 2 − 2 2 = 0 x2 + 2①x2 x2 1 + x2 2 − 2 2 = 0 The solution is given by    x1 = −1 − ①−1 1 8 + ①−2 C x2 = −1 − ①−1 1 8 + ①−2 C
59. 59. Example 2 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 22 / 40 Moreover x2 1 + x2 2 − 2 = 1 + 1 64 ①−2 + ①−4 C2 1 4 ①−1 − 2①−2 − 1 4 ①−3 C + 1 + 1 64 ①−2 + ①−4 C2 1 4 ①−1 − 2①−2 − 1 4 ①−3 C = 1 2 ①−1 + 1 32 − 4C ①−2 + − 1 2 C ①−3 + −2C2
60. 60. Inequality Constraints Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 23 / 40 min x f(x) subject to g(x) ≤ 0 h(x) = 0 where f : IRn → IR, g : IRn → IRm h : IRn → IRk. L(x, π, µ) := f(x) + m i=1 µigi(x) + k j=1 πjhj(x) = f(x) + µT g(x) + πT h(x)
61. 61. Modiﬁed LICQ condition Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 24 / 40 Let x0 ∈ IRn. The Modiﬁed LICQ (MLICQ) condition is said to hold at x0 if the vectors ∇gi(x0 ), i : gi(x0 ) ≥ 0, ∇hj(x0 ), j = 1, . . . , k are linearly independent.
62. 62. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 25 / 40 min x f(x) subject to g(x) ≤ 0 h(x) = 0 min x f(x) + ① 2 max{0, gi(x)} 2 + ① 2 h(x) 2 x∗ = x∗0 + ①−1 x∗1 + ①−2 x∗2 + . . . ⇓ (MLICQ)
63. 63. Convergence Results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 25 / 40 min x f(x) subject to g(x) ≤ 0 h(x) = 0 min x f(x) + ① 2 max{0, gi(x)} 2 + ① 2 h(x) 2 x∗ = x∗0 + ①−1 x∗1 + ①−2 x∗2 + . . . ⇓ (MLICQ) x∗0 , µ∗ = g(1) (x∗ ), π∗ = h(1) (x∗ )
64. 64. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 min x1 + x2 subject to x2 1 + x2 2 − 2 2 = 0 L(x, π) = x1 + x2 + π x2 1 + x2 2 − 2 2 The optimal solution is x∗ = −1 −1
65. 65. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 φ (x, ①) = x1 + x2 + ① 2 x2 1 + x2 2 − 2 4 First–Order Optimality Conditions    1 + 4①x1 x2 1 + x2 2 − 2 3 = 0 1 + 4①x2 x2 1 + x2 2 − 2 3 = 0
66. 66. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 Let the solution of the above system be x∗ 1 = x∗ 2 = A + B①−1 + C①−2 with A, B, and C ∈ IR. Now 4①x∗ 1 = 4A① + 4B + 4C①−1 and 1 + 4①x∗ 1 (x∗ 1)2 + (x∗ 2)2 − 2 3 = 1 + 4A① + 4B + 4C①−1 2A2 − 2 + 2AB①−1 + D①−2 3 .
67. 67. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 1 + 4①x∗ 1 (x∗ 1)2 + (x∗ 2)2 − 2 3 = 1 + 4A① + 4B + 4C①−1 2A2 − 2 + 2AB①−1 + D①−2 3 .
68. 68. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 1 + 4①x∗ 1 (x∗ 1)2 + (x∗ 2)2 − 2 3 = 1 + 4A① + 4B + 4C①−1 2A2 − 2 + 2AB①−1 + D①−2 3 . If 2A2 − 2 = 0 there is still a term of the order ① unless A = 0.
69. 69. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 1 + 4①x∗ 1 (x∗ 1)2 + (x∗ 2)2 − 2 3 = 1 + 4A① + 4B + 4C①−1 2A2 − 2 + 2AB①−1 + D①−2 3 . If 2A2 − 2 = 0 there is still a term of the order ① unless A = 0. If 2A2 − 2 = 0 a term ①−1 can be factored out 1 + 4A① + 4B + 4C①−1 ①−3 +2AB + D①−1 3 and the ﬁnite term cannot be equal to 0.
70. 70. The importance of CQs Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 26 / 40 1 + 4①x∗ 1 (x∗ 1)2 + (x∗ 2)2 − 2 3 = 1 + 4A① + 4B + 4C①−1 2A2 − 2 + 2AB①−1 + D①−2 3 . If 2A2 − 2 = 0 there is still a term of the order ① unless A = 0. If 2A2 − 2 = 0 a term ①−1 can be factored out 1 + 4A① + 4B + 4C①−1 ①−3 +2AB + D①−1 3 and the ﬁnite term cannot be equal to 0. When Constraint Qualiﬁcation conditions do not hold, the solution of ∇F(x) = 0 does not provide a KKT pair for the constrained problem.
71. 71. Conjugate Gradient Method Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 27 / 40 Data: Set k = 0, y0 = 0, r0 = b − Ay0. If r0 = 0, then STOP. Else, set p0 = r0. Step k: Compute αk = rT k pk/pT k Apk, yk+1 = yk + αkpk, rk+1 = rk − αkApk. If rk+1 = 0, then STOP. Else, set βk = −rT k+1Apk pT k Apk = rk+1 2 |rk 2 , and pk+1 = rk+1 + βkpk, k = k + 1. Go to Step k.
72. 72. pT k Apk Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 28 / 40 When the matrix A is positive deﬁnite then λm(A) pk 2 ≤ pT k Apk and pT k Apk is bounded from below. If the matrix A is not positive deﬁnite, then such a bound does not hold, being potentially pT k Apk = 0,.
73. 73. pT k Apk Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 28 / 40 When the matrix A is positive deﬁnite then λm(A) pk 2 ≤ pT k Apk and pT k Apk is bounded from below. If the matrix A is not positive deﬁnite, then such a bound does not hold, being potentially pT k Apk = 0,. R. De Leone, G. Fasano, Y.D. Sergeyev Use pT k Apk = s① where s = O(①−1 ) if the Step k is a non-degenerate CG step, and s = O(①−2 ) if the Step k is a degenerate CG step.
74. 74. Variable Metric Method for convex nonsmooth optimization Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 29 / 40 xk+1 = xk − αk Bk −1 ξk a where ξk a current aggregate subgradient, and the positive deﬁnite variable-metric n × n matrix, approximation of the Hessian matrix. Then Bk+1 = Bk + ∆k and Bk+1 δk ≈ γk with γk = gk+1 − gk (subgradients) and δk = xk+1 − xk and diagonal. The focus on the updating technique of matrix Bk
75. 75. Matrix Updating scheme Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 30 / 40 min b Bδk − γk subject to Bii ≥ ǫ Bij = 0, i = j
76. 76. Matrix Updating scheme Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 30 / 40 min b Bδk − γk subject to Bii ≥ ǫ Bij = 0, i = j Bk+1 ii = max ǫ, γk i δk 1
77. 77. Matrix Updating scheme Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 30 / 40 min b Bδk − γk subject to Bii ≥ ǫ Bij = 0, i = j M. Gaudioso, G. Giallombardo, M. Mukhametzhanov ¯γk i = γk i if |γk i | > ǫ ①−1 otherwise ¯δk i = δk i if |δk i | > ǫ ①−1 otherwise
78. 78. Matrix Updating scheme Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization The case of Equality Constraints Penalty Functions Exactness of a Penalty Function Introducing ① Convergence Results Example 1 Example 2 Inequality Constraints Modiﬁed LICQ condition Convergence Results The importance of CQs Conjugate Gradient Method pT k Apk Variable Metric Method for convex nonsmooth optimization Matrix Updating scheme LION11 30 / 40 min b Bδk − γk subject to Bii ≥ ǫ Bij = 0, i = j bk i =    ①−1 if 0 < ¯γk i ¯δk i ≤ ǫ ¯γk i ¯δk i otherwise Bk+1 ii = max ①−1 , bk i
79. 79. Some recent results Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 31 / 40
80. 80. Quadratic Problems Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 32 / 40 min x 1 2xT Mx subject to Ax = b x ≥ 0
81. 81. Quadratic Problems Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 32 / 40 min x 1 2xT Mx subject to Ax = b x ≥ 0 KKT conditions Mx + q − AT u − v = 0 Ax − b = 0 x ≥ 0, v ≥ 0, xT v = 0
82. 82. Quadratic Problems Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 32 / 40 min x 1 2xT Mx subject to Ax = b x ≥ 0 min 1 2 xT Mx + ① 2 Ax − b 2 2 + ① 2 max{0, −x} 2 2 =: F(x)
83. 83. Quadratic Problems Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 32 / 40 min x 1 2xT Mx subject to Ax = b x ≥ 0 min 1 2 xT Mx + ① 2 Ax − b 2 2 + ① 2 max{0, −x} 2 2 =: F(x) ∇F(x) = Mx + q + ①AT (Ax − b) − ① max{0, −x}
84. 84. Quadratic Problems Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 32 / 40 min x 1 2xT Mx subject to Ax = b x ≥ 0 min 1 2 xT Mx + ① 2 Ax − b 2 2 + ① 2 max{0, −x} 2 2 =: F(x) ∇F(x) = Mx + q + ①AT (Ax − b) − ① max{0, −x} x = x(0) + ①−1 x(1) + ①−2 x(2) + . . . b = b(0) + ①−1 b(1) + ①−2 b(2) + . . . A ∈ IRm×n rank(A) = m
85. 85. ∇F(x) = 0 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 33 / 40 0 = Mx + q + ①AT A x(0) + ①−1 x(1) + ①−2 x(2) + . . . −b(0) − ①−1 b(1) − ①−2 b(2) + . . . +① max 0, −x(0) − ①−1 x(1) − ①−2 x(2) + . . .
86. 86. ∇F(x) = 0 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 33 / 40 0 = Mx + q + ①AT A x(0) + ①−1 x(1) + ①−2 x(2) + . . . −b(0) − ①−1 b(1) − ①−2 b(2) + . . . +① max 0, −x(0) − ①−1 x(1) − ①−2 x(2) + . . . Looking at the ① terms Ax(0) − b(0) = 0 max 0, −x(0) = 0 and hence x(0) ≥ 0
87. 87. ∇F(x) = 0 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 33 / 40 0 = Mx + q + ①AT A x(0) + ①−1 x(1) + ①−2 x(2) + . . . −b(0) − ①−1 b(1) − ①−2 b(2) + . . . +① max 0, −x(0) − ①−1 x(1) − ①−2 x(2) + . . . Looking at the ①0 terms Mx(0) + q + AT Ax(1) − b(1) − v = 0 where vj = max 0, −x (1) j only for the indices j for which x (0) j = 0, otherwise vj = 0
88. 88. ∇F(x) = 0 Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 33 / 40 0 = Mx + q + ①AT A x(0) + ①−1 x(1) + ①−2 x(2) + . . . −b(0) − ①−1 b(1) − ①−2 b(2) + . . . +① max 0, −x(0) − ①−1 x(1) − ①−2 x(2) + . . . Set u = Ax(1) − b(1) vj = 0 if x (0) j = 0 max 0, −x (1) j otherwise Then Mx(0) + q + AT u − v = 0 v ≥ 0, vT x0 = 0
89. 89. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) f(x) = ①f(1) (x) + f(0) (x) + ①−1 f(−1) (x) + . . . ∇f(x) = ①∇f(1) (x) + ∇f(0) (x) + ①−1 ∇f(−1) (x) + . . .
90. 90. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) At iteration k
91. 91. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) At iteration k If ∇f(1) (xk ) = 0 and ∇f(0) (xk ) = 0 STOP
92. 92. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) At iteration k otherwise ﬁnd xk+1 such that
93. 93. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) At iteration k otherwise ﬁnd xk+1 such that If ∇f(1)(xk) = 0 f(1) (xk+1 ) ≤ f(1) (xk ) + σ ∇f(1) (xk ) f(0) (xk+1 ) ≤ max 0≤j≤lk f(0) (xk−j ) + σ ∇f(0) (xk )
94. 94. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) At iteration k otherwise ﬁnd xk+1 such that If ∇f(1)(xk) = 0 f(0) (xk+1 ) ≤ f(0) (xk ) + σ ∇f(0) (xk ) f(1) (xk+1 ) ≤ max 0≤j≤mk f(1) (xk−j )
95. 95. A Generic Algorithm Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 34 / 40 min x f(x) m0 = 0, mk+1 ≤ max {mk + 1, M} l0 = 0, kk+1 ≤ max {lk + 1, L} σ(.) is a forcing function. Non–monotone optimization technique, Zhang-Hager, Grippo- Lampariello-Lucidi, Dai
96. 96. Convergence Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 35 / 40 Case 1: ∃¯k such that ∇f(1)(xk) = 0, k ≥ ¯k Then f(1) (xk+1 ) ≤ max 0≤j≤mk f(1) (xk−j ), k ≥ ¯k and hence max 0≤i≤M f(1) (x ¯k+Ml+i ) ≤ max 0≤i≤M f(1) (x ¯k+M(l−1)+i ) and f(0) (xk+1 ) ≤ f(0) (xk ) + σ ∇f(0) (xk ) , k ≥ ¯k Assuming that the level sets for f(1)(x0) and f(0)(x0) are compact sets, then the sequence has at least one accumulation point x∗ and any accumulation point satisﬁes ∇f(1)(x∗) = 0 and ∇f(0)(x∗) = 0
97. 97. Convergence Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 35 / 40 Case 2: ∃ a subsequence jk such that ∇f(1)(xjk ) = 0 Then f(1) (xjk+1 ) ≤ f(1) (xjk ) + +σ ∇f(1) (xjk ) Again max 0≤i≤M f(1) (xjk+Mt+i ) ≤ max 0≤i≤M f(1) (xjk+M(t−1)+i )+σ ∇f(1) (xjk ) and hence ∇f(1)(xjk ) → 0.
98. 98. Convergence Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 35 / 40 Case 2: ∃ a subsequence jk such that ∇f(1)(xjk ) = 0 Then f(1) (xjk+1 ) ≤ f(1) (xjk ) + +σ ∇f(1) (xjk ) Again max 0≤i≤M f(1) (xjk+Mt+i ) ≤ max 0≤i≤M f(1) (xjk+M(t−1)+i )+σ ∇f(1) (xjk ) and hence ∇f(1)(xjk ) → 0. Moreover, max 0≤i≤L f(0) (xjk+Lt+i ) ≤ max 0≤i≤L f(0) (xjk+L(t−1)+i )+σ ∇f(0) (xjk ) and hence ∇f(0)(xjk ) → 0.
99. 99. Gradient Method Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 36 / 40 At iterations k calculate ∇f(xk). If ∇f(1)(xk) = 0 xk+1 = min α≥0,β≥0 f xk − α∇f(1) (xk ) − β∇f(0) (xk ) If ∇f(1)(xk) = 0 xk+1 = min α≥0 f(0) xk − α∇f(0) (xk )
100. 100. Example A Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 37 / 40 min x 1 2x2 1 + 1 6 x2 2 subject to x1 + x2 − 1 = 0 f(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 x0 = 4 1 → x1 = 0.31 0.69 → x2 = −0.1 0.39 → x3 = 0.26 0.74 → x4 = −0.12 0.38 → x5 = 0.25 0.75
101. 101. Example B Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 38 / 40 min x x1 + x2 subject to x1 1 + x2 2 − 2 = 0 f(x) = 1 2 x2 1 + 1 6 x2 2 + 1 2 ①(1 − x1 − x2)2 x0 = 0.25 0.75 → x1 = −1.22 −0.72 → x2 = −7.39 −6.89 → x3 = 1.04 0.95 x4 = −7.10 −7.19 → x5 = −1 −1
102. 102. Conclusions (?) Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 39 / 40 ■ The use of ① is extremely beneﬁcial in various aspects in Linear and Nonlinear Optimization ■ Difﬁcult problems in NLP can be approached in a simpler way using ① ■ A new convergence theory for standard algorithms (gradient, Newton’s, Quasi-Newton) needs to be developed in theis new framework
103. 103. Outline of the talk Single and Multi Objective Linear Programming Nonlinear Optimization Some recent results Quadratic Problems ∇F (x) = 0 A Generic Algorithm Convergence Gradient Method Example A Example B Conclusions (?) LION11 40 / 40 Thanks for your attention