Presented in this document is a short discussion on using IMPL’s SLPQPE algorithm to solve process optimization problems in either off- or on-line environments also known as real-time optimization (RTO). Process optimization is somewhat different than production optimization in the sense that there are more “constitutive relations” involving only intensive variables. Both types of optimizations involve “conservation laws” and “correlative equations” which usually involve a mix of extensive and intensive variables (Kelly, 2004). Whereas production optimization deals more with material, meta-material (nonlinear), logic and logistics (discrete) balances (Zyngier and Kelly, 2009 and Kelly and Zyngier, 2015), process optimization is inherently more detailed and includes energy, exergy, momentum, hydraulics, equilibrium, diffusion, kinetics and other types of transport phenomena which involve nonlinear and perhaps discontinuous functions (Pantelides and Renfro, 2012).
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Impl slpqp ev-sqp
1. IMPL-SLPQPE versus SQP for Process Optimization
(IMPL-SLPQPEvSQP)
i n d u s t r IAL g o r i t h m s LLC. (IAL)
www.industrialgorithms.com
May 2014
Introduction
Presented in this document is a short discussion on using IMPL’s SLPQPE algorithm to solve
process optimization problems in either off- or on-line environments also known as real-time
optimization (RTO). Process optimization is somewhat different than production optimization in
the sense that there are more “constitutive relations” involving only intensive variables. Both
types of optimizations involve “conservation laws” and “correlative equations” which usually
involve a mix of extensive and intensive variables (Kelly, 2004). Whereas production
optimization deals more with material, meta-material (nonlinear), logic and logistics (discrete)
balances (Zyngier and Kelly, 2009 and Kelly and Zyngier, 2015), process optimization is
inherently more detailed and includes energy, exergy, momentum, hydraulics, equilibrium,
diffusion, kinetics and other types of transport phenomena which involve nonlinear and perhaps
discontinuous functions (Pantelides and Renfro, 2012).
The purpose of this short discussion is to highlight that IMPL’s SLPQPE algorithm, which is a
relatively standard implementation of successive linear programming (SLP) found in Palacios-
Gomez et. al. (1982) and Zhang et. al. (1985), can be used effectively and efficiently to solve
industrial-sized process industry optimization problems usually solved using successive
quadratic programming (SQP) algorithms such as IPOPT, CONOPT, filterSQP, KNITRO,
SNOPT, WORHP, etc.
Starting in the late 1980’s Shell Oil Company invested a tremendous amount of resources and
effort to develop and deploy real-time process optimization around the world but especially in
the United States and Canada. At the time SLP and a more primitive form referred to as
Method of Approximate Programming (MAP) (and later Distributive Recursion (DR)), had been
used successfully since the 1960’s for nonlinear planning (Griffith and Stewart, 1961) to perform
various types of applications such as feedstock selection, refinery process target-setting
(preliminary scheduling or operations programming) and product portfolio optimization and was
also being considered as the solver. To help determine which solver to use, Dr. Mike Morshedi
(previously with DMC Corportation, Dot Products Inc., PAS and Honeywell) devised a small test
problem described below to show that SQP was superior to SLP for process optimization given
that process models usually involve many terms with a variable times itself which leads to
diagonal elements in the Hessian i.e., the second-order derivatives matrix of the Lagrange
function (Renfro, 2010). During this period, even though SLP was popular for nonlinear
planning it was considered to be inferior to SQP given recent academic studies at the time.
The well-known disadvantage of SLP is that to converge on super-basic variables i.e., degrees-
of-freedom at the solution (variables that are neither basic nor non-basic), that many more SLP
major iterations are required. This is absolutely true if we try to solve a QP problem with an SLP
algorithm but our approach with SLPQPE is to call a QP sub-problem when quadratic variables
exist in the objective function which drastically reduces the number of major iterations. This is
especially effective for solving nonlinear data reconciliation and parameter estimation problems
or other hierarchical-types of optimization problems (Kelly and Zyngier, 2008a) where we need
2. to minimize deviations from targets or setpoints in a 2-norm sense such as advanced nonlinear
process control strategies and decomposed, layered, tiered or coordinated optimization
approaches.
However, a well-known advantage of SLP is that if the Hessian of the Lagrange function is
indefinite i.e., has both positive and negative eigenvalues (defined as its inertia), then this by
itself encourages vertex or LP types of solutions and as such requires less SLP major iterations
where there are smaller numbers of super-basic variables at the solution. This explains to some
degree why SQP’s applied to nonlinear planning problems with multiple time-periods have not
been successfully applied although has been attempted for example by Imperial Oil Ltd.
(ExxonMobil Canada) in the mid 1990’s but was later abandoned for a traditional SLP approach.
Another less known advantage of SLP is the fact that it can exploit the exceptional power of
commercial LP sub-solvers at each SLP major iteration with their powerful pre-processing
capability, advanced crash basis techniques, highly efficient linear algebra exploiting hyper-
sparsity, etc. SQP’s on the other hand use somewhat less efficient basis factorization packages
and unfortunately do not (and cannot) take advantage of the recent advances in LP technology.
Small Test Problem – See Appendices B and C
This small test problem was modified from a linear programming test by simply multiplying each
variable by itself as follows:
Maximize: F3 = -1*x1*x1 - 6*x2*x2 + 7*x3*x3 - x4*x4 - 5*x5*x5
Subject to: F1 = 5*x1*x1 – 4*x2*x2 + 13*x3*x3 – 2*x4*x4 + x5*x5 – 20 = 0
F2 = x1*x1 – x2*x2 + 5*x3*x3 – x4*x4 + x5*x5 – 8 = 0
0 <= x1, x2, x3, x4, x5 <= 10
This is a non-convex problem because it is nonlinear (bilinear) and has at least one constraint
with plus and minus nonlinear terms and has exhibited more than one local optima (-28.0 and
8.0) with the largest or global maximum equal to 8.57143 with the following solution:
x1 = 0.0
x2 = 0.75593
x3 = 1.30931
x4 = 0.0
x5 = 0.0
Supposedly this problem was solved with an early version of Shell’s SQP called OPERA and
was compared to MINOS where MINOS was assumed to be similar to SLP and failed to solve
this problem adequately (Renfro, 2010). This assumption on the surface seems reasonable and
was also made in Poku et. al. (2004) but MINOS and SLP are structurally different.
Unfortunately, a commercially available SLP code was not available until 2003 with Dash
Optimization’s (now FICO) Xpress-SLP (with startup funding partially provided by Honeywell)
and thus making a true comparison of SQP with SLP difficult.
If we revisit this small test problem using IPOPT as our SQP, it solves with an average of about
30 iterations from one hundred (100) randomized starting-points and finds the global optimum
100% of the time. Comparing to SLPQPE using an LP sub-solver, it solves with an average of
3. 10 major iterations and finds the global optimum only approximately 50% of the time. However,
if we modify the objective to be a minimization of an explicit quadratic function (see Appendix C)
then IPOPT still finds the global optimum 100% of the time but with an average of 90 iterations
while SLPQPE using a QP sub-solver finds the global optimum 93% of the time with an average
number of major iterations of 11.
Discussion
As presented above, SQP (IPOPT) is able to find the global optimum reliably for both
formulations of the small test problem which confirms that the SQP is appropriate for these
types of problems. SLPQPE using an LP sub-solver was only able to find the global optimum
50% of the time but it was able to find the solution in a reasonable number of major iterations.
Using an explicit quadratic objective function formulation, again IPOPT was able to find the
global optimum 100% of the time but with an increase in the number of its iterations whereas
SLPQPE using a QP sub-solver significantly improved its chances of finding the global optimum
from 50% to 93% with a slight increase in the number of major iterations.
Now a word with regard to the practical-ness of the small test problem in terms of being a
representative process engineering optimization type of problem. Although it has variable times
itself which as mentioned can be found in both constitutive relations and correlative equations
creating diagonal elements in the Hessian of the Lagrange function (which most likely yield
super-basic variables), this is not a true process engineering problem given that it was
generated from a simple LP test problem by arbitrarily squaring each variable in each
expression. Other types of constraints such as conservation laws (usually extensive times
intensive variables forming off-diagonal elements in the Hessian) are not properly represented
in this problem which makes a true generalization that SQP is better than SLP for process
optimization incomplete. In fact, when we modify the formulation to solve with a QP sub-solver
in SLPQPE, its performance is very strong compared to IPOPT for example.
Therefore, we can conclude that generally accepting that SQP is “better” for process
optimization is perhaps not as justified as we once thought and that SLP with an LP or QP sub-
solver is a strong competitor to SQP even for process-oriented and not just production-oriented
types of problems.
With IMPL we also believe that having access to many types of solvers and sub-solvers is
paramount to having robust, accurate and fast solutions to difficult process industry types of
problems whether they are planning, scheduling, control, data reconciliation, etc. and that is why
IMPL has bindings to all types of third-party nonlinear (and mixed-integer) solvers. And, given
IMPL’s small system-architecture footprint (SIIMPLE), it is very easy to implement a “poor man’s
parallelism” by running IMPL with different solvers/sub-solvers on as many computer processors
as available further increasing the reliability, precision and speed of finding good solutions to
industrial optimization problems with significant benefits (Kelly et. al. 2014).
In addition, IMPL allows for data reconciliation and regression problems to be solved using the
same model to perform either economic and/or efficiency-oriented optimizations. Coupled with
measurement and parameter feedback (Kelly and Zyngier, 2008b), whereby the “coefficients” in
the correlative equations are fit or estimated either using off-line or on-line data, affords the use
of simpler types of sub-models which can be considered as “hybrid” modeling i.e., combining
engineering and estimated empirical models together. An advantage of IMPL for this type of
parameter estimation is that IMPL computes not only the values but their variances for any size
4. and complexity of flowsheet enabling gross-errors, defects, faults, outliers, anomalies, etc. to be
detected and isolated effectively.
References
Griffith, R.E., Stewart, R.A., “A nonlinear programming technique for the optimization continuous
processing systems”, Management Science, 7, 379, (1961).
Palacios-Gomez, F., Lasdon, L., Engquist, M., “Nonlinear optimization by successive linear
programming”, Management Science, 28, 1106, (1982).
Zhang, J., Kim, N-H., Lasdon, L., “An improved successive linear programming algorithm”,
Management Science, 31, 1312-1331, (1985).
Poku, M.Y.B., Biegler, L.T., Kelly, J.D., “Nonlinear optimization with many degrees of freedom in
process engineering”, Industrial & Engineering Chemistry Research, 43, 6803-6812, (2004).
Kelly, J.D., "Formulating large-scale quantity-quality bilinear data reconciliation problems",
Computers & Chemical Engineering, 28, 357, (2004).
Kelly, J.D., Zyngier, D., "Hierarchical decomposition heuristic for scheduling: coordinated
reasoning for decentralized and distributed decision-making problems", Computers & Chemical
Engineering, 32, 2684, (2008a).
Kelly, J.D., Zyngier, D., "Continuously improve planning and scheduling models with parameter
feedback", FOCAPO 2008, July, (2008b).
Zyngier, D., Kelly, J.D., "Multi-product inventory logistics modeling in the process industries", In:
W. Chaovalitwonse, K.C. Furman and P.M. Pardalos, Eds., Optimization and Logistics
Challenges in the Enterprise", Springer, 61-95, (2009).
Renfro, J.G., personal communication, (2010).
Pantelides, C.C., Renfro, J.G., "The online use of first-principles in process operations: review,
current status & future trends", FOCAPO/CPC 2012, January, (2012).
Kelly, J.D., Menezes, B.C., Grossmann, I.E., “Distillation blending and cutpoint temperature
optimization using monotonic interpolation”, submitted to Industrial & Engineering Chemistry
Research, (2014).
Kelly, J.D., Zyngier, D., "Unit operation nonlinear modeling for planning and scheduling
applications", K.C. Furman et.al. (eds.), Optimization and Analytics in the Oil & Gas Industries,
(2015) (accepted).
Appendix A - IMPL and SIIMPLE
To implement the mathematical formulation of this and other systems, IAL offers a unique
approach and is incorporated into our Industrial Modeling Programming Language we call IMPL.
IMPL has its own modeling language called IML (short for Industrial Modeling Language) which
is a flat or text-file interface as well as a set of API's which can be called from any computer
programming language such as C, C++, Fortran, C#, VBA, Java (SWIG), Python (CTYPES)
5. and/or Julia (CCALL) called IPL (short for Industrial Programming Language) to both build the
model and to view the solution. Models can be a mix of linear, mixed-integer and nonlinear
variables and constraints and are solved using a combination of LP, QP, MILP and NLP solvers
such as COINMP, GLPK, LPSOLVE, SCIP, CPLEX, GUROBI, LINDO, XPRESS, CONOPT,
IPOPT, KNITRO and WORHP as well as our own implementation of SLP called SLPQPE
(Successive Linear & Quadratic Programming Engine) which is a very competitive alternative to
the other nonlinear solvers and embeds all available LP and QP solvers.
In addition and specific to DRR problems, we also have a special solver called SECQPE
standing for Sequential Equality-Constrained QP Engine which computes the least-squares
solution and a post-solver called SORVE standing for Supplemental Observability, Redundancy
and Variability Estimator to estimate the usual DRR statistics. SECQPE also includes a
Levenberg-Marquardt regularization method for nonlinear data regression problems and can be
presolved using SLPQPE i.e., SLPQPE warm-starts SECQPE. SORVE is run after the
SECQPE solver and also computes the well-known "maximum-power" gross-error statistics
(measurement and nodal/constraint tests) to help locate outliers, defects and/or faults i.e., mal-
functions in the measurement system and mis-specifications in the logging system.
The underlying system architecture of IMPL is called SIIMPLE (we hope literally) which is short
for Server, Interfacer (IML), Interacter (IPL), Modeler, Presolver Libraries and Executable. The
Server, Presolver and Executable are primarily model or problem-independent whereas the
Interfacer, Interacter and Modeler are typically domain-specific i.e., model or problem-
dependent. Fortunately, for most industrial planning, scheduling, optimization, control and
monitoring problems found in the process industries, IMPL's standard Interfacer, Interacter and
Modeler are well-suited and comprehensive to model the most difficult of production and
process complexities allowing for the formulations of straightforward coefficient equations,
ubiquitous conservation laws, rigorous constitutive relations, empirical correlative expressions
and other necessary side constraints.
User, custom, adhoc or external constraints can be augmented or appended to IMPL when
necessary in several ways. For MILP or logistics problems we offer user-defined constraints
configurable from the IML file or the IPL code where the variables and constraints are
referenced using unit-operation-port-state names and the quantity-logic variable types. It is also
possible to import a foreign *.ILP file (row-based MPS file) which can be generated by any
algebraic modeling language or matrix generator. This file is read just prior to generating the
matrix and before exporting to the LP, QP or MILP solver. For NLP or quality problems we offer
user-defined formula configuration in the IML file and single-value and multi-value function
blocks writable in C, C++ or Fortran. The nonlinear formulas may include intrinsic functions
such as EXP, LN, LOG, SIN, COS, TAN, MIN, MAX, IF, NOT, EQ, NE, LE, LT, GE, GT and CIP,
LIP, SIP and KIP (constant, linear and monotonic spline interpolations) as well as user-written
extrinsic functions (XFCN). It is also possible to import another type of foreign file called the
*.INL file where both linear and nonlinear constraints can be added easily using new or existing
IMPL variables.
Appendix B – SQPtestRTO1.IML File
i M P l (c)
Copyright and Property of i n d u s t r I A L g o r i t h m s LLC.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Calculation Data (Parameters)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sCalc,@sValue
START,-1.0
6. BEGIN,0.0
END,1.0
PERIOD,1.0
&sCalc,@sValue
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Chronological Data (Periods)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
@rPastTHD,@rFutureTHD,@rTPD
START,END,PERIOD
@rPastTHD,@rFutureTHD,@rTPD
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Construction Data (Pointers)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sUnit,&sOperation,@sType,@sSubtype,@sUse
BLACKBOX,,processc,blackbox,
&sUnit,&sOperation,@sType,@sSubtype,@sUse
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Capacity Data (Prototypes)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sUnit,&sOperation,@rRate_Lower,@rRate_Upper
BLACKBOX,,0.0,0.0
&sUnit,&sOperation,@rRate_Lower,@rRate_Upper
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Condition Data (Properties)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sCondition
X1
X2
X3
X4
X5
F1
F2
F3
&sCondition
ConditionsUOCondition-&sUnit,&sOperation,&sCondition,@sType,@rValue,@sValue
BLACKBOX,,F1,?,3,5.0*X1*X1 - 4.0*X2*X2 + 13.0*X3*X3 - 2.0*X4*X4 + X5*X5 - 20.0
BLACKBOX,,F2,?,3,X1*X1 - X2*X2 + 5.0*X3*X3 - X4*X4 + X5*X5 - 8.0
BLACKBOX,,F3,?,3,-X1*X1 - 6.0* X2*X2 + 7.0*X3*X3 - X4*X4 - 5.0*X5*X5
ConditionsUOCondition-&sUnit,&sOperation,&sCondition,@sType,@rValue,@sValue
&sUnit,&sOperation,&sCondition,@rCondition_Lower,@rCondition_Upper,@rCondition_Target
BLACKBOX,,X1,0.0,10.0,
BLACKBOX,,X2,0.0,10.0,
BLACKBOX,,X3,0.0,10.0,
BLACKBOX,,X4,0.0,10.0,
BLACKBOX,,X5,0.0,10.0,
BLACKBOX,,F1,0.0,0.0,
BLACKBOX,,F2,0.0,0.0,
BLACKBOX,,F3,-100.0,100.0,
&sUnit,&sOperation,&sCondition,@rCondition_Lower,@rCondition_Upper,@rCondition_Target
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Cost Data (Pricing)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sUnit,&sOperation,&sCondition,@rConditionPro_Weight,@rConditionPer1_Weight,@rConditionPer2_Weight,@rConditionPen_Weight
BLACKBOX,,F3,1.0,,,
&sUnit,&sOperation,&sCondition,@rConditionPro_Weight,@rConditionPer1_Weight,@rConditionPer2_Weight,@rConditionPen_Weight
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Command Data (Future Provisos)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sUnit,&sOperation,@rSetup_Lower,@rSetup_Upper,@rBegin_Time,@rEnd_Time
BLACKBOX,,1,1,BEGIN,END
&sUnit,&sOperation,@rSetup_Lower,@rSetup_Upper,@rBegin_Time,@rEnd_Time
Appendix C – SQPtestRTO2.IML File
i M P l (c)
Copyright and Property of i n d u s t r I A L g o r i t h m s LLC.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Calculation Data (Parameters)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
&sCalc,@sValue
START,-1.0
BEGIN,0.0
END,1.0
PERIOD,1.0
&sCalc,@sValue