2. 356 K. Stamatelatou et al. / Control strategies for anaerobic digesters
Application of optimal control theory to fermentation • Acetate is the main constituent of the fatty acids.
processes has been an intriguing issue for numerous inves- • Acetate is the key organic compound fed to the digester,
tigators [5–7,13,16,17,19,21–24,29,32]. Many algorithms so that hydrogen utilizing methanogenic bacteria can be
and other methods have also been developed, in order to neglected. This assumption will be exact if acidogene-
overcome the difficulties entailed in applying optimal con- sis and methanogenesis occur separately in a two-stage
trol theory. Most researchers, however, have focused on anaerobic process [1,33]. In this case, the influent of
batch and semi-batch bioreactors. In an attempt to opti- the methanogenic digester can be regulated to consist
mize continuous throughput, as opposed to batch fermenta- mainly of acetate.
tion processes, a different and interesting approach has been • The biomass loss due to bacterial decay is insignificant
followed by D’Ans et al. [5] who applied Green’s theorem compared with the biomass loss due to the operating
to maximize bacterial growth during a transient state. conditions of the reactor (biomass in the effluent), so
A particular issue of importance when it comes to opti- that the endogenous decay term may be neglected [2].
mization of continuous processes, is the fact that the aris-
ing optimal control problem is singular because the control The above assumptions are necessary if we are to be able
variable, i.e., the dilution rate, is linearly included in the to formulate a singular optimal control problem with an el-
state equations [9,10]. Whenever a singular optimal control egant solution being possible. Deviations between real ap-
policy can be explicitly determined, it is rather impractical, plication and the hereby developed theoretical results may
since it usually involves a complicated function of the state be attributed to one or more of the above assumptions not
variables, often not readily measurable on-line. From this being valid.
aspect it is more useful to derive a suboptimal control law We distinguish two cases of disturbances affecting the
expressed in a simpler explicit form, in terms of quantities digester.
which may be measured on-line, and resulting in an imper-
(i) Disturbance caused by inhibitor intrusion to the system
ceptible loss of performance. Pullammanappallil et al. [29]
considered the case of inhibitors entering with the feed in Consider the case where a waste, to be treated anaerobi-
this perspective and presented a suboptimal solution of the cally, contains an inhibitory substance such as chloroform
problem, much simpler in form than the optimal one and or ammonia. Mass balances of biomass, substrate and in-
easy to implement. In this work, we thoroughly examine hibitor constitute the following model equations:
the conventional optimal control problem and its simpli-
dX
fied, suboptimal version when the normal operation of an = −DX + µX, (1)
anaerobic digester is upset by entry of an inhibitor, a feed dt
overload or underload. dS 1
= D(S0 − S) − µX, (2)
dt YX/S
2. Modeling for optimization dI
= D(I0 − I), (3)
dt
Application of optimal control theory requires a mathe- where X, S, I represent the methanogen, volatile fatty acid
matical model for the described process. Detailed modeling and inhibitor concentrations, respectively, D is the dilution
of anaerobic digestion has been the objective of many re- rate, and µ is the specific growth rate. The latter is a com-
searchers for whom their main concern has been a deeper plicated function of the state variables and is assumed to
understanding of the biochemical steps involved in the follow Andrews’ kinetics which predicts inhibition for high
anaerobic processes [3,4,11,20]. Despite the usefulness of substrate concentrations. This is the case for methanogenic
such models, their large dimensionality constitutes a seri- microorganisms since they are strongly inhibited by high
ous disadvantage in handling these models for application concentrations of what they metabolize, i.e., fatty acids:
purposes, especially when development of control schemes µmax
is involved. In such cases, the insight of the control law µ = µ(S) = . (4)
1 + Ks /S + S/Kip
the researcher gets while using a simplified model is more
important than the model itself. In this expression µmax is the maximum specific growth
This is also the case of the present work which necessi- rate, Ks is the saturation constant, and Kip is the substrate
tates the use of a simplified model based on the following inhibition constant.
assumptions: Given that the presence of an inhibitor affects only the
specific growth rate, µ, and the feed substrate concentra-
• All the feed substrate converts into organic acid rapidly, tion (S0 ) does not vary, stoichiometry may be used to
which permits us to neglect the dynamics of all the express volatile fatty acid concentration (S) in terms of
reaction steps except for the rate limiting growth of methanogen concentration, X:
methanogens and the associated production of methane
[27,30]. X = YX/S (S0 − S), (5)
• pH is assumed to be controlled to remain neutral. where YX/S is the yield constant.
3. K. Stamatelatou et al. / Control strategies for anaerobic digesters 357
In view of equation (5), the state variables and, conse- The total methane production over the interval [0, tf ] can
quently, the model equations may be reduced to only two, be expressed as
one for the methanogen concentration, X, or the substrate tf
concentration, S, and one for the inhibitor concentration, I. J D(t) = QCH4 (t) dt, (8)
As a consequence, equations (1) or (2) along with (3) are 0
adequate to describe the process. where QCH4 (t) is the methane production rate, D(t) is
The impact of an inhibitory substance upon anaerobic the dilution rate, and tf is the final time (chosen suffi-
digestion kinetics is expressed by a multiplying factor, i.e., ciently large, so that a new steady state is reached). The
f (I), which generally affects µmax or Ks of the specific methane production rate is assumed to be proportional to
growth rate [2]. The latter is modified as follows: the methanogen growth rate [27,30]
µmax f (I) QCH4 = V YCH4 /X µX, (9)
µ = µ(X, I) = S0 −X/YX/S
, (6)
Ks
1+ S0 −X/YX/S + Kip where V is the volume of the reactor, YCH4 /X is a yield
coefficient, µ is the specific growth rate, and X is the
where it is assumed that inhibition results simply in the methanogen concentration.
reduction of the maximum specific growth rate, i.e., µmax .
A variety of expressions have been suggested for f (I) [14],
among which two are considered in this work: 4. Optimization
f (I) = e−aI (7a) The Hamiltonian function for case (i), i.e., the presence
of an inhibitor in the feed, is
or
H = V YCH4 /X µ(X, I)X + λ1 −DX + µ(X, I)X
1
f (I) = . (7b) + λ2 D(I0 − I), (10)
1 + bI
Relationships (7a) and (7b) are known expressions which while for case (ii), i.e., the feed overload or underload, is
have been reported to account for substrate and/or product
H = V YCH4 /X µ(X, S)X + λ1 −DX + µ(X, S)X
inhibition [14]. Additionally, the latter is already known to
describe the noncompetitive inhibition of enzyme-catalyzed 1
+ λ2 D(S0 − S) − µ(X, S)X , (11)
reaction kinetics [8]. YX/S
where λ1 , λ2 are the co-state variables. The time deriva-
(ii) Disturbance caused by feed overload or feed tive of λi is defined for each case as the negative partial
underload derivative of Hamiltonian with respect to the state variables
(X or S).
In this case, the feed substrate concentration is subjected It is evident that we are dealing with a singular control
to a step change (increase or decrease, respectively), so problem since the Hamiltonian, in both cases, is linear with
that effluent biomass and substrate concentrations are not respect to the manipulated variable, D(t), and therefore for
related stoichiometrically at all times. The model equations all t the optimal D is either on a singular arc or on a bound
are restricted to (1) and (2) with the specific growth rate (either 0 or Dmax , depending on the initial conditions) until
simply given by (4). it reaches a singular arc and remains on that singular arc
until tf .
On the singular arc the following equations are valid:
3. Performance measure
HD = 0, (12)
dHD
The selection of an appropriate performance measure = 0, (13)
for an optimal control problem is the most important fac- dt
tor in an optimization problem definition. It enables us to d2 HD
= 0. (14)
get the most out of a transition state as it arises from the dt2
dynamic conditions prevailing in the digester. In anaero- Equations (12) and (13) are independent of D and may be
bic digestion, if one is primarily interested in the amount of used for expressing the co-state variables in terms of the
generated methane, the total methane production during the state variables, while from equation (14) an expression for
period of transition between two steady states constitutes an D can be derived, which is the optimal (feedback) control
appropriate performance measure to be maximized. In ad- policy on the singular arc (appendix B).
dition, it is an indication of the system robustness, since it In order to solve the optimal control problem, the point
monitors the methanogenic activity throughout the change. where the switching between the bound and the singular
What is more, it is based on the methane production rate arc occurs is of crucial importance. For this reason, we
which can be easily measured on-line. need to have an expression for the state variables which is
4. 358 K. Stamatelatou et al. / Control strategies for anaerobic digesters
valid only on the singular arc, so that while on the bound,
we will be able to check if we have reached the singular
arc. Such an expression can be derived by the following
argument:
Since the Hamiltonian is not an implicit function of time,
its first derivative with respect to time is zero:
˙
H = 0. (15)
As a consequence,
H = constant (16)
throughout the singular arc. It can be easily proved that the
new optimal steady state singular arc is unique, regardless
of tf , upon which the new optimal steady state lies (ap-
pendix C). Clearly, at the final time, tf , when the system
reaches a new optimal steady state corresponding to the
new feed conditions, the Hamiltonian will be equal to H ∗
(from (10) or (11) at the steady state), which is the value
of the constant in (16):
H ∗ = V YCH4 /X µ∗ X ∗ , (17)
Figure 1. Inhibitors entering with the feed: Optimal trajectory.
where the ∗ denotes the value of these quantities at the
new optimal steady state. In this way, equation (16) gives
an expression for the singular arc with respect to the state
variables.
It should be mentioned that the manipulated variable, D,
is constrained between two bounds – zero (equivalent to
batch operation) and Dmax , given by
Fmax
Dmax = , (18)
V
where Fmax is the maximum flowrate the feeding pump may
provide, and V is the volume of the reactor vessel. The
above constraint has an implication for the case in which
the first calculated value of the dilution rate on the singular
arc is higher than Dmax . In such a case, the optimal control
policy is a bang–bang control policy (sequential switching
between the upper and lower bounds) until the calculated
dilution rate on the singular arc lies within the limits.
5. Optimal and easily implementable suboptimal
control laws
An arithmetic example following the procedure de- Figure 2. Step change in the feed substrate concentration: Optimal trajec-
tory.
scribed above for each type of disturbance considered is
presented in this section. The values of the constants and
the other parameters of the problem are given in appen- leads the system optimally to the new optimal steady state,
dix A. The analytical forms of the basic relationships of by changing the dilution rate according to equations (B4)
the problem are presented in appendix B. or (B8).
Figures 1 and 2 present the transition phase plane for the A typical time evolution of the optimal dilution rate and
cases of the intrusion of an inhibitor, the effect of which on the corresponding methane production rate for the cases
the specific growth rate is given by (7a), and a disturbance examined is depicted in figures 3 and 4.
at the feed substrate concentration, respectively. Immedi- It should be mentioned that if instead of (7a) we mod-
ately following an imbalance, the digester operates either eled the effect of the inhibitors on the specific growth rate
as a batch reactor (D = 0) or as a CSTR at its maximum through (7b), we would obtain qualitatively similar results
capacity (D = Dmax ), until it reaches the singular arc that with those presented in figures 1 and 3.
5. K. Stamatelatou et al. / Control strategies for anaerobic digesters 359
Figure 3. Inhibitors entering with the feed: Optimal dilution rate and
methane production rate versus time.
Still practical problems arise when it comes to enforcing
the optimal control law, since it requires the knowledge of
the state variables which cannot easily be measured on-line.
As an alternative, a good suboptimal and easy to imple-
ment control law has been formulated.
Figures 3 and 4 indicate that almost for the entire interval
of optimization, the optimal trajectory lies on the singular
arc excluding a negligibly small initial interval when the
control variable is on a bound. The plot of the generated
methane production rate versus dilution rate while on the
singular arc is almost a straight line which passes through
the origin of the axes as can be seen from figures 5 and 6.
Thus, the suboptimal control policy that results is to sim-
ply change the dilution rate proportionally to the methane
production rate which can be readily measured on-line, i.e.,
D(t) = kQCH4 (t). (19)
Here k is the proportionality constant which can be deter-
mined by estimating the slope of the straight line of fig-
ures 5 and 6. A very good approximation of this optimal
value of k can be easily determined by the ratio of dilution Figure 4. Step change in the feed substrate concentration: (a) Optimal
dilution rate versus time; (b) optimal methane production rate versus time.
rate to methane production rate under optimal operating
conditions at the new steady state.
Equation (19) is not only easy to enforce but is also
Table 1
a very good suboptimal control law as indicated by the Performance measure values indicating the total methane produced while
comparison of the performance estimated for both control implementing the optimal or the suboptimal control law.
policies which have been implemented for each type of dis-
Kind of disturbance Performance measure, J
turbance. This comparison is presented in table 1, where (total methane, l)
the insignificant difference between the optimal and the sub-
optimal solution of the problem establishes the credibility Optimal Suboptimal
and value of the suboptimal control law. Inhibitor intrusion 302.19 302.17
In the examples with a step change in the feed substrate Overload 852.09 852.08
concentration presented here, the optimal dilution rate val- Underload 561.08 561.07
ues of the initial and the final steady state are almost iden-
6. 360 K. Stamatelatou et al. / Control strategies for anaerobic digesters
Figure 7. Feed substrate underload (from 1000 to 100 mg/l): Methane
Figure 5. Inhibitors entering with the feed: Methane production rate production rate versus dilution rate.
versus dilution rate.
here but also in a wide variety of situations. A feature of
the cases considered in this work, involves a permanent ef-
fect (step change) of a selected disturbance on the system
so that its operation has to be established at a new steady
state. Even if this disturbance lasted only for a finite pe-
riod of time (pulse change), implementation of (19) would
lead the digester to its original steady state in an almost
optimal way. The suboptimal control law given by (19)
has been incorporated in an expert system developed by
Pullammanappallil et al. [28] for stabilizing anaerobic di-
gesters. This work, therefore, really comes to justify that
particular choice made.
6. Conclusions
Operation of anaerobic digesters is sensitive to a variety
of disturbances, which may lead the digester to wash out.
An optimal control policy should be addressed to avoid the
impeding digester failure and restore its normal operation or
lead it to a new optimal steady state. This has been accom-
Figure 6. Step change in the feed substrate concentration: Methane pro-
duction rate versus dilution rate.
plished by using a simplified model of anaerobic digestion
to determine the optimal dilution rate as a function of time,
tical due to the fact that the initial and final feed substrate in response to the entry of an inhibitor with the feed or a
concentration values were specifically high. Nevertheless, sudden change in the feed substrate concentration. By ex-
even in the case this almost identity does not occur, the amining the essential features of the digester key operating
relationship between methane production rate and dilution variables, a simpler and easily implementable suboptimal
rate on the singular arc is still almost linear as can be seen control law was derived, according to which the dilution
in figure 7, and consequently the suboptimal control law rate should be changed proportionally to the methane pro-
can be used instead. duction rate, with the gain determined from the new optimal
The concept of this suboptimal control policy is not only steady state values. This suboptimal controller, with a wide
valid in the specific formulation of the problem presented field of enforcement, leads to almost optimal performance.
7. K. Stamatelatou et al. / Control strategies for anaerobic digesters 361
Appendix A Feed overload
Constant and parameter values Step change in the feed substrate concentration from 20
to 30 g/l;
a = 1 l/mg,
b = 1.77 l/mg, S0 = 30 000 mg/l.
Kip = 4 000 mg/l, Initial conditions:
Ks = 20 mg/l,
V = 5 l, S = 254.887 mg/l,
YX/S = 0.035 mg biomass/mg substrate, X = 691.079 mg/l.
YCH4 /X = 0.009 l methane/mg biomass,
µmax = 0.36 day−1 . Feed underload
Inhibitors entering with the feed Step change in the feed substrate concentration from 30
to 20 g/l;
I0 = 1 mg/l,
S0 = 20 000 mg/l.
S0 = 25 000 mg/l.
Initial conditions: Initial conditions:
I = 0 mg/l, S = 263.342 mg/l,
X = 856.905 mg/l. X = 1040.783 mg/l.
Appendix B. Optimal control law on the singular arc
(I) Inhibitors entering with the feed
(i) Costate variables:
µ(X, I)
λ1 = V YCH4 /X −1 , (B1)
∂µ(X,I)
∂I (I0 − I) − ∂µ(X,I)
∂X X
µ(X, I) X
λ2 = V YCH4 /X −1 . (B2)
∂µ(X,I)
∂I (I0 − I) − ∂µ(X,I)
∂X X
I0 − I
(ii) Equation of singular arc:
µ(X, I)2X
V YCH4 /X = constant. (B3)
∂µ(X,I)
∂I (I0 − I) − ∂µ(X,I) X
∂X
(iii) Dilution rate expression:
2
∂ 2 µ(X, I) ∂µ(X, I)
D= µ(X, I)2 − 2 µ(X, I) X 2
∂X 2 ∂X
∂µ(X, I) ∂µ(X, I) ∂ 2 µ(X, I) ∂µ(X, I)
+ 2 µ(X, I) − µ(X, I)2 (I0 − I)X + µ(X, I)2 (I0 − I)
∂X ∂I ∂X∂I ∂I
2
∂ 2 µ(X, I) ∂µ(X, I) ∂µ(X, I) ∂µ(X, I) ∂ 2 µ(X, I)
× µ(X, I) − 2 X2 + 4 −2 µ(X, I) (I0 − I)X
∂X 2 ∂X ∂X ∂I ∂X∂I
2 −1
∂ 2 µ(X, I) ∂µ(X, I)
+ µ(X, I) − 2 (I0 − I) 2
. (B4)
∂I 2 ∂I
8. 362 K. Stamatelatou et al. / Control strategies for anaerobic digesters
(II) Step change in the feed substrate concentration
(i) Costate variables:
µ(S) − ∂µ(S)
∂S (S0 − S)
λ1 = V YCH4 /X , (B5)
∂µ(S)
∂S (S0 − S) − X
YX/S
µ(S) − ∂µ(S)
∂S (S0 − S)
λ2 = V YCH4 /X X. (B6)
∂µ(S)
∂S (S0 − S) (S0 − S) − X
YX/S
(ii) Equation of singular arc:
µ(S)2
V YCH4 /X X = constant. (B7)
∂µ(S)
∂S (S0 − S)
(iii) Dilution rate expression:
2
∂µ(S) ∂ 2 µ(S) ∂µ(S) X ∂µ(S)
D= 2 µ(S)(S0 − S) − µ(S)2 (S0 − S) + µ(S)2 − µ(S)2 (S0 − S)
∂S ∂S 2 ∂S YX/S ∂S
2 −1
∂µ(S) ∂ 2 µ(S)
× 2 − µ(S) (S0 − S)2 . (B8)
∂S ∂S 2
Appendix C. Proof that the new optimal steady state Consequently, the optimal steady state is one for which
lies on the singular arc (C1)–(C3) are satisfied. On substitution of these expres-
sions into equations (12)–(14) it can be observed that they
This is shown for the case where an inhibitor intrudes are all satisfied. Moreover, the optimal steady state also sat-
into the digester. The same applies for the underload or ˙ ˙
isfies X = 0 and I = 0 and thus all optimality conditions
overload case. are met.
The state equations are:
X = −DX + µ(X, I)X,
˙ References
I = D(I0 − I).
˙
[1] A. Aivasidis, Proceedings of Conference on Industrial Wastewater
Treatment and Disposal, University of Patras, Greece (21–22 No-
At steady state, vember 1996) pp. 127–137.
[2] I.M. Alatiqi, A.A. Dadkhah and N.M. Jabr, Chem. Engrg. J. 43
D = µ(X, I), (C1) (1990) B81.
[3] J.F. Andrews and S.F. Graef, in: Anaerobic Biological Treatment
I = I0 . (C2) Processes, Advances in Chemistry Series, Vol. 105 (American
Chemical Society, Washington, 1971) p. 126.
For optimality we have: [4] I. Angelidaki, L. Ellegaard and B.K. Ahring, Biotech. Bioengrg. 42
(1993) 159.
dJSS dµ(X, I)X [5] G.D. Ans, P. Kokotovic and D. Cottlieb, IEEE Trans. Automat. Con-
= V YCH4 /X =0
dD dD trol 16 (1971) 341.
[6] G.D. Ans, P. Kokotovic and D. Cottlieb, Journal of Optimization
∂µ(X, I)X ∂µ(X, I)X dI
⇒ + Theory and Applications 7 (1971) 61.
∂D ∂I dD [7] G.D. Ans, P. Kokotovic and D. Cottlieb, Automatica 8 (1972) 729.
∂µ(X, I)X dX [8] J.E. Bailey and D.F. Ollis, Biochemical Engineering Fundamentals
+ = 0. (McGraw Hill, Singapore, 1986).
∂X dD [9] D.J. Bell and D.H. Jacobson, in: Singular Optimal Control Prob-
In the above equation lems, Mathematics in Science and Engineering, Vol. 117 (Academic
Press, 1975).
∂µ(X, I)X [10] A.E. Bryson and Y.J. Ho, Applied Optimal Control (Halsted Press,
= 0, Willey, 1975).
∂D
[11] D.J. Costello, P.F. Greenfield and P.L. Lee, Water Res. 25 (1991)
and at steady state I is independent of D, i.e., dI/dD = 0. 847.
Since dX/dD = 0, [12] D. Dochain, G. Bastin, A. Rozzi and A. Pauss, in: Adaptive Estima-
tion and Control of Biotechnological Processes, eds. S.L. Dhah and
∂µ(X, I)X G. Dumont, Adaptive Control Strategies for Industrial Use (Springer,
= 0. (C3) Berlin, 1989).
∂X
9. K. Stamatelatou et al. / Control strategies for anaerobic digesters 363
[13] R. Guthke and W.A. Knorre, Biotech. Bioengrg. 23 (1981) 2771. Water Quality Management Library, Vol. 7 (Technomic Publishing
[14] K. Han and O. Levenspiel, Biotech. Bioengrg. 32 (1988) 430. Company, 1992).
[15] J. Harmon, P. Pullammanappallil, G. Lyberatos, S.A. Svoronos and [26] F.M. Podruzny and L. van den Berg, Biotech. Bioengrg. 26 (1984)
D.P. Chynoweth, Proc. Amer. Control Conf. 2 (1990) 2156. 392.
[16] O. Hiromu, E. Nakanishi and T. Takamatsu, Biotech. Bioengrg. 18 [27] M. Polihronakis, L. Petrou and A. Deligiannis, Comp. Chem. Engrg.
(1976) 847. 17 (1993) 1167.
[17] L.Y. Ho and A.E. Humphrey, Biotech. Bioengrg. 12 (1970) 291. [28] P. Pullammanappallil, J. Harmon, D.P. Chynoweth, G. Lyberatos and
[18] I.S. Kim, J.C. Young and H.H. Tabak, Water Environ. Res. 66 (1994) S.A. Svoronos, Appl. Biochem. Biotech. 28/29 (1991) 33.
119. [29] P. Pullammanappallil, S.A. Svoronos and G. Lyberatos, Proc. ACC
[19] L.C. Liu, G.J. Prokopakis and J.A. Asenjo, in: 7th ALCHE Annual (1991) 1341.
Meeting, Florida (1986). [30] P. Renard, D. Dochain, G. Bastin, H. Naveau and E.J. Nyns, Biotech.
[20] P.L. McCarty and F.E. Mosey, Water Sci. Technol. 24 (1991) 17. Bioengrg. 31 (1988) 287.
[21] J.M. Modak and H.C. Lim, Biotech. Bioengrg. 30 (1987) 528. [31] A. Rozzi, A.C. Di Pinto and A. Brunetti, Environ. Technol. Lett. 6
[22] J.M. Modak and H.C. Lim, Biotech. Bioengrg. 33 (1989) 11. (1985) 594.
[23] R. Mutharasan and D.R. Coughanowr, in: Singular Optimal Con- [32] J. Staniˇkis and D. Leviˇauskas, Biotech. Bioengrg. 25 (1983) 985.
s s
trol of Bioreactors, Proc. 194th National Meeting of the American [33] A.M. Viturtia, P.L. Luengo, F. Cecchi and J.M. Alvarez, Environ.
Society Division of Microbial and Biochemical Technology (1987). Technol. 16 (1995) 379.
[24] S.H. Park, S.B. Lee and R.D.Y. Ryu, Biotech. Bioengrg. 23 (1981) [34] T.N. Whitmore and D. Lloyd, Biotech. Lett. 8 (1986) 203.
1237. [35] T.N. Whitmore, D. Lloyd, G. Jones and T.N. Williams, Appl. Mi-
[25] J. Patterson, in: Design of Anaerobic Processes for the Treatment of crob. Bioengrg. 26 (1987) 383.
Industrial and Municipal Wastes, eds. J.F. Malina and F.G. Pohland,