An Efficient Algorithm for Total Variation Denoising

1. Artyom Makovetskii
Department of Mathematics, Chelyabinsk State University,
Russian Federation
Sergei Voronin
Department of Mathematics, Chelyabinsk State University,
Russian Federation
Vitaly Kober
Division of Applied Physics, CICESE, Ensenada, Mexico
E-mails: artemmac@csu.ru, voron@csu.ru,
vkober@cicese.mx
An efficient algorithm for total variation denoising
AIST'2016

2. Introduction
L𝐞𝐭 𝑢0 be an observed signal that is distorted by additive
noise 𝑛:
𝒖 𝟎 = 𝒗 + 𝒏
Initial function 𝑣

3. Noisy function 𝑢0
One of the most known techniques for denosing of noisy
signals was proposed by Rudin and Osher. This is a
total variation (TV) regularization problem..

4. Let 𝑱 𝒖 be the following functional :
𝑱 𝒖 = ∥ 𝒖 − 𝒖 𝟎 ∥ 𝑳 𝟐
𝟐
+ 𝝀 𝑻𝑽 𝒖 ,
where ∥ 𝒖 − 𝒖 𝟎 ∥ 𝑳 𝟐
𝟐
is called a fidelity term and
𝝀𝑻𝑽(𝒖) is called a regularization term.
Consider the following variational problem:
𝒖∗ = 𝒂𝒓𝒈 𝒎𝒊𝒏 𝑱 𝒖 ,
𝒖 ∈ 𝑩𝑽 𝜴
where 𝒖∗ is extremal function for 𝑱 𝒖 .

5. Here we consider a one-dimensional TV (1D TV)
regularization problem in framework of Strong and
Chan. They considered the behavior of explicit solutions
of the 1D TV problem when the parameter 𝝀 in TV
problem is sufficiently small. In this paper, explicit
solutions of TV problem for any parameter 𝝀 are
analyzed.
Strong, D. M. and Chan, T. F, “ Edge-preserving and
scale-dependent properties of total variation
regularization, ” Inverse Problems 19, 165–187 (2003).

6. How to describe the exact solutions
of 1D TV regularization problem
Continuous case
TV( 𝒖) = | 𝒖 𝒙 | 𝒅𝒙
∥ 𝒖 − 𝒖 𝟎 ∥ 𝑳 𝟐
𝟐
= (𝒖 − 𝒖 𝟎) 𝟐 𝒅𝒙
Discrete case
TV( 𝒖) = 𝒊=𝟏
𝒏−𝟏
|𝒖𝒊+𝟏 − 𝒖𝒊|
∥ 𝒖 − 𝒖 𝟎 ∥ 𝑳 𝟐
𝟐
= 𝒊=𝟏
𝒏
(𝒖𝒊 − 𝒖𝒊
𝟎
) 𝟐
where 𝒖 = (𝒖 𝟏, … , 𝒖 𝒏)
𝜴
𝜴

7. From BV class to 𝑩𝑽 𝟎 class
Let the function 𝒖 belongs to the 𝐁𝐕 class and 𝒖
has a discontinuity in the point 𝒙 𝒅:
𝑪± = 𝒍𝒊𝒎
𝒙→𝒙 𝒅±𝟎
𝒖(𝒙).
If the value 𝐮(𝒙 𝒅) is such that
𝒖 𝒙 𝒅 ∉ [𝑪−; 𝑪+] ,
then by replacing 𝒖(𝒙 𝒅) by either 𝑪− or 𝑪+ we obtain a
function with the same value of the fidelity term and a
smaller value of the regularization term.

8. The solution 𝒖∗ in the BV space to be found in
the space 𝑩𝑽 𝟎 of bounded variation functions
with the condition 𝒖 𝒙 𝒅 ∈ [𝑪−; 𝑪+] for each
discontinuity point 𝒙 𝒅. Thus, the solution in the
class BV is reduced to the solution of the problem
in the class 𝑩𝑽 𝟎.

9. Piecewise constant functions of
continuous argument
Let 𝒂 = 𝒓 𝟎 < 𝒓 𝟏 < ⋯ < 𝒓 𝒏 = 𝒃
be a partition of the segment [𝒂; 𝒃], ∆𝒊 be a
segment [𝒓𝒊−𝟏; 𝒓𝒊], 𝒊 = 𝟏, … , 𝒏. Let the partition
𝜹 = {∆𝒊} be a set of such segments.
The function 𝒖 is called piecewise constant,
i.e. 𝒖 ∈ 𝑷𝑪(𝜴), if 𝒖 𝒙 = 𝒖𝒊 when 𝒙 ∈ ∆𝒊.
Let 𝑷𝑪(𝜹) be a class of piecewise constant function
on the segment 𝜴 = [𝒂; 𝒃] with the partition 𝜹.

10. Suppose that 𝒖 𝟎 ∈ 𝑷𝑪(𝜹), then there is a
function 𝒖 ∈ 𝑷𝑪(𝜹) such that 𝑱 𝒖 ≤ 𝑱 𝒖 for
any 𝒖 ∈ 𝑩𝑽 𝟎.
Theorem 1

11. FORMULATION OF 1D TV
REGULARIZATION AS FINITE-
DIMENSIONAL PROBLEM
By using Theorem 1, the TV problem in the space
𝑩𝑽 𝟎 can be reduced to the problem in the space
𝑷𝑪(𝜹). Let a function 𝒖 belongs to 𝑷𝑪(𝜹). It means
that 𝒖 𝒙 = 𝒖𝒊
, 𝒙 ∈ ∆𝒊, 𝒊 = 𝟏, … , 𝒏 , i.e. 𝒖 𝒙 =
{𝒖 𝟏, … , 𝒖 𝒏}. Therefore, the functional space 𝑷𝑪(𝜹)
can be identified with the space ℝ 𝒏
. For functions
from 𝑷𝑪(𝜹), 𝑱 𝒖 takes the following form:
𝑱 𝒖 =
𝒊=𝟏
𝒏
(𝒖𝒊 − 𝒖 𝟎
𝒊
) 𝟐 |∆𝒊| + 𝛌 𝒊=𝟏
𝒏−𝟏
| 𝒖𝒊+𝟏 − 𝒖𝒊|.

12. Suppose that the function 𝒖 𝒙 = {𝒖 𝟏, … , 𝒖 𝒏} satisfies
the following condition:
𝒖𝒊 ≠ 𝒖𝒊+𝟏, 𝒊 = 𝟏, … , 𝒏 − 𝟏.
Let a vector 𝜺(𝒖 ) = 𝜺 𝟏, … , 𝜺 𝒏−𝟏 for the function
𝒖 𝒙 = {𝒖 𝟏, … , 𝒖 𝒏} be defined as follows:
𝜺𝒊
=
𝟏, 𝒖𝒊
< 𝒖𝒊+𝟏
−𝟏, 𝒖𝒊
> 𝒖𝒊+𝟏
.
Then total variation of the function 𝒖 𝒙 can be
expressed with the vector 𝜺 𝒖 ,
𝑻𝑽 𝒖 = 𝒊=𝟏
𝒏−𝟏
| 𝒖𝒊+𝟏 − 𝒖𝒊| = 𝒊=𝟏
𝒏−𝟏
𝜺𝒊( 𝒖𝒊+𝟏 − 𝒖𝒊).

13. One can observe that each component 𝒖𝒊 of the vector
𝒖 𝒙 is included in 𝑻𝑽 𝒖 with the coefficient 𝒆𝒊 =
+ 𝟏( 𝒆𝒊
= −𝟏), if 𝒖𝒊
is boundary maximum (boundary
minimum); 𝒆𝒊 = +𝟐 𝒆𝒊 = −𝟐 , if 𝒖𝒊 is local maximum
(local minimum) and 𝒆𝒊
= 𝟎, if 𝒖𝒊
is a “step” region of the
function 𝒖 𝒙 ,
𝑻𝑽 𝒖 = 𝒊=𝟏
𝒏
𝒆𝒊 𝒖𝒊,
where the vector 𝒆(𝒖) = 𝒆 𝟏
, … , 𝒆 𝒏
.
Let 𝒆 𝟎 be a vector of the initial function 𝒖 𝟎, 𝒖∗ be a
solution, 𝒆∗ be a vector of the function 𝒖∗.

14. Strong-Chan formulas for small 𝝀
Coefficient 𝒌𝒊 = ±
𝟏
𝟐
if the segment is boundary
maximum or boundary minimum, 𝒌𝒊 = −𝟏 if the
segment is local minimum, 𝒌𝒊 = 𝟎 if the segment is
the step, 𝒌𝒊 = 𝟏 if the segment is local maximum.
𝒉_𝒏𝒆𝒘 𝒊 = 𝒉 𝒊 − 𝒌𝒊
𝝀
∆𝒊

15. Theorem 2.
If 𝒆 𝟎
𝒊
≥ 𝟎 (𝒆 𝟎
𝒊
≤ 𝟎), then 𝒆∗
𝒊
≥ 𝟎 (𝒆∗
𝒊
≤ 𝟎), 𝒊 =
𝟏, … , 𝒏.

16. REDUCTION OF 1D TV PROBLEM BY 𝝀
Let 𝜹 𝟎 be a partition of the segment [𝒂; 𝒃] generated by
the function 𝒖 𝟎. Let a function 𝒖 𝟏 be a solution of the
following variational problem:
𝒂𝒓𝒈 𝒎𝒊𝒏 𝒊=𝟏
𝒏
(𝒖𝒊 − 𝒖 𝟎
𝒊
) 𝟐 |∆𝒊| + 𝝀 𝟏 𝒊=𝟏
𝒏−𝟏
| 𝒖𝒊+𝟏 − 𝒖𝒊|,
𝐮 ∈ 𝐏𝐂(𝜹 𝟎) (1)
where 𝒏 is the number of segments in the partition 𝜹 𝟎
and 𝝀 𝟏 is the value corresponding to the first meeting of
the two neighboring segments. Let 𝜹 𝟏 be a partition of
the segment [𝒂; 𝒃] generated by the function 𝒖 𝟏.

17. Let a function 𝒖 𝜺 be a solution of the following problem:
𝒂𝒓𝒈 𝒎𝒊𝒏 𝒊=𝟏
𝒌
(𝒖𝒊 − 𝒖 𝟏
𝒊
) 𝟐 |∆𝒊| + 𝝀 𝜺 𝒊=𝟏
𝒌−𝟏
| 𝒖𝒊+𝟏 − 𝒖𝒊|.
𝐮 ∈ 𝐏𝐂(𝜹 𝟏) (2)
where 𝒌 is the number of segments in the partition 𝜹 𝟏
and 𝝀 𝜺 is sufficiently small.

18. Consider the following variational problem:
𝒂𝒓𝒈 𝒎𝒊𝒏 𝒊=𝟏
𝒏
(𝒖𝒊 − 𝒖 𝟎
𝒊
) 𝟐 |∆𝒊| + (𝝀 𝟏 + 𝝀 𝜺) 𝒊=𝟏
𝒏−𝟏
| 𝒖𝒊+𝟏 − 𝒖𝒊|.
𝐮 ∈ 𝐏𝐂(𝜹 𝟎)
(3)
Theorem 3.
The solution 𝒖 𝜺 of the variational problem in (2) is
also a solution of (2).

19. EXPLICIT SOLUTIONS OF 1D TV
REGULARIZATION PROBLEM
From Theorems 1-3 follows that the exact solution of the
variational problem can be described by the following
way.
The parameter 𝝀 can be considered as time. When time 𝝀
is small the solution of (1) can be can be obtained by the
Strong and Chan formulas. Let 𝝀 𝟏 be the moment of
time corresponding to the first meeting of the two
neighboring segments. If value of parameter 𝝀 less than
𝝀 𝟏 we have description of the extremal function.

20. If value of parameter 𝝀 more than 𝝀 𝟏 do following.
Instead the initial variational problem consider
variational problem with substitution of obtained in
moment 𝝀 𝟏 function 𝒖 𝟏 instead 𝒖 𝟎 and with new
value of parameter 𝝀 ≔ 𝝀 − 𝝀 𝟏 . We obtain new
variational problem with smaller number of
segments and smaller value of parameter 𝝀.

21. 𝜆 = 4

22. 𝜆 = 50

23. 𝜆 = 1000

24. 𝜆 = 5000

25. Conclusion
In this paper we considered exact solutions of the 1D
TV problem. We proved the existence and uniqueness
of the solutions and described their properties.
Recently, was proposed explicit solutions of the 1DTV
problem as well as a direct fast algorithm for the case of
discrete functions. The algorithm is very fast and has
complexity of 𝑶(𝒏) for typical discrete functions. In
contrast, the proposed approach to finding exact
solutions has a clear geometrical meaning.