Design and Analysis of Algorithms: NP-Completeness

Design and
Analysis of
Algorithms
NP-COMPLETENESS

 Instructor
Prof. Amrinder Arora
amrinder@gwu.edu
Please copy TA on emails
Please feel free to call as well

 Available for study sessions
Science and Engineering Hall
GWU
Algorithms NP-Completeness 2
LOGISTICS

Algorithms
Analysis
Asymptotic
NP-
Completeness
Design
D&C
Greedy
DP
Graph
B&B
Applications
WHERE WE ARE
 Done
 Done
 Done
 Done
 Done
 Done
 Starting now..

Now that we
have
finished
studying
algorithmic
techniques,
we are..
Algorithms 4NP-Completeness
ON TO ANALYZING
CLASSES OF PROBLEMS

“Any question can be made
immaterial by subsuming all its
answers under a common head.
The sovereign road to
indifference, whether to evils or
to goods, lies in the thought of
the higher genus.”
THE HIGHER GENUS
William James

 https://www.youtube.com/watch?v=YX40hbAHx3s
FIRST, A VIDEO

 P = { L | L is accepted by a deterministic Turing
Machine in polynomial time}
 The TM takes at most O(nc) steps to accept a string of length n
 NP = { L | L is accepted by a non-deterministic Turing
Machine in polynomial time }
 The TM takes at most O(nc) steps on each computation path to
accept a string of length n
 They are sets of languages
THE CLASS P AND THE CLASS NP

 A language is defined as a set of strings
 For example, if an alphabet is {a, b, c}, then a language can be:
{“a”, “ab”, “ac”}. Another language can be: {“ab”, “abab”,
“ababab”, “abababab”, “ababababab”, …}. Languages can be
finite or infinite.
 Problem is typically defined as when we are asked to
find/solve, or at least confirm something.
 For example, given a graph G does it have a Hamiltonian Cycle?
LANGUAGES VS. PROBLEMS

 A problem instance can also be encoded as a String.
For example, a graph G can be encoded where the
first line of the input file contains the number of
nodes n, then the edges, etc.
 If the algorithm (Turing Machine) accepts that input,
that input is in the language of the Turing Machine.
 All inputs that are valid graphs and have Hamiltonian
Cycles, can be accepted. All inputs that are either
malformed (not a graph) or don’t have Hamiltonian
cycle, can be rejected.
ENCODING A PROBLEM

 Thus, a language L is a mathematical way of
representing what we usually define as a problem.
LANGUAGES ARE PROBLEMS

 P = { L | L is accepted by a deterministic Turing
Machine in polynomial time}
 The TM takes at most O(nc) steps to accept a string of length n
 NP = { L | L is accepted by a non-deterministic Turing
Machine in polynomial time }
 The TM takes at most O(nc) steps on each computation path to
accept a string of length n
 They are sets of languages
 Or, as we now know, P and NP are classes (sets) of
problems.
THE CLASS P AND THE CLASS NP

TURING MACHINE REFRESHER
. . .
Infinite tape: Γ*
Tape head: read current square on tape,
write into current square,
move one square left or right
FSM: like PDA, except:
transitions also include direction (left/right)
final accepting and rejecting states
FSM

 Are non-deterministic Turing machines really more
powerful (efficient) than deterministic ones?
 Essence of P vs. NP problem
 Does non-determinism help?
 In case of automata – there is no difference
 In case of PDA – yes, there is a difference
 In case of TM, we do not know
D TM VS. N TM

 One of the most important unanswered questions since
1960s
 Many optimization problems appear amenable only to brute force,
i.e. (near-)exhaustive enumeration.
 Edmonds 1966: “The classes of problems which are respectively
known and not known to have good algorithms are of great
theoretical interest […] I conjecture that there is no good
algorithm for the traveling salesman problem. My reasons are the
same as for any mathematical conjecture: (1) It’s a legitimate
mathematical possibility, and (2) I do not know.”
 How can we make progress on this problem?
 We could find an algorithm for every NP problem
 We could use polynomial time reductions to find the “hardest”
problems and just work on those
P = NP?

Input for
Problem B
Output for
Problem B
Algorithm for Problem B
Reduction
from
B to A
Algorithm
for A
x R(x)
Yes/No
REDUCIBILITY

 How would you define NP-Complete?
 They are the “hardest” problems in NP
NP-COMPLETENESS
P
NP
NP-Complete

Q is an NP-Complete problem if:
1) Q is in NP
2) Every other NP problem polynomial time reducible
to Q
DEFINITION OF NP-COMPLETE

 Satisfiability problem: Given a boolean formula, find whether it
has a satisfying assignment or not.
 For example, say the formula is:
(x1 or x2 or x3) and (x1 or n(x2) or n(x3)) and
(n(x1) or x2 or n(x3)) and (n(x1) or n(x2) or n(x3)) and
(n(x1) or x2 or x3) and (x1 or n(x2) or x3) and
(x1 or n(x2) or n(x3)) and (n(x1) or x2 or n(x3))
 Here n(x1) represents the negation of x1. So, if x1 is true, then
n(x1) is false, and vice versa.
 So, if we assign x1 = x2 = x3 = true, then overall clause
becomes:
(T or T or T) and (T or F or F) and (F or T or F) and (F or F or F)
and
(F or T or T) and (T or F or T) and (T or F or F) and (F or T or F),
which becomes: T and T and T and F and T and T and T and T = F
= false.
 So, this assignment does not satisfy this clause.
 Some other true/false assignment to variables may satisfy this
clause, or it is possible that this clause is not satisfiable.
SAT

Save your job
Well, I can’t solve it, but so can’t thousands of
other computer scientists working for past 40
years on this problem X (even if we just invented
X)
Approximation
Chances of coming with an optimal solution to X
are slim, perhaps we can focus on approximate
solution instead?
Computability
X may really be impossible to solve in
polynomial time, and perhaps we need to
simplify it to solve it in reasonable amount of
time.
TOP 3 REASONS TO PROVE PROBLEM X IS
NP-COMPLETE

Input for
Problem B
Output for
Problem B
Reduction
from
B to A
Algorithm
for A
x R(x) Yes/No
 Algorithm for Problem B
REDUCIBILITY
Problem A is at least as hard as Problem B.

 Satisfiability problem is that given a boolean formula, we have to
find whether it has a satisfying assignment or not.
 For example, say the formula is:
(x1 or x2 or x3)
and (x1 or n(x2) or n(x3))
and (n(x1) or x2 or n(x3))
and (n(x1) or n(x2) or n(x3))
and (n(x1) or x2 or x3)
and (x1 or n(x2) or x3)
and (x1 or n(x2) or n(x3))
and (n(x1) or x2 or n(x3))
 Here n(x1) represents the negation of x1. So, if x1 is true, then
n(x1) is false, and vice versa.
SAT

22
Suppose we are given a NTM N and a string w of length
n which is decided by N in f(n) or fewer
nondeterministic steps.
Then there is an explicit CNF formula f of length O
(f(n)3) which is satisfiable iff N accepts w.
In particular, when f(n) is a polynomial, f has
polynomial length in terms of n so that every language
in NP reduces to CSAT in polynomial time. Thus CSAT
is NP-hard.
Finally, as CSAT is in NP, CSAT is NP-complete.
Algorithms NP-Completeness
COOK - LEVIN
THEOREM

To show that X is NP-Complete:
1. Show that X is in NP, i.e., a polynomial time verifier
exists for X.
2. Pick a suitable known NP-complete problem, S (ex:
SAT)
3. Show a polynomial algorithm to transform an
instance of S into an instance of X
SOX RESTOX mnemonic can help.
SAT Outside the Box
Reduce SAT To X
NP-COMPLETENESS PROOF METHOD

To show that X is NP-Complete:
1. Show that X is in NP, i.e., a polynomial time verifier
exists for X.
2. Pick a suitable known NP-complete problem, S (ex:
SAT)
3. Show a polynomial algorithm to transform an
instance of S into an instance of X
a) Draw the boxes, including inputs and outputs
b) Write relationship between the inputs
c) Describe the transformation
NP-COMPLETENESS PROOF METHOD (CONT.)

Input for
Problem B
Output for
Problem B
Reduction
from
B to A
Algorithm
for A
x R(x) Yes/No
 Algorithm for Problem B
REDUCIBILITY
Problem A is at least as hard as Problem B.

 CLIQUE = { <G,k> | G is a graph with a clique of size
k }
 A clique is a subset of vertices that are all connected
 Why is CLIQUE in NP?
EXAMPLE: CLIQUE

Textbook Section 9.5.1
 Pick an instance of 3-SAT, Φ, with k clauses
 Make a vertex for each literal
 Connect each vertex to the literals in other clauses
that are not the negation
 Any k-clique in this graph corresponds to a satisfying
assignment
REDUCING 3-SAT TO CLIQUE

REDUCING 3-SAT TO CLIQUE (CONT.)

 INDEPENDENT SET = { <G,k> | where G has an
independent set of size k }
 An independent set is a set of vertices, such that
there are no edge between any two vertices in that
set (two people are “independent” if they do not
know each other).
 How can we reduce the clique problem to IS?
EXAMPLE: INDEPENDENT SET (IS)

 This is the dual problem!
 Complement of a graph: Same vertices, but
“reversed” edges – remove the ones that exist, and
add the ones that don’t.
CLIQUE TO INDEPENDENT SET

HAMILTONIAN PATH TO HAMILTONIAN CYCLE
 Given a graph G = (V,E), construct a graph G’ =
(V’,E’), such that:
 V’ = V union {z}
 E’ = E union {(z,v) | all v in V}
 G’ has one more vertex, and n more edges
 Claim:
 G has a Hamiltonian Path if and only if G’ has a Hamiltonian
Cycle

HAMILTONIAN CYCLE TO HAMILTONIAN PATH
 Given a graph G = (V,E), construct a graph G’ =
(V’,E’), such that:
 V’ = V union {z, w, x}
 E’ = E union new edges.
 Firstly, select a random edge {u,v} in E.
 {w,u}
 {x,v} for all v connected to u.
 {z,x}
[Thus, G’ has 3 more vertices, and a few more edges]
 Claim:
 G has a Hamiltonian Cycle if and only if G’ has a Hamiltonian
Path

 Algorithm for Independent Set
INDEPENDENT SET TO VERTEX COVER
Input for
Independent
Set
Output for
Independent
Set
Transformation
G’ = G
Algorithm for
Vertex Cover
G,k G’,n-k Yes/No

 Show the following polynomial time reductions
 Independent Set P Vertex Cover
 Vertex Cover P Dominating Set
 3-SAT P Vertex Cover
 Hamiltonian Cycle P Hamiltonian Path
PRACTICE NP-COMPLETENESS REDUCTIONS

 NP complete problems are the HARDEST problems in
NP
 Reducibility – Art of reducing one problem to another
 Any problem in NP can be reduced to any NP-
complete problem in polynomial time.
 Is P = NP? This is one of the most fascinating
question in Computer Science today, with
phenomenal impacts if proven in affirmative.
 Reducing problems can be hard, takes practice
 SAT and 3-SAT are one of the earlier known NP-
complete problems, today there are thousands.
SUMMARY

 To prove a problem X is NP-complete, remember to
do two things:
 Show X is in NP
 Take a well known NP-complete problem, such as SAT, and
reduce SAT to X.
 SOX RESTOX mnemonic may help
SUMMARY

Design and Analysis of Algorithms: NP-Completeness

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (8)

Similar a Design and Analysis of Algorithms: NP-Completeness

Similar a Design and Analysis of Algorithms: NP-Completeness (20)

Más de Amrinder Arora

Más de Amrinder Arora (20)

Último

Último (20)

Design and Analysis of Algorithms: NP-Completeness