SlideShare una empresa de Scribd logo
1 de 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Algorithmic Game Theory for Critical Infrastructure
Security and Resilience
Linan Huang Quanyan Zhu
Department of Electrical and Computer Engineering
New York University, USA
Game Solving: Theory and Practice, Prague, Czech Republic
Monday, July 9, 2018
July 9, 2018 1 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Critical Infrastructure
Presidential Policy Directive 21 (PPD-21) identifies 16 sectors.
Critical infrastructure sectors must be secure and resilient from all natural
hazards and human attacks.
Source: http://www.sandia.gov/nisac
July 9, 2018 2 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Catastrophe and Cyber-physical Threats
Hurricane Sandy (NJ), Harvey (Texas), Irma (Florida).
Large-scale power cut.
Flooding subway stations.
Collapsed and submerged roads.
July 9, 2018 3 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Hurricane Sandy
Source: New York Times
July 9, 2018 4 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Resilience
Figure: Resilience of PSE&G Electric Power System during Hurricane Sandy. Source:
New York Times.
July 9, 2018 5 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Resilience
Figure: Resilience of ConEd Electric Power System during Hurricane Sandy. Source: New
York Times.
July 9, 2018 6 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Catastrophe and Cyber-physical Threats
Advanced persistent threats: Stuxnet
Specifically targeted v.s. spray-and-pray
Long-term persistent v.s. smash-and-grab
Methodical v.s. opportunistic
Source: https://www.extremetech.com/computing/200898-windows-pcs-vulnerable-to-stuxnet-attack-
five-years-after-patches
July 9, 2018 7 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Security and Resilience
Security: Deter attackers from reaching and sabotaging their targets.
Resilience: What if an attack succeeds?
Guarantee essential services.
Reduce economic loss.
Recover wholly and quickly from failures.
Security Resilience
Deter	attacks	
from	success
Mitigate	
attack	
impact
Recover	quickly	
and	entirely
July 9, 2018 8 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Network-level Security and Resilience
Connected infrastructure sectors to enhance information, energy and material
exchanges.
Multi-layer networks model the interdependent infrastructure sectors.
Nodes are abstractions of systems or network components.
Links represent logical, physical or geographical dependencies.
http://energyskeptic.com/2011/em/
July 9, 2018 9 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Interdependencies among different critical infrastructures
Source: Gao et al. Natl. Sci. Rev. 2014.
July 9, 2018 10 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Cascading Failures
Interdependencies cause cross-infrastructure cascading failures.
Demos of cascading failures and recovery
https://drive.google.com/drive/u/0/folders/0B6-Q8-SnvO6lYmlxX2FuN3ZuV1U
Mitigate cascading failures through
Agile response to disasters and attacks
Dynamic long-term planning of resources
July 9, 2018 11 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Node-level Security and Resilience
Zooming-in: From high-level network models to node-level models.
Modeling the attack path of APTs for a facility.
Initial entry: compromise, but not breach the network.
Privilege escalation: control the (C&C) servers to receive additional
instructions and malicious code.
Lateral movement: establish additional points of compromise so that the
attack can continue if one point is closed.
Leave backdoors and the network remains compromised.
Developing cost-effective proactive defense strategies.
July 9, 2018 12 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Game-theoretic Framework
Advantages of game-theoretic framework
Worst-case analysis of natural failures with unknown statistics
Modeling attackers and defenders with distinctive objectives
Security from a strategic perspective
Computational challenges
NE computation of stochastic games over large-scale networks
PBNE computation of dynamic games under incomplete information
July 9, 2018 13 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Part I:
Network-Level Infrastructure Protection
Multi-layer networks under cyber-physical attacks
Game-theoretic modeling of cascading failures and resilient
policies
Approximation algorithm to tackle the curse of
dimensionality
July 9, 2018 14 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Cyber and Physical Attacks
Natural disasters or attacks cause a component failure
Cyber, physical, and logical interdependence
Negative effects on other components
Systematic failures
July 9, 2018 15 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Related Work
Modeling and understanding large-scale interdependent infrastructure
network.
Physical, cyber, geographic, logical dependency [Rinaldi et al. 2001].
Risk management based on network flows [Lee et al. 2007], numerical
simulation [Korkali et al. 2014], and interacting dynamic coupling [Rosato et
al. 2008].
CASCADE [Dobson et al. 2005]: High-level probabilistic model.
Game-theoretic methods for security and resilience of cyberphysical control
systems [Zhu and Ba¸sar 2015] and decentralized decision-making [Chen and Zhu
2016].
Scalable methods for curse of dimensionality: constraint sampling in
approximate dynamic programming [Farias and Roy 2004] and factored MDP
[Guestrin et al. 2003].
July 9, 2018 16 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Multi-layer Networks
Nodes represent components and links represent dependencies: G = (N, E).
The jth
node at layer i has a state of being normal xi
j = 1 or faulty xi
j = 0.
The jth
node at layer i can be attacked (resp. defended) ai
j = 1 (resp.
di
j = 1) or not ai
j = 0 (resp. di
j = 0).
July 9, 2018 17 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Single-layer Network
A global index l to unify the 2D index (i, j), e.g., Ω1,1 = {n1
1, n1
2, n2
1, n3
7} as
Ω1 = {n1, n2, n6, n17}.
System state x = [xl]l=1,··· ,17 ∈ X.
System action of defender d ∈ D and attacker a ∈ A.
Exponential growth of the state size.
July 9, 2018 18 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Stationary Policy
Defender policy µ : X → D := l Dl.
Attacker policy ν : X → A := l Al.
Information structure Fl of node l.
July 9, 2018 19 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Zero-sum Stochastic Markov Game
A dynamic game with probabilistic transitions.
Generalization of both Markov decision processes and repeated games.
Markov transition probability Pr(xt+1
|xt
, a, d).
Node l’s utility cl(xl, dl, al).
System utility at state x, c(x, d, a) = l cl.
Long-term objective J(x0
,µ, νµ, νµ, ν) :=
∞
t=0 γt
Eµ,νµ,νµ,ν,x0 [c(Xt, d, a)].
July 9, 2018 20 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Zero-Sum Stochastic Markov Game
Goal: find secure strategy µ∗
, ν∗µ∗
, ν∗
µ∗
, ν∗
and the value function of the game
J∗
(x0
) = min
µµµ∈U
max
ννν∈V
J(x0
,µ, νµ, νµ, ν) = J(x0
,µ∗
, ν∗
µ∗
, ν∗
µ∗
, ν∗
).
Risk quantification: J∗
(x0
) : X → R provides a security measure of state x0
.
Saddle-point equilibrium:
J(x0
,µ, ν∗
µ, ν∗
µ, ν∗
) ≥ J(x0
,µ∗
, ν∗
µ∗
, ν∗
µ∗
, ν∗
) ≥ J(x0
,µ∗
, νµ∗
, νµ∗
, ν), ∀ννν,µµµ, ∀x0
Minimax theorem:
min
µµµ∈U
max
ννν∈V
J(x0
,µ, νµ, νµ, ν) = max
ννν∈V
min
µµµ∈U
J(x0
,µ, νµ, νµ, ν).
Feasible stationary mixed strategy:
µ∗
µ∗
µ∗
(x) ∈ Ux
:= {φd
(x, d) ∈ R≥0
:
d
φd
(x, d) = 1}, ∀x
φd
(x, d) is the probability of taking action d at the global state x for a
defender.
July 9, 2018 21 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Dynamic Programming
Bellman Equation J∗
(x) = c(x, d∗
, a∗
) + γ x Pr(x |x, a∗
, d∗
)J∗
(x ), ∀x.
The first term is the reward of current stage x.
The second term is the expectation of the value function over all the
possible next stage x .
Mixed-strategy generalization
J∗
(x) =
a∈A
φa∗
(x, a) ×
f(x,a)
d∈D

c(x, d, a) + γ
x ∈ I
i=1 Xi
Pr(x |x, a, d)J∗
(x )

 φd∗
(x, d), ∀x.
July 9, 2018 22 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Bilinear Programming
min
J∗(x),φd(x,d)
x
α(x)J∗
(x)
subject to :
J∗
(x) ≥
d
c(x, d, a) + γ
x
Pr(x |x, a, d)J∗
(x ) φd
(x, d), ∀x, ∀a
d∈D
φd
(x, d) = 1, ∀x
φd
(x, d) ≥ 0, ∀x, d.
Bilinear programming is nonlinear and the current computation tools do not
succeed in providing the global optimal.
The direct computation of J∗
(x) is hard, but we can use value iteration
Jt+1
(x) := min
φd
max
φa
a∈A
φa
(x, a)
d∈D
[c(x, d, a) + γ
x
Pr(x |x, a, d)Jt
(x )]φd
(x, d).
July 9, 2018 23 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Value Iteration
Define z(x) :=
maxφa
a∈A φa
(x, a) d∈D[c(x, d, a) + γ
x
Pr(x |x, a, d)Jt
(x )]φd
(x, d).
Solve iteratively for the following linear programming with initial guess J0
(x).
min
z(x),φd(x,d)
x
α(x)z(x)
subject to :
z(x) ≥
d
c(x, d, a) + γ
x
Pr(x |x, a, d)Jt
(x ) φd
(x, d), ∀x, ∀a
d∈D
φd
(x, d) = 1, ∀x
φd
(x, d) ≥ 0, ∀x, d.
The optimal value of variable z(x) is the Jt+1
(x).
Replace Jt
(x) with Jt+1
(x) and iterate.
July 9, 2018 24 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Single controller
Single controller assumption Pr(x |x, a, d) = Pr(x |x, a) results in the
following linear program.
Prime LP:
min
J∗(x),φd(x,d)
x
α(x )J∗
(x )
subject to :
J∗
(x) ≥
d∈D
c(x, d, a)φd
(x, d) + γ
x
Pr(x |x, a)J∗
(x ), ∀x, a
d∈D
φd
(x, d) = 1, ∀x
φd
(x, d) ≥ 0, ∀x, d.
Large-scale network with system state x.
LP variables J∗
(x) and φd
(x, d).
LP constraints ∀x ∈ X, ∀a ∈ A.
July 9, 2018 25 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Approximation
Approximate LP: J∗
(x) =
k
j=1 wjhj(x).
Restricted information structure of the defender
φd
(x, d) =
n
l=1
φd
l (x, dl) =
n
l=1
φd
l (Fl, dl)
and Fl is the set of nodes which node l can observe, e.g., Fl = xl.
Factored graph to exploit the sparsity of dependencies:
P(x |x, a) =
i∈N
P(xi|x, a) =
i∈N
P(xi|xi, xΩi
, ai).
Variable elimination: sum and max → max and sum.
July 9, 2018 26 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Example of Variable Elimination
(1 − γ)w0 ≥ max
x1,...,x4
e1(x1) + e2(x1, x2) + e3(x2, x3, x4) + e4(x3, x4).
With an elimination order O = {x3, x2, x4, x1}, the RHS
max
x1,x2,x4
e1(x1) + e2(x1, x2) + max
x3
e3(x2, x3, x4) + e4(x3, x4)
= max
x1,x2,x4
e1(x1) + e2(x1, x2) + E1(x2, x4).
A new constraint is generated, i.e.,
E1(x2, x4) ≥ e3(x2, x3, x4) + e4(x3, x4), ∀x2, x3, x4.
21 3 4
Unattackable a2 ≡ 0
xΩ1 = ∅ xΩ2 = [x1] xΩ2 = [x2, x4] xΩ2 = [x3]
July 9, 2018 27 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Computation Reduction
ALP in red is insensitive to network size.
Exact LP in blue grows exponentially1
.
1Huang et al., MSCPES, CPS-Week, 2017; Huang et al., GameSec 2017
July 9, 2018 28 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Acceptable Approximate Error
Small absolute errors in green and red.
Relative error in blue decreases as the network size increases.
July 9, 2018 29 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
High-Level Connections Between Two Models:
Connections between Network-Level and Node-Level Models
The node-level model provides a zoomed-in model of nodes at the
network-level model.
The node-level analysis provides ways to estimate parameters for the
network-level analysis.
Transition probability Pr(xt+1
|xt
, a, d).
Node l’s utility cl(xl, dl, al).
July 9, 2018 30 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Part II:
Node-Level Infrastructure Protection
Industrial control systems under multi-stage multi-phase
APTs
Game-theoretic modeling of their dynamic, stealthy, and
deceptive nature
Adaptive Bayesian learning for incomplete information
Proactive and reactive information structures for insider and
outsider threats
July 9, 2018 31 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Multistage Infiltration
APTs infiltrate stage by stage.
The attack graph has a tree structure without loops.
The stages are discrete and the horizon T is finite.
Defender
Stage 0 Stage 1 Stage t Stage T
Attacker
σ0
2 = R2(σ0
1) σ1
2 = R2(σ1
1) σt
2 = R2(σt
1) σT
2 = R2(σT
1 )
σ0
1 σ1
1 σt
1 σT
1
h0
= Ø h0
= {a0
1, a0
2} ht
= {ht−1
, at−1
1 , at−1
2 } hT
= {a0
1, ..., aT −1
1 , a0
2, ..., aT −1
2 }
a0
1 a1
1 at
1 aT
1
a0
2 a1
2 at
2 aT
2
July 9, 2018 32 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Game-theoretic Modeling of Strategic Attackers2
Acknowledge the entry: Traditional intrusion prevention can be ineffective for
APTs.
Steal full cryptographic key by zero-day vulnerabilities.
Bridge the air gap, e.g., infect other insecure clients of the same services
provider and propagate through USB.
Strategic attackers: APTs operated by human experts can analyze system
responses and learn the detection rule, thus evade traditional intrusion
detection.
2Huang and Zhu, CINS, Sigmetrics, 2018
July 9, 2018 33 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Related Work
Identification of APTs [Cole 2012].
Dynamic game-theoretic framework for APTs in CIs.
A security game plus an information-trading game for insider threats [Hu
et al. 2015].
Multi-layer and multi-phase game model of APTs [Zhu and Rass 2018].
Flip-It game [Dijk et al. 2013]: APTs steal the private key so that they
stealthily take over the system alternately with the defender.
Incomplete information and deception.
Use random variable to model the incomplete information in a game
[Harsanyi 1967].
Cyber denial and deception [Stech et al. 2016]: Reverse deceptions from
defenders to counter the deceptive and stealthy nature of APTs.
Bayesian learning and conjugate prior assumptions [Ryzhov 2012].
July 9, 2018 34 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Type as the Incomplete Information
A random variable models the incomplete information caused by the
deceptive and stealthy nature of APTs.
The realization of the random variable is the type of attackers.
Attacker’s type θ2 ∈ Θ2 distinguishes between legitimate users and APTs
with different targets.
Defender does not know the realization of the type, and needs to form a
belief Bt
1(θ2).
July 9, 2018 35 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Action and History
Discrete actions for player i ∈ {1, 2} at stage t: at
i ∈ At
i.
The feasible action set At
i is stage-dependent.
Observable history and perfect recall:
ht
:= [a0
1, · · · , at−1
1 , a0
2, · · · , at−1
2 ] ∈ Ht
.
Observing history is not sufficient for strategic decision making.
Behaviors do not directly reveal the type.
Different defensive methods work for different types of attacks.
Defender
Stage 0 Stage 1 Stage t Stage T
Attacker
σ0
2 = R2(σ0
1) σ1
2 = R2(σ1
1) σt
2 = R2(σt
1) σT
2 = R2(σT
1 )
σ0
1 σ1
1 σt
1 σT
1
h0
= Ø h0
= {a0
1, a0
2} ht
= {ht−1
, at−1
1 , at−1
2 } hT
= {a0
1, ..., aT −1
1 , a0
2, ..., aT −1
2 }
a0
1 a1
1 at
1 aT
1
a0
2 a1
2 at
2 aT
2
July 9, 2018 36 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Markov State Transition
The cardinality of the history is increasing with stages.
History update: ht
= ht−1
∪ {at
1, at
2}.
State xt
shows the current system status, e.g., pressure, location of APTs,
compromised sensors, etc.
Initial state x0
and the history ht
determine state xt
∈ Xt
at stage t.
Markov state transition: xt+1
= ft
(xt
, at
1, at
2).
0 TT − 1t − 1 tStage
V T −1
i (hT −1
, θi)
V T
i (hT
, θi)
V t−1
i (ht−1
, θi)
Cost-to-go from stage t − 1
V T
i (hT
, θi)V t
i (ht
, θi)V t−1
i (ht−1
, θi)
DP
n0
3
nT
1
nT
2
nT
3
nT
4
nT
5
July 9, 2018 37 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Dynamic Bayesian Bimatrix Game
Stage-dependent type belief: Bt
1 : Ht
→ Θ2.
P1 forms belief according to the current observation Ht
.
Θ2 is a probability distribution over the type space Θ2.
Behavioral mixed strategy: σt
i (·|ht
, θi) : Ht
× Θi → At
i
Probability measure: at
i∈At
i
σt
i (at
i|ht
, θi) = 1.
Action at
i is a realization of the policy σt
i .
July 9, 2018 38 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Adaptive Belief Update
Multistage Bayesian update:
Pr(Par|data, M) =
Pr(Par|M) × Pr(Par, data|M)
Pr(data|M)
.
Type belief depends on the mixed strategy σt
2 which serves as the likelihood
function of the new observation at
2.
Bt+1
1 (θ2|[ht
, at
1, at
2]) =
Bt
1(θ2|ht
)σt
2(at
2|ht
, θ2)
1
0
Bt
1(ˆθ2|ht)σt
2(at
2|ht, ˆθ2)dˆθ2
.
One action may not directly reveal the type, e.g., behavioral analysis rather
than signature analysis for encrypted outbound traffic.
Length of the connection.
Number of packets.
Amount of data.
Destination IP.
Adversarial objective is gradually learned via the multistage transition.
July 9, 2018 39 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Utility Function
Stage utility: Jt
i (xt
, at
1, at
2, θ1, θ2).
State-dependent: Increasing rotor’s speed under pressure state xt
leads
to different utilities for both players.
Type-related: Same action can result in different utilities for different
types.
Cumulative utility for complete information:
ˆUt :T
1 (σt :T
1 , σt :T
2 , hT +1
, θ1, θ2) =
T
t=t
Eσt
1,σt
2
[Jt
1(xt
, σt
1, σt
2, θ1, θ2)]
=
T
t=t at
1∈At
1
σt
1(at
1|ht
, θ1)
at
2∈At
2
σt
2(at
2|ht
, θ2)Jt
1(xt
, at
1, at
2, θ1, θ2).
Expected cumulative utility for incomplete information:
Ut :T
1 (σt :T
1 , σt :T
2 , hT +1
, θ1) :=
1
0
Bt
1(θ2|ht
) ˆUt :T
1 dθ2.
July 9, 2018 40 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Proactive Perfect Bayesian Nash Equilibrium (P-PBNE)
Proactive information structure for insider threats.
The attacker P2 as the agent perceives policy σt
1 via insiders and chooses
policy σt
2 = R2(σt
1) as the best response to σt
1, i.e., to maximize his own
accumulated utility Ut :T
2 :
σ∗,t :T
2 = arg max
σt :T
2 ∈Σt :T
2
Ut :T
2 (σ∗,t :T
1 , σt :T
2 ) := U∗,t :T
2 .
APTs have to follow rules to evade detection and defender P1 considers the
worst-case policy.
U∗,t :T
1 := inf
σt :T
2 ∈R2(σ∗,t :T
1 )
Ut :T
1 (σ∗,t :T
1 , σt :T
2 )
= sup
σt :T
1 ∈Σt :T
1
inf
σt :T
2 ∈R2(σt :T
1 )
Ut :T
1 (σt :T
1 , σt :T
2 ).
Such equilibrium is called Proactive Perfect Bayesian Nash Equilibrium
(P-PBNE).
July 9, 2018 41 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Reactive Perfect Bayesian Nash Equilibrium (R-PBNE)
Reactive information structure for outsider threats.
Each player does not know the policy of the other player at every stage.
A sequence of strategies σ∗,t :T
i ∈ Σt :T
i is called the ε-reactive perfect
Bayesian Nash equilibrium for player Pi if, for a given ε ≥ 0, i ∈ {1, 2}:
Ut:T
i (σ∗,t:T
i , σ∗,t:T
−i , hT +1
, θi) ≥ sup
σt:T
i ∈Σt:T
i
Ut:T
i (σt:T
i , σ∗,t:T
−i , hT +1
, θi) − ε.
If ε = 0, we have a Reactive Perfect Bayesian Nash Equilibrium (R-PBNE).
Each player cannot gain if deviating unilaterally at any stage.
July 9, 2018 42 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Forward and Backward Process
Optimality principle and dynamic programming: Value function V t
i is the
optimal utility-to-go from stage t for player i.
Incomplete information: Forward belief update coupled with backward PBNE
policy computation.
Stage 0 Stage 1 Stage T
V T
i
V 1
i
V 0
i
B0
i B1
i BT
i
Forward Belief Update
Backward Policy Computation
Bt+1
1 (θ2|[ht
, at
1, at
2]) =
Bt
1(θ2|ht
)σt
2(at
2|ht
,θ2)
1
0
Bt
1(ˆθ2|ht)σt
2(at
2|ht,ˆθ2)dˆθ2
Conjugate prior assumption.
July 9, 2018 43 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Bayesian Games with Two-Sided Incomplete Information
Type θ1 ∈ Θ1
Belief formation
Bayesian update
Utility optimization
Player 1: defender
Bt
2 ∈ △Θ1
Type belief
Type belief
Bt
1 ∈ △Θ2
History ht
=
ht−1
∪ {at−1
1 , at−1
2 }
Belief formation
Bayesian update
Perfect Bayesian
Nash equilibrium
Utility optimization
Mixed strategy σt
1 ∈ △At
1
Mixed strategy σt
2 ∈ △At
2
Action at
1 ∈ At
1
Action at
2 ∈ At
2
Implementation
Implementation
Observable history and perfect recall
Type θ2 ∈ Θ2
Player 2: attacker
Observable history and perfect recall
July 9, 2018 44 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
One-sided Incomplete Information
Beta-binomial conjugate prior assumption to change the distribution update
into the parameter update.
Dynamic programming with an expanded state yt
= {xt
, αt
1, βt
1} to unify two
processes.
July 9, 2018 45 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Benchmark of Complete Information: Mitigate Attack
Economically
We study defender and attacker’s policies under different types of attackers.
For benign users who do not attack and inflict damages, the defender will not
take defensive actions and the system will operate normally.
When the type value increases:
P1 defends with a higher probability because an attack with a larger
type value incurs more loss once succeeds.
The increasing probability of defensive actions reduces the probability of
attacks to a relatively low level.
Defender's policy
Attacker's policy
0.0 0.4 0.6 0.8 1.0 1.2
Type
0.2
0.4
0.6
0.8
1.0
Probability
July 9, 2018 46 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Complete v.s. Incomplete Information
The deception of APTs creates (one-sided) uncertainties for defenders and
decreases defenders’ utilities.
NE and SE are obtained under complete information.
R-PBNE and P-PBNE are obtained under incomplete information.
More information yields better defender’s utilities for stronger types of
attacker. (Information is valuable.)
NE
R-PBNE
0.2 0.4 0.6 0.8 1.0
Type
0.2
0.4
0.6
0.8
1.0
P-PBNE
SE
Overlap
0.2 0.4 0.6 0.8 1.0
Type
0.7
0.8
0.9
1.0
July 9, 2018 47 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Proactive v.s. Reactive Information Structure
SE and P-PBNE are proactive solutions.
NE and R-PBNE are reactive solutions.
P-PBNE may not exist. Use supremum as the upper bound for P-PBNE.
Proactive solutions yield a higher level of utility for stronger attackers.
NE
SE
0.2 0.4 0.6 0.8 1.0
Type
0.85
0.90
0.95
1.00
R-PBNE
Supremum P-PBNE
0.2 0.4 0.6 0.8 1.0
Type
0.2
0.4
0.6
0.8
1.0
1.2
Acquiring the best-response set of the attacker via analysis of the attack tree
and honeypots can effectively confront the insider threat of APTs.
July 9, 2018 48 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Connections Between Two Models
Attack at the network-level aims to propagate over the network.
Attack at the node-level aims to compromise the facility.
An intelligent attacker can create both node level and network level damages
using coordinated attacks to maximize the attack impact at the network level.
July 9, 2018 49 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Connections Between Two Models
Defense at the network-level aims to allocate network-level resources to
prevent the spreading of the failures and recover the failures.
Defense at the node-level aims to proactively deter the attacker from
reaching the target and mitigate the damage on the facility.
July 9, 2018 50 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Think Fast and Slow
At the slow time scale: The equilibrium analysis of the fine-grained node-level
game model provides parameter inputs to the network-level model for
high-level resiliency planning.
At the slow time scale: The online behavior at each node determines the
real-time spreading rates (or probabilities).
July 9, 2018 51 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Challenges
Large-scale interdependent network
Incomplete information of attacks
Composition of attacks on different layers of network
Human behavior modeling and human-in-the-loop cyber-physical system
July 9, 2018 52 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Impacts of Solution
Mechanism design to deter or engage attackers in the system
Prediction of the attack policies by analyzing the game equilibrium
Proactive defense to deter attacks rather than remedy actions
Long-term dynamic resilience planning
July 9, 2018 53 / 54
Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution
Thank You!
July 9, 2018 54 / 54

Más contenido relacionado

Similar a Algorithmic Game Theory for Critical Infrastructure Security and Resilience

Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkNAVER Engineering
 
Analytics for Smart Grid Cyber security
Analytics for Smart Grid Cyber securityAnalytics for Smart Grid Cyber security
Analytics for Smart Grid Cyber securityBoston Global Forum
 
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...AM Publications
 
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIO
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIOTOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIO
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIOIJCNCJournal
 
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...Sri Ram
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)MeetupDataScienceRoma
 
Monitoring Smart Grid Operations and Maintaining Missions Assurance
Monitoring Smart Grid Operations and Maintaining Missions AssuranceMonitoring Smart Grid Operations and Maintaining Missions Assurance
Monitoring Smart Grid Operations and Maintaining Missions Assurancenamblasec
 
Self-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecuritySelf-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecurityKim Hammar
 
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTS
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTSACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTS
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTSIJCNCJournal
 
Actor Critic Approach based Anomaly Detection for Edge Computing Environments
Actor Critic Approach based Anomaly Detection for Edge Computing EnvironmentsActor Critic Approach based Anomaly Detection for Edge Computing Environments
Actor Critic Approach based Anomaly Detection for Edge Computing EnvironmentsIJCNCJournal
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsIRJET Journal
 
Security optimization of dynamic networks with probabilistic graph modeling a...
Security optimization of dynamic networks with probabilistic graph modeling a...Security optimization of dynamic networks with probabilistic graph modeling a...
Security optimization of dynamic networks with probabilistic graph modeling a...Pvrtechnologies Nellore
 
Self-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecuritySelf-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecurityKim Hammar
 
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTING
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTINGFAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTING
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTINGIJNSA Journal
 
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...IRJET Journal
 
Resilience Metrics and Approaches for Quantification, Igor LINKOV
Resilience Metrics and Approaches for Quantification, Igor LINKOVResilience Metrics and Approaches for Quantification, Igor LINKOV
Resilience Metrics and Approaches for Quantification, Igor LINKOVGlobal Risk Forum GRFDavos
 
Self-learning systems for cyber security
Self-learning systems for cyber securitySelf-learning systems for cyber security
Self-learning systems for cyber securityKim Hammar
 
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...IJCNCJournal
 

Similar a Algorithmic Game Theory for Critical Infrastructure Security and Resilience (20)

Research of adversarial example on a deep neural network
Research of adversarial example on a deep neural networkResearch of adversarial example on a deep neural network
Research of adversarial example on a deep neural network
 
Analytics for Smart Grid Cyber security
Analytics for Smart Grid Cyber securityAnalytics for Smart Grid Cyber security
Analytics for Smart Grid Cyber security
 
O01021101112
O01021101112O01021101112
O01021101112
 
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...
Advanced Security Mechanism for Mobile Ad hoc Networks using Game Theoretic A...
 
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIO
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIOTOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIO
TOPOLOGY MAP ANALYSIS FOR EFFECTIVE CHOICE OF NETWORK ATTACK SCENARIO
 
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...
Towards Evaluating the Robustness of Deep Intrusion Detection Models in Adver...
 
Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)Adversarial examples in deep learning (Gregory Chatel)
Adversarial examples in deep learning (Gregory Chatel)
 
Monitoring Smart Grid Operations and Maintaining Missions Assurance
Monitoring Smart Grid Operations and Maintaining Missions AssuranceMonitoring Smart Grid Operations and Maintaining Missions Assurance
Monitoring Smart Grid Operations and Maintaining Missions Assurance
 
Self-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecuritySelf-Learning Systems for Cyber Security
Self-Learning Systems for Cyber Security
 
Adversarial ml
Adversarial mlAdversarial ml
Adversarial ml
 
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTS
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTSACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTS
ACTOR CRITIC APPROACH BASED ANOMALY DETECTION FOR EDGE COMPUTING ENVIRONMENTS
 
Actor Critic Approach based Anomaly Detection for Edge Computing Environments
Actor Critic Approach based Anomaly Detection for Edge Computing EnvironmentsActor Critic Approach based Anomaly Detection for Edge Computing Environments
Actor Critic Approach based Anomaly Detection for Edge Computing Environments
 
Survey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning ModelsSurvey of Adversarial Attacks in Deep Learning Models
Survey of Adversarial Attacks in Deep Learning Models
 
Security optimization of dynamic networks with probabilistic graph modeling a...
Security optimization of dynamic networks with probabilistic graph modeling a...Security optimization of dynamic networks with probabilistic graph modeling a...
Security optimization of dynamic networks with probabilistic graph modeling a...
 
Self-Learning Systems for Cyber Security
Self-Learning Systems for Cyber SecuritySelf-Learning Systems for Cyber Security
Self-Learning Systems for Cyber Security
 
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTING
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTINGFAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTING
FAST DETECTION OF DDOS ATTACKS USING NON-ADAPTIVE GROUP TESTING
 
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...
SECURING THE DIGITAL FORTRESS: ADVERSARIAL MACHINE LEARNING CHALLENGES AND CO...
 
Resilience Metrics and Approaches for Quantification, Igor LINKOV
Resilience Metrics and Approaches for Quantification, Igor LINKOVResilience Metrics and Approaches for Quantification, Igor LINKOV
Resilience Metrics and Approaches for Quantification, Igor LINKOV
 
Self-learning systems for cyber security
Self-learning systems for cyber securitySelf-learning systems for cyber security
Self-learning systems for cyber security
 
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...
Trust Metric-Based Anomaly Detection Via Deep Deterministic Policy Gradient R...
 

Último

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)Samir Dash
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoffsammart93
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...apidays
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodJuan lago vázquez
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusZilliz
 

Último (20)

MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 

Algorithmic Game Theory for Critical Infrastructure Security and Resilience

  • 1. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Algorithmic Game Theory for Critical Infrastructure Security and Resilience Linan Huang Quanyan Zhu Department of Electrical and Computer Engineering New York University, USA Game Solving: Theory and Practice, Prague, Czech Republic Monday, July 9, 2018 July 9, 2018 1 / 54
  • 2. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Critical Infrastructure Presidential Policy Directive 21 (PPD-21) identifies 16 sectors. Critical infrastructure sectors must be secure and resilient from all natural hazards and human attacks. Source: http://www.sandia.gov/nisac July 9, 2018 2 / 54
  • 3. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Catastrophe and Cyber-physical Threats Hurricane Sandy (NJ), Harvey (Texas), Irma (Florida). Large-scale power cut. Flooding subway stations. Collapsed and submerged roads. July 9, 2018 3 / 54
  • 4. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Hurricane Sandy Source: New York Times July 9, 2018 4 / 54
  • 5. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Resilience Figure: Resilience of PSE&G Electric Power System during Hurricane Sandy. Source: New York Times. July 9, 2018 5 / 54
  • 6. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Resilience Figure: Resilience of ConEd Electric Power System during Hurricane Sandy. Source: New York Times. July 9, 2018 6 / 54
  • 7. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Catastrophe and Cyber-physical Threats Advanced persistent threats: Stuxnet Specifically targeted v.s. spray-and-pray Long-term persistent v.s. smash-and-grab Methodical v.s. opportunistic Source: https://www.extremetech.com/computing/200898-windows-pcs-vulnerable-to-stuxnet-attack- five-years-after-patches July 9, 2018 7 / 54
  • 8. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Security and Resilience Security: Deter attackers from reaching and sabotaging their targets. Resilience: What if an attack succeeds? Guarantee essential services. Reduce economic loss. Recover wholly and quickly from failures. Security Resilience Deter attacks from success Mitigate attack impact Recover quickly and entirely July 9, 2018 8 / 54
  • 9. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Network-level Security and Resilience Connected infrastructure sectors to enhance information, energy and material exchanges. Multi-layer networks model the interdependent infrastructure sectors. Nodes are abstractions of systems or network components. Links represent logical, physical or geographical dependencies. http://energyskeptic.com/2011/em/ July 9, 2018 9 / 54
  • 10. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Interdependencies among different critical infrastructures Source: Gao et al. Natl. Sci. Rev. 2014. July 9, 2018 10 / 54
  • 11. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Cascading Failures Interdependencies cause cross-infrastructure cascading failures. Demos of cascading failures and recovery https://drive.google.com/drive/u/0/folders/0B6-Q8-SnvO6lYmlxX2FuN3ZuV1U Mitigate cascading failures through Agile response to disasters and attacks Dynamic long-term planning of resources July 9, 2018 11 / 54
  • 12. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Node-level Security and Resilience Zooming-in: From high-level network models to node-level models. Modeling the attack path of APTs for a facility. Initial entry: compromise, but not breach the network. Privilege escalation: control the (C&C) servers to receive additional instructions and malicious code. Lateral movement: establish additional points of compromise so that the attack can continue if one point is closed. Leave backdoors and the network remains compromised. Developing cost-effective proactive defense strategies. July 9, 2018 12 / 54
  • 13. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Game-theoretic Framework Advantages of game-theoretic framework Worst-case analysis of natural failures with unknown statistics Modeling attackers and defenders with distinctive objectives Security from a strategic perspective Computational challenges NE computation of stochastic games over large-scale networks PBNE computation of dynamic games under incomplete information July 9, 2018 13 / 54
  • 14. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Part I: Network-Level Infrastructure Protection Multi-layer networks under cyber-physical attacks Game-theoretic modeling of cascading failures and resilient policies Approximation algorithm to tackle the curse of dimensionality July 9, 2018 14 / 54
  • 15. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Cyber and Physical Attacks Natural disasters or attacks cause a component failure Cyber, physical, and logical interdependence Negative effects on other components Systematic failures July 9, 2018 15 / 54
  • 16. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Related Work Modeling and understanding large-scale interdependent infrastructure network. Physical, cyber, geographic, logical dependency [Rinaldi et al. 2001]. Risk management based on network flows [Lee et al. 2007], numerical simulation [Korkali et al. 2014], and interacting dynamic coupling [Rosato et al. 2008]. CASCADE [Dobson et al. 2005]: High-level probabilistic model. Game-theoretic methods for security and resilience of cyberphysical control systems [Zhu and Ba¸sar 2015] and decentralized decision-making [Chen and Zhu 2016]. Scalable methods for curse of dimensionality: constraint sampling in approximate dynamic programming [Farias and Roy 2004] and factored MDP [Guestrin et al. 2003]. July 9, 2018 16 / 54
  • 17. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Multi-layer Networks Nodes represent components and links represent dependencies: G = (N, E). The jth node at layer i has a state of being normal xi j = 1 or faulty xi j = 0. The jth node at layer i can be attacked (resp. defended) ai j = 1 (resp. di j = 1) or not ai j = 0 (resp. di j = 0). July 9, 2018 17 / 54
  • 18. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Single-layer Network A global index l to unify the 2D index (i, j), e.g., Ω1,1 = {n1 1, n1 2, n2 1, n3 7} as Ω1 = {n1, n2, n6, n17}. System state x = [xl]l=1,··· ,17 ∈ X. System action of defender d ∈ D and attacker a ∈ A. Exponential growth of the state size. July 9, 2018 18 / 54
  • 19. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Stationary Policy Defender policy µ : X → D := l Dl. Attacker policy ν : X → A := l Al. Information structure Fl of node l. July 9, 2018 19 / 54
  • 20. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Zero-sum Stochastic Markov Game A dynamic game with probabilistic transitions. Generalization of both Markov decision processes and repeated games. Markov transition probability Pr(xt+1 |xt , a, d). Node l’s utility cl(xl, dl, al). System utility at state x, c(x, d, a) = l cl. Long-term objective J(x0 ,µ, νµ, νµ, ν) := ∞ t=0 γt Eµ,νµ,νµ,ν,x0 [c(Xt, d, a)]. July 9, 2018 20 / 54
  • 21. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Zero-Sum Stochastic Markov Game Goal: find secure strategy µ∗ , ν∗µ∗ , ν∗ µ∗ , ν∗ and the value function of the game J∗ (x0 ) = min µµµ∈U max ννν∈V J(x0 ,µ, νµ, νµ, ν) = J(x0 ,µ∗ , ν∗ µ∗ , ν∗ µ∗ , ν∗ ). Risk quantification: J∗ (x0 ) : X → R provides a security measure of state x0 . Saddle-point equilibrium: J(x0 ,µ, ν∗ µ, ν∗ µ, ν∗ ) ≥ J(x0 ,µ∗ , ν∗ µ∗ , ν∗ µ∗ , ν∗ ) ≥ J(x0 ,µ∗ , νµ∗ , νµ∗ , ν), ∀ννν,µµµ, ∀x0 Minimax theorem: min µµµ∈U max ννν∈V J(x0 ,µ, νµ, νµ, ν) = max ννν∈V min µµµ∈U J(x0 ,µ, νµ, νµ, ν). Feasible stationary mixed strategy: µ∗ µ∗ µ∗ (x) ∈ Ux := {φd (x, d) ∈ R≥0 : d φd (x, d) = 1}, ∀x φd (x, d) is the probability of taking action d at the global state x for a defender. July 9, 2018 21 / 54
  • 22. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Dynamic Programming Bellman Equation J∗ (x) = c(x, d∗ , a∗ ) + γ x Pr(x |x, a∗ , d∗ )J∗ (x ), ∀x. The first term is the reward of current stage x. The second term is the expectation of the value function over all the possible next stage x . Mixed-strategy generalization J∗ (x) = a∈A φa∗ (x, a) × f(x,a) d∈D  c(x, d, a) + γ x ∈ I i=1 Xi Pr(x |x, a, d)J∗ (x )   φd∗ (x, d), ∀x. July 9, 2018 22 / 54
  • 23. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Bilinear Programming min J∗(x),φd(x,d) x α(x)J∗ (x) subject to : J∗ (x) ≥ d c(x, d, a) + γ x Pr(x |x, a, d)J∗ (x ) φd (x, d), ∀x, ∀a d∈D φd (x, d) = 1, ∀x φd (x, d) ≥ 0, ∀x, d. Bilinear programming is nonlinear and the current computation tools do not succeed in providing the global optimal. The direct computation of J∗ (x) is hard, but we can use value iteration Jt+1 (x) := min φd max φa a∈A φa (x, a) d∈D [c(x, d, a) + γ x Pr(x |x, a, d)Jt (x )]φd (x, d). July 9, 2018 23 / 54
  • 24. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Value Iteration Define z(x) := maxφa a∈A φa (x, a) d∈D[c(x, d, a) + γ x Pr(x |x, a, d)Jt (x )]φd (x, d). Solve iteratively for the following linear programming with initial guess J0 (x). min z(x),φd(x,d) x α(x)z(x) subject to : z(x) ≥ d c(x, d, a) + γ x Pr(x |x, a, d)Jt (x ) φd (x, d), ∀x, ∀a d∈D φd (x, d) = 1, ∀x φd (x, d) ≥ 0, ∀x, d. The optimal value of variable z(x) is the Jt+1 (x). Replace Jt (x) with Jt+1 (x) and iterate. July 9, 2018 24 / 54
  • 25. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Single controller Single controller assumption Pr(x |x, a, d) = Pr(x |x, a) results in the following linear program. Prime LP: min J∗(x),φd(x,d) x α(x )J∗ (x ) subject to : J∗ (x) ≥ d∈D c(x, d, a)φd (x, d) + γ x Pr(x |x, a)J∗ (x ), ∀x, a d∈D φd (x, d) = 1, ∀x φd (x, d) ≥ 0, ∀x, d. Large-scale network with system state x. LP variables J∗ (x) and φd (x, d). LP constraints ∀x ∈ X, ∀a ∈ A. July 9, 2018 25 / 54
  • 26. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Approximation Approximate LP: J∗ (x) = k j=1 wjhj(x). Restricted information structure of the defender φd (x, d) = n l=1 φd l (x, dl) = n l=1 φd l (Fl, dl) and Fl is the set of nodes which node l can observe, e.g., Fl = xl. Factored graph to exploit the sparsity of dependencies: P(x |x, a) = i∈N P(xi|x, a) = i∈N P(xi|xi, xΩi , ai). Variable elimination: sum and max → max and sum. July 9, 2018 26 / 54
  • 27. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Example of Variable Elimination (1 − γ)w0 ≥ max x1,...,x4 e1(x1) + e2(x1, x2) + e3(x2, x3, x4) + e4(x3, x4). With an elimination order O = {x3, x2, x4, x1}, the RHS max x1,x2,x4 e1(x1) + e2(x1, x2) + max x3 e3(x2, x3, x4) + e4(x3, x4) = max x1,x2,x4 e1(x1) + e2(x1, x2) + E1(x2, x4). A new constraint is generated, i.e., E1(x2, x4) ≥ e3(x2, x3, x4) + e4(x3, x4), ∀x2, x3, x4. 21 3 4 Unattackable a2 ≡ 0 xΩ1 = ∅ xΩ2 = [x1] xΩ2 = [x2, x4] xΩ2 = [x3] July 9, 2018 27 / 54
  • 28. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Computation Reduction ALP in red is insensitive to network size. Exact LP in blue grows exponentially1 . 1Huang et al., MSCPES, CPS-Week, 2017; Huang et al., GameSec 2017 July 9, 2018 28 / 54
  • 29. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Acceptable Approximate Error Small absolute errors in green and red. Relative error in blue decreases as the network size increases. July 9, 2018 29 / 54
  • 30. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution High-Level Connections Between Two Models: Connections between Network-Level and Node-Level Models The node-level model provides a zoomed-in model of nodes at the network-level model. The node-level analysis provides ways to estimate parameters for the network-level analysis. Transition probability Pr(xt+1 |xt , a, d). Node l’s utility cl(xl, dl, al). July 9, 2018 30 / 54
  • 31. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Part II: Node-Level Infrastructure Protection Industrial control systems under multi-stage multi-phase APTs Game-theoretic modeling of their dynamic, stealthy, and deceptive nature Adaptive Bayesian learning for incomplete information Proactive and reactive information structures for insider and outsider threats July 9, 2018 31 / 54
  • 32. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Multistage Infiltration APTs infiltrate stage by stage. The attack graph has a tree structure without loops. The stages are discrete and the horizon T is finite. Defender Stage 0 Stage 1 Stage t Stage T Attacker σ0 2 = R2(σ0 1) σ1 2 = R2(σ1 1) σt 2 = R2(σt 1) σT 2 = R2(σT 1 ) σ0 1 σ1 1 σt 1 σT 1 h0 = Ø h0 = {a0 1, a0 2} ht = {ht−1 , at−1 1 , at−1 2 } hT = {a0 1, ..., aT −1 1 , a0 2, ..., aT −1 2 } a0 1 a1 1 at 1 aT 1 a0 2 a1 2 at 2 aT 2 July 9, 2018 32 / 54
  • 33. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Game-theoretic Modeling of Strategic Attackers2 Acknowledge the entry: Traditional intrusion prevention can be ineffective for APTs. Steal full cryptographic key by zero-day vulnerabilities. Bridge the air gap, e.g., infect other insecure clients of the same services provider and propagate through USB. Strategic attackers: APTs operated by human experts can analyze system responses and learn the detection rule, thus evade traditional intrusion detection. 2Huang and Zhu, CINS, Sigmetrics, 2018 July 9, 2018 33 / 54
  • 34. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Related Work Identification of APTs [Cole 2012]. Dynamic game-theoretic framework for APTs in CIs. A security game plus an information-trading game for insider threats [Hu et al. 2015]. Multi-layer and multi-phase game model of APTs [Zhu and Rass 2018]. Flip-It game [Dijk et al. 2013]: APTs steal the private key so that they stealthily take over the system alternately with the defender. Incomplete information and deception. Use random variable to model the incomplete information in a game [Harsanyi 1967]. Cyber denial and deception [Stech et al. 2016]: Reverse deceptions from defenders to counter the deceptive and stealthy nature of APTs. Bayesian learning and conjugate prior assumptions [Ryzhov 2012]. July 9, 2018 34 / 54
  • 35. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Type as the Incomplete Information A random variable models the incomplete information caused by the deceptive and stealthy nature of APTs. The realization of the random variable is the type of attackers. Attacker’s type θ2 ∈ Θ2 distinguishes between legitimate users and APTs with different targets. Defender does not know the realization of the type, and needs to form a belief Bt 1(θ2). July 9, 2018 35 / 54
  • 36. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Action and History Discrete actions for player i ∈ {1, 2} at stage t: at i ∈ At i. The feasible action set At i is stage-dependent. Observable history and perfect recall: ht := [a0 1, · · · , at−1 1 , a0 2, · · · , at−1 2 ] ∈ Ht . Observing history is not sufficient for strategic decision making. Behaviors do not directly reveal the type. Different defensive methods work for different types of attacks. Defender Stage 0 Stage 1 Stage t Stage T Attacker σ0 2 = R2(σ0 1) σ1 2 = R2(σ1 1) σt 2 = R2(σt 1) σT 2 = R2(σT 1 ) σ0 1 σ1 1 σt 1 σT 1 h0 = Ø h0 = {a0 1, a0 2} ht = {ht−1 , at−1 1 , at−1 2 } hT = {a0 1, ..., aT −1 1 , a0 2, ..., aT −1 2 } a0 1 a1 1 at 1 aT 1 a0 2 a1 2 at 2 aT 2 July 9, 2018 36 / 54
  • 37. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Markov State Transition The cardinality of the history is increasing with stages. History update: ht = ht−1 ∪ {at 1, at 2}. State xt shows the current system status, e.g., pressure, location of APTs, compromised sensors, etc. Initial state x0 and the history ht determine state xt ∈ Xt at stage t. Markov state transition: xt+1 = ft (xt , at 1, at 2). 0 TT − 1t − 1 tStage V T −1 i (hT −1 , θi) V T i (hT , θi) V t−1 i (ht−1 , θi) Cost-to-go from stage t − 1 V T i (hT , θi)V t i (ht , θi)V t−1 i (ht−1 , θi) DP n0 3 nT 1 nT 2 nT 3 nT 4 nT 5 July 9, 2018 37 / 54
  • 38. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Dynamic Bayesian Bimatrix Game Stage-dependent type belief: Bt 1 : Ht → Θ2. P1 forms belief according to the current observation Ht . Θ2 is a probability distribution over the type space Θ2. Behavioral mixed strategy: σt i (·|ht , θi) : Ht × Θi → At i Probability measure: at i∈At i σt i (at i|ht , θi) = 1. Action at i is a realization of the policy σt i . July 9, 2018 38 / 54
  • 39. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Adaptive Belief Update Multistage Bayesian update: Pr(Par|data, M) = Pr(Par|M) × Pr(Par, data|M) Pr(data|M) . Type belief depends on the mixed strategy σt 2 which serves as the likelihood function of the new observation at 2. Bt+1 1 (θ2|[ht , at 1, at 2]) = Bt 1(θ2|ht )σt 2(at 2|ht , θ2) 1 0 Bt 1(ˆθ2|ht)σt 2(at 2|ht, ˆθ2)dˆθ2 . One action may not directly reveal the type, e.g., behavioral analysis rather than signature analysis for encrypted outbound traffic. Length of the connection. Number of packets. Amount of data. Destination IP. Adversarial objective is gradually learned via the multistage transition. July 9, 2018 39 / 54
  • 40. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Utility Function Stage utility: Jt i (xt , at 1, at 2, θ1, θ2). State-dependent: Increasing rotor’s speed under pressure state xt leads to different utilities for both players. Type-related: Same action can result in different utilities for different types. Cumulative utility for complete information: ˆUt :T 1 (σt :T 1 , σt :T 2 , hT +1 , θ1, θ2) = T t=t Eσt 1,σt 2 [Jt 1(xt , σt 1, σt 2, θ1, θ2)] = T t=t at 1∈At 1 σt 1(at 1|ht , θ1) at 2∈At 2 σt 2(at 2|ht , θ2)Jt 1(xt , at 1, at 2, θ1, θ2). Expected cumulative utility for incomplete information: Ut :T 1 (σt :T 1 , σt :T 2 , hT +1 , θ1) := 1 0 Bt 1(θ2|ht ) ˆUt :T 1 dθ2. July 9, 2018 40 / 54
  • 41. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Proactive Perfect Bayesian Nash Equilibrium (P-PBNE) Proactive information structure for insider threats. The attacker P2 as the agent perceives policy σt 1 via insiders and chooses policy σt 2 = R2(σt 1) as the best response to σt 1, i.e., to maximize his own accumulated utility Ut :T 2 : σ∗,t :T 2 = arg max σt :T 2 ∈Σt :T 2 Ut :T 2 (σ∗,t :T 1 , σt :T 2 ) := U∗,t :T 2 . APTs have to follow rules to evade detection and defender P1 considers the worst-case policy. U∗,t :T 1 := inf σt :T 2 ∈R2(σ∗,t :T 1 ) Ut :T 1 (σ∗,t :T 1 , σt :T 2 ) = sup σt :T 1 ∈Σt :T 1 inf σt :T 2 ∈R2(σt :T 1 ) Ut :T 1 (σt :T 1 , σt :T 2 ). Such equilibrium is called Proactive Perfect Bayesian Nash Equilibrium (P-PBNE). July 9, 2018 41 / 54
  • 42. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Reactive Perfect Bayesian Nash Equilibrium (R-PBNE) Reactive information structure for outsider threats. Each player does not know the policy of the other player at every stage. A sequence of strategies σ∗,t :T i ∈ Σt :T i is called the ε-reactive perfect Bayesian Nash equilibrium for player Pi if, for a given ε ≥ 0, i ∈ {1, 2}: Ut:T i (σ∗,t:T i , σ∗,t:T −i , hT +1 , θi) ≥ sup σt:T i ∈Σt:T i Ut:T i (σt:T i , σ∗,t:T −i , hT +1 , θi) − ε. If ε = 0, we have a Reactive Perfect Bayesian Nash Equilibrium (R-PBNE). Each player cannot gain if deviating unilaterally at any stage. July 9, 2018 42 / 54
  • 43. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Forward and Backward Process Optimality principle and dynamic programming: Value function V t i is the optimal utility-to-go from stage t for player i. Incomplete information: Forward belief update coupled with backward PBNE policy computation. Stage 0 Stage 1 Stage T V T i V 1 i V 0 i B0 i B1 i BT i Forward Belief Update Backward Policy Computation Bt+1 1 (θ2|[ht , at 1, at 2]) = Bt 1(θ2|ht )σt 2(at 2|ht ,θ2) 1 0 Bt 1(ˆθ2|ht)σt 2(at 2|ht,ˆθ2)dˆθ2 Conjugate prior assumption. July 9, 2018 43 / 54
  • 44. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Bayesian Games with Two-Sided Incomplete Information Type θ1 ∈ Θ1 Belief formation Bayesian update Utility optimization Player 1: defender Bt 2 ∈ △Θ1 Type belief Type belief Bt 1 ∈ △Θ2 History ht = ht−1 ∪ {at−1 1 , at−1 2 } Belief formation Bayesian update Perfect Bayesian Nash equilibrium Utility optimization Mixed strategy σt 1 ∈ △At 1 Mixed strategy σt 2 ∈ △At 2 Action at 1 ∈ At 1 Action at 2 ∈ At 2 Implementation Implementation Observable history and perfect recall Type θ2 ∈ Θ2 Player 2: attacker Observable history and perfect recall July 9, 2018 44 / 54
  • 45. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution One-sided Incomplete Information Beta-binomial conjugate prior assumption to change the distribution update into the parameter update. Dynamic programming with an expanded state yt = {xt , αt 1, βt 1} to unify two processes. July 9, 2018 45 / 54
  • 46. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Benchmark of Complete Information: Mitigate Attack Economically We study defender and attacker’s policies under different types of attackers. For benign users who do not attack and inflict damages, the defender will not take defensive actions and the system will operate normally. When the type value increases: P1 defends with a higher probability because an attack with a larger type value incurs more loss once succeeds. The increasing probability of defensive actions reduces the probability of attacks to a relatively low level. Defender's policy Attacker's policy 0.0 0.4 0.6 0.8 1.0 1.2 Type 0.2 0.4 0.6 0.8 1.0 Probability July 9, 2018 46 / 54
  • 47. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Complete v.s. Incomplete Information The deception of APTs creates (one-sided) uncertainties for defenders and decreases defenders’ utilities. NE and SE are obtained under complete information. R-PBNE and P-PBNE are obtained under incomplete information. More information yields better defender’s utilities for stronger types of attacker. (Information is valuable.) NE R-PBNE 0.2 0.4 0.6 0.8 1.0 Type 0.2 0.4 0.6 0.8 1.0 P-PBNE SE Overlap 0.2 0.4 0.6 0.8 1.0 Type 0.7 0.8 0.9 1.0 July 9, 2018 47 / 54
  • 48. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Proactive v.s. Reactive Information Structure SE and P-PBNE are proactive solutions. NE and R-PBNE are reactive solutions. P-PBNE may not exist. Use supremum as the upper bound for P-PBNE. Proactive solutions yield a higher level of utility for stronger attackers. NE SE 0.2 0.4 0.6 0.8 1.0 Type 0.85 0.90 0.95 1.00 R-PBNE Supremum P-PBNE 0.2 0.4 0.6 0.8 1.0 Type 0.2 0.4 0.6 0.8 1.0 1.2 Acquiring the best-response set of the attacker via analysis of the attack tree and honeypots can effectively confront the insider threat of APTs. July 9, 2018 48 / 54
  • 49. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Connections Between Two Models Attack at the network-level aims to propagate over the network. Attack at the node-level aims to compromise the facility. An intelligent attacker can create both node level and network level damages using coordinated attacks to maximize the attack impact at the network level. July 9, 2018 49 / 54
  • 50. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Connections Between Two Models Defense at the network-level aims to allocate network-level resources to prevent the spreading of the failures and recover the failures. Defense at the node-level aims to proactively deter the attacker from reaching the target and mitigate the damage on the facility. July 9, 2018 50 / 54
  • 51. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Think Fast and Slow At the slow time scale: The equilibrium analysis of the fine-grained node-level game model provides parameter inputs to the network-level model for high-level resiliency planning. At the slow time scale: The online behavior at each node determines the real-time spreading rates (or probabilities). July 9, 2018 51 / 54
  • 52. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Challenges Large-scale interdependent network Incomplete information of attacks Composition of attacks on different layers of network Human behavior modeling and human-in-the-loop cyber-physical system July 9, 2018 52 / 54
  • 53. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Impacts of Solution Mechanism design to deter or engage attackers in the system Prediction of the attack policies by analyzing the game equilibrium Proactive defense to deter attacks rather than remedy actions Long-term dynamic resilience planning July 9, 2018 53 / 54
  • 54. Critical Infrastructure Security and Resilience Network-level Model Node-Level Model Connections Challenge and Solution Thank You! July 9, 2018 54 / 54