1. Monitoring Least Squares Models of
Distributed Streams
M. Gabel*, D.Keren+, A. Schuster*
*Israel Institute of Technology,
+Haifa University
発表者 : 田部井 靖生 (JST/東工大)
KDD2015読み会@京大, 2015年8月29日(土)
9. 手法 (詳細)
• と書けるので, βは
AjとCjの平均で書ける
• (Δ,δ)の凸部分空間Cを以下満たすのように定
義する
• [Lemma1] Cに関して, 以下が成立する
If for all j, then
10. Sliding window と Infinite window
• Sliding window : βをWの範囲のデータから計算, β0を
最後のsync前のWの範囲のデータから計算
Ø になる条件 :
• Infinite window : βをこれまでのすべてのデータから計
算, β0を最後のsyncまでのすべてのデータから計算
Ø になる条件 :
shall be denoted with ˆ·. Hence initial values
kX
j=1
Aj
0 , ˆc0 =
1
k
kX
j=1
cj
0 , ˆ0 = ˆA 1
0 ˆc0 ,
lues
kX
j=1
Aj
, ˆc =
1
k
kX
j=1
cj
, ˆ = ˆA 1
ˆc .
= kA 1
thus ˆ = ˆA 1
ˆc = A 1
c = and
0. In other words, we can compute the OLS
averages of local Aj
, cj
rather than the sums:
1
k
X
j
Aj
! 1
1
k
X
j
cj
!
= ˆA 1
ˆc (3)
Time
sync nownow
Aj
0
W
Aj
W
Aj
0 Aj
old common
new
Aj
0 Aj
sliding
window
infinite window
Figure 3: Sliding and infinite window models. When
Aj
overlaps Aj
0, j
= Aj
Aj
0 =
P
new xixT
i
P
old xixT
i .
12. 実験
• Distributed Least Square monitor(DILSQ)とT間隔毎
にモデルを更新するPER(T)を比較
• DILSQは, sliding windowを採用
• 評価尺度として, モデルエラーとnormalized message
を用いた
– それぞれのノードで送られるメッセージの平均
• データセットは, 人工データ, Traffic Monitoring, Gas
Sensor Time Seriesを用いた
13. 人口データを用いた実験
• それぞれのRoundにおいて, y=xTβtrue+nにおいて, xは
N(0,1)のi.i.d, n N(0,σ2)
• DILSQのエラーは閾値ε=1.35を超えることはない
• DILSQは, βの変化に応じてモデルを変化させる
Figure 4: DILSQ model error (black) and syncs (bottom vertical lines) per round, compared to PER(100)
error (green), for k = 10 simulated nodes with m = 10 dimensions, and threshold ✏ = 1.35. Both algorithms
reduce communication to 1%, but DILSQ only syncs when changes (bottom purple line shows k k). PER(100)
syncs every 100 rounds, but is unable to maintain error below the threshold (dashed horizontal line).
guarantees maximum model error below the user-selected
threshold ✏, but PER does not. Hence, when comparing
the two, we find a posteriori the maximum period T (hence
minimum communication) for which the maximum error of
PER(T) is equal or below that of DILSQ. Note this gives
PER an unrealistic advantage. First, in a realistic setting we
cannot know a priori the optimal period T. Second, model
changes in realistic settings are not necessarily stationary:
the rate of model change may evolve, which DILSQ will
handle gracefully while PER cannot.
14. 閾値εがモデルの更新コストに影響
• (a)真のモデルは固定, (b)真のモデルは変化
• PER(T)のパラメータは, 最大エラーがDILSQと
同じになるように設定
ack) and syncs (bottom vertical lines) per round, compared to PER(100)
d nodes with m = 10 dimensions, and threshold ✏ = 1.35. Both algorithms
DILSQ only syncs when changes (bottom purple line shows k k). PER(100)
able to maintain error below the threshold (dashed horizontal line).
w the user-selected
e, when comparing
um period T (hence
e maximum error of
SQ. Note this gives
a realistic setting we
d T. Second, model
cessarily stationary:
which DILSQ will
In the fixed dataset,
elements drawn i.i.d
with k nodes, each
ctor x of size m and
and y = xT
true + n
noise of strength .
(a) Fixed dataset (b) Drift dataset
Figure 5: Communication for DILSQ (black) and
periodic algorithm tuned to achieve same max error
(green) at di↵erent threshold values. DILSQ com-
munication on fixed model drops to zero for more
permissive ✏ (not shown on logarithmic scale).
17. 閾値εがモデルの更新コストに影響
(a) Window size W = 60 (b) Window size W = 30
Figure 8: Communication for DILSQ (black) and
periodic algorithm (green) on the tra c dataset at
di↵erent ✏ values.
it
res
ne
a c
Sim
we
ind
to
ad
co
39
tri
e
of
syn
ite
co
