my presentation of the paper "FAST'12 NCCloud"

1. Background – 2. Solution – 3. Evaluation – 4. Conclusion

.
FAST’12
NCCloud: Applying Network Coding for the
Storage Repair in a Cloud-of-Clouds
.

Yuchong Hu, Henry C. H. Chen, Patrick P. C. Lee, Yang Tang

presented by Shuai YUAN

NTHU
LSA lab

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
presented by Shuai YUAN FAST’12: NCCloud 1/20


. Outline
Background
Background
Erasure Codes
Regenerating Codes
Solution
the Paper’s Contributions
F-MSR Code
Evaluation
Repair Traﬃc Analysis
Cost Analysis
Experiments
Conclusion

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

1. Background – 2. Solution – 3. Evaluation – 4. Conclusion Background Erasure Codes Regenerating Codes

. Background
Cloud storage is an emerging service model for remote backup
and data synchronization.
Single-cloud storage raises concerns:
Cloud outage
Vendor lock-ins: Costly to switch cloud providers

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Proposed Solution
Multi-cloud storage
Deploy a proxy between users and multiple clouds
Stripe data across multiple clouds

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Proposed Solution
Problem: Multi-cloud storage repair
The proxy reads the essential data pieces from other surviving
clouds, reconstructs new data pieces, and writes these new pieces
to a new cloud.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Proposed Solution

Table : Monthly price plans (in US dollars) for Amazon S3 (US
Standard), Rackspace Cloud Files and Windows Azure Storage, as of
September, 2011.
Amazon S3 Rackspace Azure
Storage (per GB) $0.14 $0.15 $0.15

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Proposed Solution

September, 2011.
Storage (per GB) $0.14 $0.15 $0.15

Replication is expensive!
e.g. To achieve double-fault tolerance, we need 2 replicas.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Proposed Solution

September, 2011.
Storage (per GB) $0.14 $0.15 $0.15

Replication is expensive!
e.g. To achieve double-fault tolerance, we need 2 replicas.
; Use erasure code instead.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Erasure Codes
Given a message/file of k equal-size native chunks, encode it into
n code chunks (n > k)
Optimal erasure codes: any k out of the n code blocks/chunks
are sufficient to recover the original message/file.
i.e. can tolerate arbitrary (n − k) failures.
We call this (n,k) MDS (maximum distance separable)
property.
e.g. Reed-Solomon Codes [6]
Near-optimal erasure codes: require (1 + ε)k code
blocks/chunks to recover (ε > 0).
i.e. cannot tolerate arbitrary (n − k) failures.
e.g. Local Reconstruction Codes (used in Windows Azure
Storage System) [4]

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Reed Solomon Codes (RAID-6) [6]

Achieve double-fault tolerance
Reed-Solomon Codes: additional data size = M
Relication: additional data size = 2M (2 replicas)

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Reed-Solomon Codes (RAID-6) [6]
How do Reed-Solomon Codes work?
Given ﬁle F : divide it into k equal-size native chunks:
F = [Fi ]i=1,2,...,k . Encode them into (n − k) code chunks:
C = [Ci ]i=1,2,...,n−k .
Use Encoding Matrix EM(n−k)×k to produce code chunks:

CT = EM × F T

Ci is the linear combination of F1 , F2 , . . . , Fk .

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


In our case, ( )
F = ( B
A )
1 1
EM =
(1 2) ( )
1 1 A
CT = ×
( 1 2 ) B
A+B
=
A + 2B

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


We can rewrite it in another equivalent representation:
Let P = [Pi ]i=1,2,...,n = [F1 , F2 , . . . ,) k , C1 , C2 , . . . , Cn−k ] be the
( F
I
n chunks in storage, EM ′ = , Here
EM
 
1 0 ... 0
0 1 . . . 0
 
I = . . . .
. . .. .
. . .
0 0 ... 1

then
PT = EM ′ ) F T
( T×
F
=
CT

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


In our case,  
( ) 1 0
I 0 1
EM ′ = = 1 1

EM
1 2
PT = EM ′ × F T
 
1 0 ( )
0 1
=  × A
1 1 B
1 2 
A
 B 
=  
A+B 
A + 2B

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..



Why Reed-Solomon Codes have MDS property?
EM ′ is the key of MDS property!

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


.
Theorem (necessary and suﬃcient condition)
.
Every possible k × k submatrix obtained by removing (n − k) rows
from EM ′ has full rank.
equivalent expression of full rank:
rank = k
non-singular
.
.
. .

Alternative view:
Consider the linear space of
P = [Pi ]i=1,2,...,n = [F1 , F2 , . . . , Fk , C1 , C2 , . . . , Cn−k ], its
dimension is k, and any k out of n vectors form a basis of the
linear space.
. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


Reed-Solomon Codes uses Vandermonde Matrix V as EM
 
1 1 1 ... 1
 1 2 3 ... k 
 
 12 2 2 3 2 ... k2 
V = 
 . . . .. . 
 . . .
. .
. . . 
.
1(n−k) 2(n−k) 3(n−k) . . . k (n−k)

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


So matrix EM ′ is:
 
1 0 0 ... 0
 0 1 0 ... 0 
 
 0 0 1 ... 0 
 
 . . . .. . 
 .
. .
. .
. . .
. 
 
 0 0 0 ... 1 
EM = 
′
 1


 1 1 ... 1 
 1 2 3 ... k 
 
 12 22 32 ... k2 
 
 . . . .. . 
 . . .
. .
. . .
. 
1 (n−k) 2(n−k) 3(n−k) . . . k (n−k)

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


Remark:
All arithmetic operations in Galois Field GF(2x ). Then every
number is less than 2x .
For more details of Galois Field, please refer to [7] and a
implementation tutorial [5].
EM ′ satisﬁes MDS property.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Problem of Traditional Erasure Codes (e.g. R-S Codes)
.
Deﬁnition (Repair traﬃc)
.
the amount of outbound data being read from other surviving
clouds during the single-cloud failure recovery.
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


.
.
.
Why do we only consider outbound traﬃc without taking inbound
traﬃc into account?

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


.
.
.
Why do we only consider outbound traﬃc without taking inbound
traﬃc into account?
Date transfer in (per GB) free free free
Date transfer out (per GB) $0.12 $0.18 $0.15

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..



.
Problem of Reed-Solomon Codes
.
Need to obtain otain size of the whole ﬁle to repair.
; Repair traﬃc is high!
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Regenerating Codes [1] [2] [3]
Goal:
Minimize repair traffic while maintaining same fault tolerance as
erasure codes ; recover faster than erasure codes
Idea:
Do not reconstruct the whole file as in erasure codes
Instead, read only the chunks (smaller than whole file) that
are needed to recover the lost chunks
Build on network coding:
network bandwidth is more critical resource compared to disk
access.
encode chunks in storage nodes

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


An example:

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


.
Problem of Regenerating Codes
.
In our case, the storage nodes provided by cloud venders:
Support read/write
Do not have encoding/decoding capabilities
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


.
Problem of Regenerating Codes
.
In our case, the storage nodes provided by cloud venders:
Support read/write
Do not have encoding/decoding capabilities
.
Next we will see how this paper of NCCloud solves these problems.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

1. Background – 2. Solution – 3. Evaluation – 4. Conclusion the Paper’s Contributions F-MSR Code

. the Paper’s Contributions
Build NCCloud, a proxy-based storage system that applies
regenerating codes in multiple-cloud storage
Design goals:
Propose an implementable design of functional
minimum-storage regenerating (F-MSR) code
Support basic read/write operations and the repair function
Preserve storage overhead as in MDS codes, while reducing
repair traﬃc
Implement and evaluate NCCloud in real storage setting
focus on double-fault tolerance (k = n − 2)
focus on single-fault recovery

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. F-MSR Key Idea
non-systematic: don’t keep original data as in systematic
codes.
functional repair: don’t need to exactly regenerate the failed
chunks, only require the repaired system maintains MDS
property.
Let the proxy do the encoding/decoding.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Upload
Given ﬁle F : divide it into k(n − k) equal-size native chunks:
F = [Fi ]i=1,2,...,k(n−k) . Encode them into n(n − k) code chunks:
P = [Pi ]i=1,2,...,n(n−k) .

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Upload
Upload process:
Encode
Produce Encoding Matrix (Use Vandermonde Matrix)
EM = [ECV1 , ECV2 , . . . , ECVn(n−k) ] = [αi,j ]n(n−k)×k(n−k)
Then produce code chunks:

PT = EM × F T

For i = 1, 2, . . . , n(n − k)

Pi = ECVi × F T
∑k(n−k)
= j=1 αi,j Fj

ECVi : encoding coeﬃent vector of Pi .
All arithmetic operations are in Galois Field GF(28 ).
Duplicate encoding matrix to all nodes as metadata..
. . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Download

Download process:
Download all the chunks from any k of n clouds
Multiply inverted encoding matrix with downloaded chunks

F T = P T × EM −1
. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
6 steps:

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
.
Simulations
.
Consider multiple rounds of permanent node failures for diﬀerent
values of n. In each round, we randomly pick a node to
permanently fail and trigger a repair.
.
.
Simulation result
.
If the loop of Steps 2 to 5 is repeated over 10 times ; bad repair
Only checking the MDS property, we see a bad repair very quickly:
after no more than 7 and 2 rounds for n = 8 and n = 12,
respectively.
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
Solution:
Two-phrase checking
MDS property check: Current repair maintains MDS property
Repair MDS property check: Next repair for any possible
failure maintains MDS property

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
Solution:
Two-phrase checking

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
Cost of two-phrase checking
k
MDS property check: enumerating Cn subsets of n nodes to
see if each of their corresponding encoding matrices forms a
full rank.
Repair MDS property check: for any failed node (out of n
nodes), we collect any one out of (n − k) chunks from the
other (n − 1) surviving nodes to reconstruct, therefore the
cost is n(n − k)(n−1) Cn .
k

Return to .. Unsolved Problems

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. File Repair
Cost of two-phrase checking
k
MDS property check: enumerating Cn subsets of n nodes to
see if each of their corresponding encoding matrices forms a
full rank.
Repair MDS property check: for any failed node (out of n
nodes), we collect any one out of (n − k) chunks from the
other (n − 1) surviving nodes to reconstruct, therefore the
cost is n(n − k)(n−1) Cn .
k

Return to .. Unsolved Problems
We have to check more times for the current repair, but bad repair
will be rare in the future iterative repairs.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..

1. Background – 2. Solution – 3. Evaluation – 4. Conclusion Repair Traffic Analysis Cost Analysis Experiments

. Repair Traffic
Native data size M
k(n − k) native chunks of size M/k(n − k)
k−1
Repair Traffic: M/k(n − k) × (n − 1) = M/k × (1 + )
n−k
1
For k = n − 2, Repair Traffic: M/2 × (1 + )
n−2
limn→∞ Repair Traffic = M/2

.
Save the repair traffic by close to 50% when n is large.
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Cost Analysis

September, 2011.

Storage (per GB) $0.14 $0.15 $0.15
Date transfer in (per GB) free free free
Date transfer out (per GB) $0.12 $0.18 $0.15
PUT,POST (per 10K requests) $0.10 free $0.01
GET (per 10K requests) $0.01 free $0.01

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Cost Analysis
Metadata of F-MSR
Metadata size = 160B; ﬁle size = several MBs
Overhead due to GET requests during repair
Assuming S3 plan in Sep 2011, n = 4, k = 2, ﬁle size = 4MB
Conventional repair: 0.427%
F-MSR repair: 0.854%

.
Overhead cost is low.
.

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Experiments
NCCloud deployment
Single machine connected to a cloud-of-clouds
n = 4, k = 2
Coding schemes
Reed-Solomon-based RAID-6 vs. F-MSR
Metric
Response time
Cloud environments:
Local cloud: OpenStack Swift
Commercial cloud: multiple containers in Azure

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Response Time: Local Cloud

F-MSR has higher response
time due to
encoding/decoding
overhead.
F-MSR has slightly less
response time in repair, due
to less data download

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Response Time: Commercial Cloud

No distinct response time diﬀerence, as network ﬂuctuations play a
. . . . . . . . . . . . . . . . . . . .
bigger role in actual response time. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Conclusion & Unsolved Problems
Conclusion:
Propose an implementable design of F-MSR:
Preserve storage cost.
Use less repair traﬃc.
Do not require storage nodes to have encoding capabilities.
Build NCCloud, which realizes F-MSR
Source code:
http://ansrlab.cse.cuhk.edu.hk/software/nccloud/

. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


. Conclusion & Unsolved Problems
Unsolved problems:
Repair costs is high when n and k are large: As we mentioned
before (Click .. here ), F-MSR uses two-phrase checking, which
consumes a lot of checking costs in the current repair phrase.
Just as Reed-Solomon codes use Vandermonde Matrix to
ensure MDS property, a better algorithm is still seeking to
replace the check-after-trying approach.
The reason why F-MSR chooses to download chunks from all
(n − 1) nodes for repairing a ﬁle comes from an argument in
[1]: The more nodes we download chunks from, the lower
repair traﬃc is. However, [1]’s conclusion is based on a
homogeneity model, and NCCloud’s multi-cloud solution is
actually a heterogeneous environment. Such a basis may be
invalid.
. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


A.G. Dimakis, P.B. Godfrey, Y. Wu, M.J. Wainwright, and
K. Ramchandran.
Network coding for distributed storage systems.
Information Theory, IEEE Transactions on, 56(9):4539–4551,
2010.
A.G. Dimakis, K. Ramchandran, Y. Wu, and C. Suh.
A survey on network codes for distributed storage.
Proceedings of the IEEE, 99(3):476–489, 2011.
A. Duminuco and E. Biersack.
A practical study of regenerating codes for peer-to-peer
backup systems.
In Distributed Computing Systems, 2009. ICDCS’09. 29th
IEEE International Conference on, pages 376–384. IEEE, 2009.
C. Huang, H. Simitci, Y. Xu, A. Ogus, B. Calder, P. Gopalan,
J. Li, and S. Yekhanin.
Erasure coding in windows azure storage.
. . . . . . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..


In USENIX Annual Technical Conference (USENIX ATC),
2012.
J.S. Plank et al.
A tutorial on reed-solomon coding for fault-tolerance in
raid-like systems.
Software Practice and Experience, 27(9):995–1012, 1997.
I.S. Reed and G. Solomon.
Polynomial codes over certain ﬁnite ﬁelds.
Journal of the Society for Industrial & Applied Mathematics,
8(2):300–304, 1960.
B. Sklar.
Reed-solomon codes.
Downloaded from URL http://www. informit.
com/content/images/art. sub.–sklar7.
sub.–reed-solomo-n/elementLinks/art. sub.–sklar7.
sub.–reed-solomon. pdf,(unknown pub date), pages 1–33,
2001. . . . . . . . . . . . . . . .
.. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
.
..
. . . .
.. .. ..

my presentation of the paper "FAST'12 NCCloud"

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (19)

Similar a my presentation of the paper "FAST'12 NCCloud"

Similar a my presentation of the paper "FAST'12 NCCloud" (20)

Último

Último (20)

my presentation of the paper "FAST'12 NCCloud"