1.
‹#›© Cloudera, Inc. All rights reserved.
Juliet Hougland
Data Scientist
@j_houg
Matrix Decomposition
at Scale

2.
‹#›© Cloudera, Inc. All rights reserved.
The Singular Value
Decomposition

3.
‹#›© Cloudera, Inc. All rights reserved.
• Dimensionality
Reduction/PCA
• Feature dimension
reduction
• Visualization of gene
expression data
• Latent Semantic
Indexing
• Low Rank
Approximations
• Digital Signals Processing
SVD is applied everywhere
A Global Map of Human Gene Expression. Lukk Et al. [1]

4.
‹#›© Cloudera, Inc. All rights reserved.
Define SVD

5.
‹#›© Cloudera, Inc. All rights reserved.
Totally awesome LANL video

6.
‹#›© Cloudera, Inc. All rights reserved.
This doesn’t work on distributed,
commodity setups
Good ClusterBad Cluster

7.
‹#›© Cloudera, Inc. All rights reserved.
3 Distributed OSS
SVD Implementations
Mahout: Lanczos
Mahout: Stochastic
Spark: Lanczos

8.
‹#›© Cloudera, Inc. All rights reserved.
Lanczos’ Method

9.
‹#›© Cloudera, Inc. All rights reserved.
• Iterative, with the dominant
cost a matrix-vector multiply
• Requires at least k iterations to
get k singular vectors
Lanczos’ Method

10.
‹#›© Cloudera, Inc. All rights reserved.
• Randomly project
original matrix to lower
dimensional space
• Factorize the projected
matrix.
• Unproject
Stochastic SVD
M ⇡ QQ⇤
M
Finding Structure in Randomness. Halko Et al. http://bit.ly/19VVRXp

11.
‹#›© Cloudera, Inc. All rights reserved.
• What I test is written on
MapReduce
• Driver programs launch the series
of required map reduce jobs
• Lots of writing intermediate data
to disk
Frameworks
• Using the MLLib component
• Relies on Spark core
• => tries to pin data in memory

12.
‹#›© Cloudera, Inc. All rights reserved.
Note!
Mahout Scala & Spark Bindings are integrated in Mahout.
Version 0.10 release next month will move these methods
The Scala DSL for linear algebra:
val g = bt.t %*% bt - c - c.t + (s_q cross s_q) * (xi dot xi)

13.
‹#›© Cloudera, Inc. All rights reserved.
Performance Comparisons

14.
‹#›© Cloudera, Inc. All rights reserved.
[3]

15.
‹#›© Cloudera, Inc. All rights reserved.
MapReduce
[4]

16.
‹#›© Cloudera, Inc. All rights reserved.
Go Bananas tuning!
[5]

17.
‹#›© Cloudera, Inc. All rights reserved.
My Cluster
6 Nodes running CDH 5.3*
Per Node:
2 physical cores
24, with hyper threading
=> 144 total available cores
64 GB Memory
100 TB free in HDFS
!
*Running Spark 1.3
[6]

18.
‹#›© Cloudera, Inc. All rights reserved.
What am I factorizing?
[7]

19.
‹#›© Cloudera, Inc. All rights reserved.
What am I timing?
[8]

20.
‹#›© Cloudera, Inc. All rights reserved.
Think of the polar bears
[9]

21.
‹#›© Cloudera, Inc. All rights reserved.
Varying Columns

22.
‹#›© Cloudera, Inc. All rights reserved.
Varying Rows

23.
‹#›© Cloudera, Inc. All rights reserved.
Varying Sparsity

24.
‹#›© Cloudera, Inc. All rights reserved.
Progress in Numerical Computation
[10]

25.
‹#›© Cloudera, Inc. All rights reserved.
1. Genome PCA: http://bit.ly/1OxXMRy
2. SVD at LANL: http://bit.ly/193IIdY
3. Apples and Oranges: http://bit.ly/1xd1Q4d
4. Sound Board: http://bit.ly/19okavV
5. Bananas: http://bit.ly/1EGxh4p
6. Eniac: http://bit.ly/1F0GOWC
7. Big data pix tumblr: http://bigdatapix.tumblr.com/
8. Watch: http://bit.ly/1FZtIKX
9. Polar Bears: http://bit.ly/1G0gXQw
10.Progress in numerical computing: http://bit.ly/1ID8WR5
Thanks for the images!

26.
‹#›© Cloudera, Inc. All rights reserved.
Thanks!
juliet@cloudera.com
@j_houg
https://github.com/jhlch/svd-benchmark