Enviar búsqueda
Cargar
Pregel
•
0 recomendaciones
•
1,932 vistas
Julia Proskurnia
Seguir
Denunciar
Compartir
Denunciar
Compartir
1 de 30
Descargar ahora
Descargar para leer sin conexión
Recomendados
Ex1 ef sumprdfns
Ex1 ef sumprdfns
giovanniL
Pregel: A System For Large Scale Graph Processing
Pregel: A System For Large Scale Graph Processing
Riyad Parvez
2013.09.10 Giraph at London Hadoop Users Group
2013.09.10 Giraph at London Hadoop Users Group
Nitay Joffe
Real-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache Flink
DataWorks Summit
Pregel: A System for Large-Scale Graph Processing
Pregel: A System for Large-Scale Graph Processing
Chris Bunch
Graph processing - Powergraph and GraphX
Graph processing - Powergraph and GraphX
Amir Payberah
Introducing Apache Giraph for Large Scale Graph Processing
Introducing Apache Giraph for Large Scale Graph Processing
sscdotopen
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
nathanmarz
Recomendados
Ex1 ef sumprdfns
Ex1 ef sumprdfns
giovanniL
Pregel: A System For Large Scale Graph Processing
Pregel: A System For Large Scale Graph Processing
Riyad Parvez
2013.09.10 Giraph at London Hadoop Users Group
2013.09.10 Giraph at London Hadoop Users Group
Nitay Joffe
Real-time Stream Processing with Apache Flink
Real-time Stream Processing with Apache Flink
DataWorks Summit
Pregel: A System for Large-Scale Graph Processing
Pregel: A System for Large-Scale Graph Processing
Chris Bunch
Graph processing - Powergraph and GraphX
Graph processing - Powergraph and GraphX
Amir Payberah
Introducing Apache Giraph for Large Scale Graph Processing
Introducing Apache Giraph for Large Scale Graph Processing
sscdotopen
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
nathanmarz
MapReduce for scientific simulation analysis
MapReduce for scientific simulation analysis
David Gleich
An Intro to Hadoop
An Intro to Hadoop
Matthew McCullough
Hadoop at JavaZone 2010
Hadoop at JavaZone 2010
Matthew McCullough
Graphlab dunning-clustering
Graphlab dunning-clustering
Ted Dunning
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
IBM India Smarter Computing
Hadoop v0.3.1
Hadoop v0.3.1
Matthew McCullough
Introduction to R for Data Mining
Introduction to R for Data Mining
Revolution Analytics
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Josh Patterson
Scalding
Scalding
Mario Pastorelli
RAPIDS cuGraph – Accelerating all your Graph needs
RAPIDS cuGraph – Accelerating all your Graph needs
Connected Data World
Strata + Hadoop World 2012: Knitting Boar
Strata + Hadoop World 2012: Knitting Boar
Cloudera, Inc.
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Achim Friedland
Performance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ Applications
Michael Kopp
Generator
Generator
Ramasubbu .P
SUBJECT
SUBJECT
Ramasubbu .P
Scheduling in distributed systems - Andrii Vozniuk
Scheduling in distributed systems - Andrii Vozniuk
Andrii Vozniuk
Hadoop & Greenplum: Why Do Such a Thing?
Hadoop & Greenplum: Why Do Such a Thing?
Ed Kohlwey
Crude-Oil Scheduling Technology: moving from simulation to optimization
Crude-Oil Scheduling Technology: moving from simulation to optimization
Brenno Menezes
Beginner’s guide to sap abap 1
Beginner’s guide to sap abap 1
Panduka Bandara
New Directions for Mahout
New Directions for Mahout
Ted Dunning
Thesis finalpresentation
Thesis finalpresentation
Julia Proskurnia
Last.fm under the BFS "attack"
Last.fm under the BFS "attack"
Julia Proskurnia
Más contenido relacionado
Similar a Pregel
MapReduce for scientific simulation analysis
MapReduce for scientific simulation analysis
David Gleich
An Intro to Hadoop
An Intro to Hadoop
Matthew McCullough
Hadoop at JavaZone 2010
Hadoop at JavaZone 2010
Matthew McCullough
Graphlab dunning-clustering
Graphlab dunning-clustering
Ted Dunning
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
IBM India Smarter Computing
Hadoop v0.3.1
Hadoop v0.3.1
Matthew McCullough
Introduction to R for Data Mining
Introduction to R for Data Mining
Revolution Analytics
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Josh Patterson
Scalding
Scalding
Mario Pastorelli
RAPIDS cuGraph – Accelerating all your Graph needs
RAPIDS cuGraph – Accelerating all your Graph needs
Connected Data World
Strata + Hadoop World 2012: Knitting Boar
Strata + Hadoop World 2012: Knitting Boar
Cloudera, Inc.
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Achim Friedland
Performance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ Applications
Michael Kopp
Generator
Generator
Ramasubbu .P
SUBJECT
SUBJECT
Ramasubbu .P
Scheduling in distributed systems - Andrii Vozniuk
Scheduling in distributed systems - Andrii Vozniuk
Andrii Vozniuk
Hadoop & Greenplum: Why Do Such a Thing?
Hadoop & Greenplum: Why Do Such a Thing?
Ed Kohlwey
Crude-Oil Scheduling Technology: moving from simulation to optimization
Crude-Oil Scheduling Technology: moving from simulation to optimization
Brenno Menezes
Beginner’s guide to sap abap 1
Beginner’s guide to sap abap 1
Panduka Bandara
New Directions for Mahout
New Directions for Mahout
Ted Dunning
Similar a Pregel
(20)
MapReduce for scientific simulation analysis
MapReduce for scientific simulation analysis
An Intro to Hadoop
An Intro to Hadoop
Hadoop at JavaZone 2010
Hadoop at JavaZone 2010
Graphlab dunning-clustering
Graphlab dunning-clustering
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
ITG whitepaper: Value Proposition for AIX on IBM Power Systems: Ownership Exp...
Hadoop v0.3.1
Hadoop v0.3.1
Introduction to R for Data Mining
Introduction to R for Data Mining
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Hadoop Summit EU 2013: Parallel Linear Regression, IterativeReduce, and YARN
Scalding
Scalding
RAPIDS cuGraph – Accelerating all your Graph needs
RAPIDS cuGraph – Accelerating all your Graph needs
Strata + Hadoop World 2012: Knitting Boar
Strata + Hadoop World 2012: Knitting Boar
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Fosdem 2011 - A Common Graph Database Access Layer for .Net and Mono
Performance Management in ‘Big Data’ Applications
Performance Management in ‘Big Data’ Applications
Generator
Generator
SUBJECT
SUBJECT
Scheduling in distributed systems - Andrii Vozniuk
Scheduling in distributed systems - Andrii Vozniuk
Hadoop & Greenplum: Why Do Such a Thing?
Hadoop & Greenplum: Why Do Such a Thing?
Crude-Oil Scheduling Technology: moving from simulation to optimization
Crude-Oil Scheduling Technology: moving from simulation to optimization
Beginner’s guide to sap abap 1
Beginner’s guide to sap abap 1
New Directions for Mahout
New Directions for Mahout
Más de Julia Proskurnia
Thesis finalpresentation
Thesis finalpresentation
Julia Proskurnia
Last.fm under the BFS "attack"
Last.fm under the BFS "attack"
Julia Proskurnia
ZooKeeper - wait free protocol for coordinating processes
ZooKeeper - wait free protocol for coordinating processes
Julia Proskurnia
Planet Lab
Planet Lab
Julia Proskurnia
Group7 presentation
Group7 presentation
Julia Proskurnia
Fluidanimate:PARSEC Application Analysis
Fluidanimate:PARSEC Application Analysis
Julia Proskurnia
Más de Julia Proskurnia
(6)
Thesis finalpresentation
Thesis finalpresentation
Last.fm under the BFS "attack"
Last.fm under the BFS "attack"
ZooKeeper - wait free protocol for coordinating processes
ZooKeeper - wait free protocol for coordinating processes
Planet Lab
Planet Lab
Group7 presentation
Group7 presentation
Fluidanimate:PARSEC Application Analysis
Fluidanimate:PARSEC Application Analysis
Pregel
1.
PREGEL A Systems for
Large Scale Graph Processing by Iuliia Proskurnia ber 2012 ovem 6 th of M
2.
GRAPHS ARE EVERYWHERE
Graphs Examples 2
3.
GRAPHS ARE EVERYWHERE
Algorithms 3
4.
Outline Mo#va#on Basic Concepts in
design Implementa#on Details Evalua#on Conclusions 4
5.
MOTIVATION Large
Graph Processing No Such System Exist Single Machine Algorithms Parallel Solution 5
6.
MAPREDUCE LIKE SOLUTION
MapReduce is great :) 6
7.
MAPREDUCE LIKE SOLUTION
MapReduce is great :) BUT 6
8.
MAPREDUCE LIKE SOLUTION
MapReduce is great :) BUT Pregel 6
9.
Inspira@on Valiant’s Bulk Synchronous
Parallel Model 7
10.
BASIC DESIGN CONCEPTS
8
11.
VERTEX-‐CENTRIC APPROACH
Uses supersteps for computa@on Send/Receive Messages Change the state Modify topology VERTEX Termination? 9
12.
MAXIMUM VALUE EXAMPLE
Chicken Chicken • Dotted Arrows - messages • Grey Nodes - InActive 10
13.
API
DETAILS Combiners 11
14.
API
DETAILS Aggregators 12
15.
IMPLEMENTATION DETAILS
13
16.
IMPLEMENTATION
Master is chosen 14
17.
IMPLEMENTATION
Master is chosen Cluster Management System’s name service 14
18.
IMPLEMENTATION
Par@@on hash(VertexID) mod R hash(VertexID) mod R hash(VertexID) mod R 15
19.
IMPLEMENTATION
Reading the input 16
20.
IMPLEMENTATION
Reading the input GFS, BigTable 16
21.
IMPLEMENTATION
SuperStep 17
22.
IMPLEMENTATION
SuperStep 17
23.
IMPLEMENTATION
SuperStep Termination if (all VoteToHalt) { terminate();} 17
24.
IMPLEMENTATION
Saving the results Save graph state 18
25.
FAULT-‐TOLERANCE
CheckPoin@ng. Chicken Chicken. 19
26.
FAULT-‐TOLERANCE
CheckPoin@ng. Chicken Chicken. 19
27.
EVALUATION
20
28.
EVALUATION Number of
Worker Tasks 300 PCs Multicore Billion Vertices Binary Tree 21
29.
EVALUATION Number
of Ver@ces 300 PCs Multicore Tree with 127 average node degree 22
30.
Conclusion Vertex-‐Centric Approach Computa#on over
SuperSteps Usability and Scalability Fault Tolerance with checkpoints Performance -‐ almost linear to the size of the graph 23
Descargar ahora