Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

How Does MapReduce Work? map(k1, Getting Started on Hadoop

20.560 visualizaciones

Publicado el

How Does MapReduce Work?


map(k1, v1) → list(k2, v2)
reduce(k2, list(v2)) → list(v3)

Several phases, which partition a problem into many tasks:
• load data into DFS…
• map phase: input split → (key, value) pairs, with optional combiner
• shuffle phase: sort on keys to group pairs… load-test your network!
• reduce phase: each task receives the values for one key
• pull data from DFS…
NB: “map” phase is required, the rest are optional.
Think of set operations on tuples (and check out Cascading.org).
Meanwhile, given all those (key, value) pairs listed above, it’s no
wonder that key/value stores have become such a popular topic of
conversation…

Publicado en: Tecnología

×