Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
DEVFEST NANTES 16
Bruno Bonnin - @_bruno_b_
Apache Spark avec NodeJS ?
Oui, c’est possible avec EclairJS !
2DEVFEST NANTES 16
About me
Bruno Bonnin - @_bruno_b_
Architecte / Développeur
http://webdemo.myscript.com/
3DEVFEST NANTES 16
Java, Scala,
Python, R
JavaScript
4DEVFEST NANTES 16
EclairJS
5DEVFEST NANTES 16
Spark Worker
Spark Worker
EclairJS
Cluster
Manager
Spark WorkersSpark Driver
Libs SparkLibs Spark
Scrip...
6DEVFEST NANTES 16
EclairJS: implémentation des composants
7DEVFEST NANTES 16
EclairJS: API Spark Core
val lines = sc.textFile("dream.txt")
val words = lines
.flatMap(
line => line....
8DEVFEST NANTES 16
EclairJS: API Spark SQL
val df = sqlCtx.read.json("people.json")
df.printSchema()
df.select("name", "ag...
9DEVFEST NANTES 16
app.get('/words', (req, res) => {
var lines = sc.textFile('dream.txt').cache();
var something = 0;
var ...
10DEVFEST NANTES 16
app.get('/words', (req, res) => {
var lines = sc.textFile('dream.txt').cache();
var something = 0;
var...
11DEVFEST NANTES 16
app.get('/words', (req, res) => {
var lines = sc.textFile('dream.txt').cache();
var something = 0;
var...
12DEVFEST NANTES 16
app.get('/words', (req, res) => {
var lines = sc.textFile('dream.txt').cache();
var something = 0;
var...
13DEVFEST NANTES 16
EclairJS: déploiement
Spark WorkerSpark Workers
Application
EclairJS - API
Node
Cluster
Manager Eclair...
14DEVFEST NANTES 16
Démo
Client SparkApplication
EclairJS - API Node EclairJS - API Nashorn
Spark Workers
EclairJS - API N...
15DEVFEST NANTES 16
Conclusion
Facile à apprendre (surtout pour ceux qui connaissent l’API Scala/Java)
Idéal pour les déve...
16DEVFEST NANTES 16 16DEVFEST NANTES 16
Merci !
@_bruno_b_
https://github.com/bbonnin/devfestnantes2016
Próxima SlideShare
Cargando en…5
×

Apache Spark avec NodeJS ? Oui, c'est possible avec EclairJS !

786 visualizaciones

Publicado el

Présentation d'EclairJS faite lors du DevFest Nantes 2016

Publicado en: Software
  • Yes you are right. There are many research paper writing services available now. But almost services are fake and illegal. Only a genuine service will treat their customer with quality research papers. ⇒ www.HelpWriting.net ⇐
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí
  • Njce! Thanks for sharing.
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Apache Spark avec NodeJS ? Oui, c'est possible avec EclairJS !

  1. 1. DEVFEST NANTES 16 Bruno Bonnin - @_bruno_b_ Apache Spark avec NodeJS ? Oui, c’est possible avec EclairJS !
  2. 2. 2DEVFEST NANTES 16 About me Bruno Bonnin - @_bruno_b_ Architecte / Développeur http://webdemo.myscript.com/
  3. 3. 3DEVFEST NANTES 16 Java, Scala, Python, R JavaScript
  4. 4. 4DEVFEST NANTES 16 EclairJS
  5. 5. 5DEVFEST NANTES 16 Spark Worker Spark Worker EclairJS Cluster Manager Spark WorkersSpark Driver Libs SparkLibs Spark Script EngineScript Engine EclairJS Nashorn EclairJS Nashorn JVM JVM App NodeJS EclairJS NodeJS
  6. 6. 6DEVFEST NANTES 16 EclairJS: implémentation des composants
  7. 7. 7DEVFEST NANTES 16 EclairJS: API Spark Core val lines = sc.textFile("dream.txt") val words = lines .flatMap( line => line.split(" ")) .filter( word => word.trim.length > 0) val counts = words .mapToPair( word => (word, 1)) .reduceByKey(_ + _) var lines = sc.textFile("dream.txt"); var words = lines .flatMap(function (line) { return line.split(" "); }) .filter(function (word) { return word.trim().length > 0; }); var counts = words .mapToPair(function (word, Tuple2) { return new Tuple2(word, 1); }, [Tuple2]) .reduceByKey(function (a, b) { return a + b; });
  8. 8. 8DEVFEST NANTES 16 EclairJS: API Spark SQL val df = sqlCtx.read.json("people.json") df.printSchema() df.select("name", "age") .filter(col("age").gt(30)) .show() df.registerTempTable("people") val sqlDF = sqlCtx .sql("SELECT * FROM people") sqlDF.show() var df = sqlCtx.read().json('people.json'); df.printSchema(); df.select('name', 'age') .filter(col('age').gt(30)) .show(); df.registerTempTable('people'); var sqlDF = sqlCtx .sql('SELECT * FROM people'); sqlDF.show();
  9. 9. 9DEVFEST NANTES 16 app.get('/words', (req, res) => { var lines = sc.textFile('dream.txt').cache(); var something = 0; var words = lines .flatMap(line => { something += 1; console.log('flatmap:' + line); return line.split(' '); }) .collect() }); EclairJS: code dans NodeJS
  10. 10. 10DEVFEST NANTES 16 app.get('/words', (req, res) => { var lines = sc.textFile('dream.txt').cache(); var something = 0; var words = lines .flatMap(line => { something += 1; console.log('flatmap:' + line); return line.split(' '); }) .collect() }); EclairJS: code dans NodeJS .then(results => { res.json({result: results}); }) .catch(err => { res.status(500).send(err); });
  11. 11. 11DEVFEST NANTES 16 app.get('/words', (req, res) => { var lines = sc.textFile('dream.txt').cache(); var something = 0; var words = lines .flatMap(line => { something += 1; console.log('flatmap:' + line); return line.split(' '); }) .collect() }); EclairJS: code dans NodeJS .then(results => { res.json({result: results}); }) .catch(err => { res.status(500).send(err); });
  12. 12. 12DEVFEST NANTES 16 app.get('/words', (req, res) => { var lines = sc.textFile('dream.txt').cache(); var something = 0; var words = lines .flatMap(line => { function (line) { something += 1; console.log('flatmap:' + line); return line.split(' '); }) .collect() .then(results => { res.json({result: results}); }) .catch(err => { res.status(500).send(err); }); }); EclairJS: code dans NodeJS Code métier interpréter dans Nashorn (pas dans NodeJS) : ● Ne connait pas console ● Est compliant ES5
  13. 13. 13DEVFEST NANTES 16 EclairJS: déploiement Spark WorkerSpark Workers Application EclairJS - API Node Cluster Manager EclairJS - API Nashorn Apache Toree (Spark Driver) EclairJS - API Nashorn Jupyter Notebook Gateway EclairJS Shell EclairJS - API Nashorn
  14. 14. 14DEVFEST NANTES 16 Démo Client SparkApplication EclairJS - API Node EclairJS - API Nashorn Spark Workers EclairJS - API Nashorn
  15. 15. 15DEVFEST NANTES 16 Conclusion Facile à apprendre (surtout pour ceux qui connaissent l’API Scala/Java) Idéal pour les développeurs JS que le Scala peut rebuter :-) Implémentation en cours : dernière version avec le support de Spark 2.0 Pour aller plus loin: ● https://eclairjs.github.io/ ● https://github.com/EclairJS/eclairjs
  16. 16. 16DEVFEST NANTES 16 16DEVFEST NANTES 16 Merci ! @_bruno_b_ https://github.com/bbonnin/devfestnantes2016

×