SlideShare una empresa de Scribd logo
1 de 23
Descargar para leer sin conexión
Parallel Computing
      (with GPars)




                     NOV 2011
Evolution

● Old Days
  ○ threads
  ○ synchronize
  ○ locks
  ○ wait-notify
Evolution

● Recent enhancements
  ○ java.util.concurrent.*
  ○ Future task
  ○ Executor Pools
  ○ Blocking Queues
  ○ ConcurrentMap
Evolution

● Emerging Technologies
  ○ Map-Reduce
  ○ API's (Google Collections, JSR-166)
  ○ Non-blocking I/O
  ○ Parallel Friendly Languages (Scala, Clojure, Groovy)
Groovy Parallel Systems

High level abstractions for writing concurrent code!

 ● Code-level Helpers
    ○ Parallel Collections
    ○ Asynchronous Processing
    ○ Fork/Join - Map/Reduce

 ● Architecture-level Concepts
    ○ Dataflows
    ○ Actors

 ● Protecting Shared Mutable State
    ○ Agents
Groovy Parallel Systems

High level abstractions for writing concurrent code!

 ● Code-level Helpers
    ○ Parallel Collections
    ○ Asynchronous Processing
    ○ Fork/Join - Map/Reduce

 ● Architecture-level Concepts
    ○ Dataflows
    ○ Actors (For next time...)

 ● Protecting Shared Mutable State
    ○ Agents (For next time...)
DEMO 1
 (Parallel encryption)
Optimal Thread Pool Size




   Pool size = ???
Optimal Thread Pool Size




   Pool size = Ncores
Optimal Thread Pool Size




   Pool size = Ncores + 1
Optimal Thread Pool Size




   Pool size = Ncores + Block. Op. Ratio
Optimal Thread Pool Size




   Pool size = Ncores + wait time
                         cpu time
DEMO 2
"Dado un rango obtener la cantidad de
nr. primos agrupados por él ultimo digito"
Danger



 Don't mess up with mutable shared data!
DEMO 3
 (Facebook Search)
Collections
 ● eachParallel()
 ● eachWithIndexParallel()
 ● collectParallel()
 ● findAllParallel()
 ● findAnyParallel
 ● findParallel()
 ● everyParallel()
 ● anyParallel()
 ● grepParallel()
 ● groupByParallel()
 ● foldParallel()
 ● minParallel()
 ● maxParallel()
 ● sumParallel()
 ● splitParallel()
 ● countParallel()
 ● foldParallel()
DEMO 4
 (Facebook Singles)
Fork-Join




Map-Reduce
DEMO 5
 (API Calls)
DEMO 6
  (Multiget)
Multiget
+ Info
  ● http://gpars.org/0.12/guide/guide/single.html
  ● http://www.slideshare.net/paulk_asert/concurrency-with-
    gpars
  ● http://gee.cs.oswego.edu/dl/papers/fj.pdf
  ● http://static.googleusercontent.
    com/external_content/untrusted_dlcp/labs.google.
    com/en/us/papers/mapreduce-osdi04.pdf
  ● Java Concurrency in Practice (ISBN-0321349601)


Pics Credits
  ● Stone age: http://tinyurl.com/bvnlv74
  ● Steam train: http://tinyurl.com/6lp3krq
  ● Speed train: http://tinyurl.com/7jjc3v5
  ● CPU: http://www.flickr.com/photos/olivander/149865626/
https://github.com/pablomolnar/gpars_samples




                                    @pmolnar

Más contenido relacionado

La actualidad más candente

Lofar python meetup jan9 2013
Lofar python meetup jan9 2013Lofar python meetup jan9 2013
Lofar python meetup jan9 2013
Gijs Molenaar
 
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Eric Evans
 
証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議
Hiroki Mizuno
 

La actualidad más candente (19)

[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
 
Your data isn't that big @ Big Things Meetup 2016-05-16
Your data isn't that big @ Big Things Meetup 2016-05-16Your data isn't that big @ Big Things Meetup 2016-05-16
Your data isn't that big @ Big Things Meetup 2016-05-16
 
Toy Model Overview
Toy Model OverviewToy Model Overview
Toy Model Overview
 
Compiler basics: lisp to assembly
Compiler basics: lisp to assemblyCompiler basics: lisp to assembly
Compiler basics: lisp to assembly
 
.NET Memory Primer (Martin Kulov)
.NET Memory Primer (Martin Kulov).NET Memory Primer (Martin Kulov)
.NET Memory Primer (Martin Kulov)
 
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
 
Fantastic caches and where to find them
Fantastic caches and where to find themFantastic caches and where to find them
Fantastic caches and where to find them
 
Lofar python meetup jan9 2013
Lofar python meetup jan9 2013Lofar python meetup jan9 2013
Lofar python meetup jan9 2013
 
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
 
Syslog-ng Performance
Syslog-ng Performance Syslog-ng Performance
Syslog-ng Performance
 
Всеволод Поляков (DevOps Team Lead в Grammarly)
Всеволод Поляков (DevOps Team Lead в Grammarly)Всеволод Поляков (DevOps Team Lead в Grammarly)
Всеволод Поляков (DevOps Team Lead в Grammarly)
 
bup backup system (2011-04)
bup backup system (2011-04)bup backup system (2011-04)
bup backup system (2011-04)
 
Linux-Permission
Linux-PermissionLinux-Permission
Linux-Permission
 
証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議
 
“Show Me the Garbage!”, Understanding Garbage Collection
“Show Me the Garbage!”, Understanding Garbage Collection“Show Me the Garbage!”, Understanding Garbage Collection
“Show Me the Garbage!”, Understanding Garbage Collection
 
High Performance Systems Without Tears - Scala Days Berlin 2018
High Performance Systems Without Tears - Scala Days Berlin 2018High Performance Systems Without Tears - Scala Days Berlin 2018
High Performance Systems Without Tears - Scala Days Berlin 2018
 
Is It Faster to Go with Redpanda Transactions than Without Them?!
Is It Faster to Go with Redpanda Transactions than Without Them?!Is It Faster to Go with Redpanda Transactions than Without Them?!
Is It Faster to Go with Redpanda Transactions than Without Them?!
 
Restinio - header-only http and websocket server
Restinio - header-only http and websocket serverRestinio - header-only http and websocket server
Restinio - header-only http and websocket server
 
JSON's big problem android_taipei_201709
JSON's big problem android_taipei_201709JSON's big problem android_taipei_201709
JSON's big problem android_taipei_201709
 

Similar a Parallel computing with GPars

Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)
Geoffrey De Smet
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmap
Ruslan Meshenberg
 

Similar a Parallel computing with GPars (20)

The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
 
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
 
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
 
Real-time analytics with Druid at Appsflyer
Real-time analytics with Druid at AppsflyerReal-time analytics with Druid at Appsflyer
Real-time analytics with Druid at Appsflyer
 
Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)
 
Journey through high performance django application
Journey through high performance django applicationJourney through high performance django application
Journey through high performance django application
 
SystemML - Datapalooza Denver - 05.17.16 MWD
SystemML - Datapalooza Denver - 05.17.16 MWDSystemML - Datapalooza Denver - 05.17.16 MWD
SystemML - Datapalooza Denver - 05.17.16 MWD
 
Scio - A Scala API for Google Cloud Dataflow & Apache Beam
Scio - A Scala API for Google Cloud Dataflow & Apache BeamScio - A Scala API for Google Cloud Dataflow & Apache Beam
Scio - A Scala API for Google Cloud Dataflow & Apache Beam
 
Stripe CTF3 wrap-up
Stripe CTF3 wrap-upStripe CTF3 wrap-up
Stripe CTF3 wrap-up
 
Handout: 'Open Source Tools & Resources'
Handout: 'Open Source Tools & Resources'Handout: 'Open Source Tools & Resources'
Handout: 'Open Source Tools & Resources'
 
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overviewFlink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
 
Sorry - How Bieber broke Google Cloud at Spotify
Sorry - How Bieber broke Google Cloud at SpotifySorry - How Bieber broke Google Cloud at Spotify
Sorry - How Bieber broke Google Cloud at Spotify
 
Developing games and graphic visualizations in Pascal
Developing games and graphic visualizations in PascalDeveloping games and graphic visualizations in Pascal
Developing games and graphic visualizations in Pascal
 
MacGyver Learns Spark
MacGyver Learns SparkMacGyver Learns Spark
MacGyver Learns Spark
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmap
 
Machine learning at Scale with Apache Spark
Machine learning at Scale with Apache SparkMachine learning at Scale with Apache Spark
Machine learning at Scale with Apache Spark
 
Gpu workshop cluster universe: scripting cuda
Gpu workshop cluster universe: scripting cudaGpu workshop cluster universe: scripting cuda
Gpu workshop cluster universe: scripting cuda
 
GSoC2014 - Uniritter Presentation May, 2015
GSoC2014 - Uniritter Presentation May, 2015GSoC2014 - Uniritter Presentation May, 2015
GSoC2014 - Uniritter Presentation May, 2015
 
24 uses for perl6
24 uses for perl624 uses for perl6
24 uses for perl6
 

Último

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 

Último (20)

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Tech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdfTech Trends Report 2024 Future Today Institute.pdf
Tech Trends Report 2024 Future Today Institute.pdf
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 

Parallel computing with GPars

  • 1. Parallel Computing (with GPars) NOV 2011
  • 2. Evolution ● Old Days ○ threads ○ synchronize ○ locks ○ wait-notify
  • 3. Evolution ● Recent enhancements ○ java.util.concurrent.* ○ Future task ○ Executor Pools ○ Blocking Queues ○ ConcurrentMap
  • 4. Evolution ● Emerging Technologies ○ Map-Reduce ○ API's (Google Collections, JSR-166) ○ Non-blocking I/O ○ Parallel Friendly Languages (Scala, Clojure, Groovy)
  • 5. Groovy Parallel Systems High level abstractions for writing concurrent code! ● Code-level Helpers ○ Parallel Collections ○ Asynchronous Processing ○ Fork/Join - Map/Reduce ● Architecture-level Concepts ○ Dataflows ○ Actors ● Protecting Shared Mutable State ○ Agents
  • 6. Groovy Parallel Systems High level abstractions for writing concurrent code! ● Code-level Helpers ○ Parallel Collections ○ Asynchronous Processing ○ Fork/Join - Map/Reduce ● Architecture-level Concepts ○ Dataflows ○ Actors (For next time...) ● Protecting Shared Mutable State ○ Agents (For next time...)
  • 7. DEMO 1 (Parallel encryption)
  • 8. Optimal Thread Pool Size Pool size = ???
  • 9. Optimal Thread Pool Size Pool size = Ncores
  • 10. Optimal Thread Pool Size Pool size = Ncores + 1
  • 11. Optimal Thread Pool Size Pool size = Ncores + Block. Op. Ratio
  • 12. Optimal Thread Pool Size Pool size = Ncores + wait time cpu time
  • 13. DEMO 2 "Dado un rango obtener la cantidad de nr. primos agrupados por él ultimo digito"
  • 14. Danger Don't mess up with mutable shared data!
  • 15. DEMO 3 (Facebook Search)
  • 16. Collections ● eachParallel() ● eachWithIndexParallel() ● collectParallel() ● findAllParallel() ● findAnyParallel ● findParallel() ● everyParallel() ● anyParallel() ● grepParallel() ● groupByParallel() ● foldParallel() ● minParallel() ● maxParallel() ● sumParallel() ● splitParallel() ● countParallel() ● foldParallel()
  • 17. DEMO 4 (Facebook Singles)
  • 19. DEMO 5 (API Calls)
  • 20. DEMO 6 (Multiget)
  • 22. + Info ● http://gpars.org/0.12/guide/guide/single.html ● http://www.slideshare.net/paulk_asert/concurrency-with- gpars ● http://gee.cs.oswego.edu/dl/papers/fj.pdf ● http://static.googleusercontent. com/external_content/untrusted_dlcp/labs.google. com/en/us/papers/mapreduce-osdi04.pdf ● Java Concurrency in Practice (ISBN-0321349601) Pics Credits ● Stone age: http://tinyurl.com/bvnlv74 ● Steam train: http://tinyurl.com/6lp3krq ● Speed train: http://tinyurl.com/7jjc3v5 ● CPU: http://www.flickr.com/photos/olivander/149865626/