Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Berlin Apache Flink Meetup #12
Community Update & 2015 – The Year in Review
December 2015
Robert Metzger
Committer and PMC...
What happened?
• 0.10.0 and 0.10.1 released
– Event-time streaming (out of order streams, sessions)
– Master HA  no singl...
Now in master (1.0-SNAPSHOT)
flink.apache.org 2
• Kafka Connector improvements: Multiple topics,
keys, deleted messages, p...
Pending Pull Requests
• Savepoints: Trigger state checkpoints manually
and restore a job from a savepoint
• Submit Flink j...
Reading List
• Realtime dashboards with Flink and ElasticSearch
https://www.elastic.co/blog/building-real-time-dashboard-a...
Flink 2015: The journey to
mainstream
2015 – The Year in Review
flink.apache.org 5
Let the numbers speak:
GitHub
flink.apache.org 6
February 2015
December 2015
Let the numbers speak:
Meetup Groups
flink.apache.org 7
From one Meetup group to a global community
Let the facts speak:
Organizations at Flink Forward 2015
flink.apache.org 88
Let the facts speak:
Articles about Flink featured in
9
Let the facts speak:
Hadoop Summit 2016: 17 Flink talks
10
flink.apache.org 11
• Check out the slides: http://flink-
forward.org/?post_type=session
• Video recordings on YouTube, “F...
Feature Timeline 2015
• January
– Flink 0.8
• February
– Gelly
– Table API
• March
– Flink-Google Data Flow
Runner (batch)...
Feature Timeline 2015
• August
– Gelly Scala API
– DataSet.sample()
• September
– Cascading on Flink
– Scala 2.11 (and 2.1...
git Stats 2015
git log --pretty=oneline --after=1/1/2015 | wc -l
2201 commits in 2015
flink.apache.org 14
Lines of code
Co...
We are not done yet
• Flink Forward 2016
• Roadmap 2016 (pending community discussion)
– Autoscaling
– Stream SQL
– Manage...
Próxima SlideShare
Cargando en…5
×

Flink Community Update December 2015: Year in Review

829 visualizaciones

Publicado el

Community Update of Apache Flink as presented at the Flink Meetup in Berlin, Germany

Publicado en: Tecnología
  • Have u ever tried external professional writing services like ⇒ www.WritePaper.info ⇐ ? I did and I am more than satisfied.
       Responder 
    ¿Estás seguro?    No
    Tu mensaje aparecerá aquí

Flink Community Update December 2015: Year in Review

  1. 1. Berlin Apache Flink Meetup #12 Community Update & 2015 – The Year in Review December 2015 Robert Metzger Committer and PMC Member rmetzger@apache.org @rmetzger_
  2. 2. What happened? • 0.10.0 and 0.10.1 released – Event-time streaming (out of order streams, sessions) – Master HA  no single point of failure – DataStream API graduated – New monitoring interface – Offheap memory – Outer joins – Gelly Scala API • Otto released a Flink testing framework: https://github.com/ottogroup/flink-spector flink.apache.org 1
  3. 3. Now in master (1.0-SNAPSHOT) flink.apache.org 2 • Kafka Connector improvements: Multiple topics, keys, deleted messages, periodic offset commit • Flink in firewalled setups • Out of core state backend for JDBC • Major improvements in the flink-storm module • RabbitMQ Source with exactly-once • Many other operational and usability improvements
  4. 4. Pending Pull Requests • Savepoints: Trigger state checkpoints manually and restore a job from a savepoint • Submit Flink jobs through JobManager web interface • Add range partitioner with sampling (FLINK-7) • “stop” signal for streaming topologies • Monitoring: JMX beans & checkpoint metrics flink.apache.org 3
  5. 5. Reading List • Realtime dashboards with Flink and ElasticSearch https://www.elastic.co/blog/building-real-time-dashboard-applications-with-apache-flink-elasticsearch-and-kibana • Introduction to streaming windows http://flink.apache.org/news/2015/12/04/Introducing-windows.html • Flink + Storm: Run Storm topologies on Flink http://flink.apache.org/news/2015/12/11/storm-compatibility.html • New streaming use cases http://data-artisans.com/how-apache-flink-enables-new-streaming-applications-part-1/ • Streaming with Flink Intro (reading twitter data) http://blog.brakmic.com/stream-processing-with-apache-flink/ flink.apache.org 4
  6. 6. Flink 2015: The journey to mainstream 2015 – The Year in Review flink.apache.org 5
  7. 7. Let the numbers speak: GitHub flink.apache.org 6 February 2015 December 2015
  8. 8. Let the numbers speak: Meetup Groups flink.apache.org 7 From one Meetup group to a global community
  9. 9. Let the facts speak: Organizations at Flink Forward 2015 flink.apache.org 88
  10. 10. Let the facts speak: Articles about Flink featured in 9
  11. 11. Let the facts speak: Hadoop Summit 2016: 17 Flink talks 10
  12. 12. flink.apache.org 11 • Check out the slides: http://flink- forward.org/?post_type=session • Video recordings on YouTube, “Flink Forward” channel Apropos
  13. 13. Feature Timeline 2015 • January – Flink 0.8 • February – Gelly – Table API • March – Flink-Google Data Flow Runner (batch) – FlinkML • April – Python API • May – Static Code Analysis – Zeppelin Integration • June – 0.9: Akka, batch exchange, exactly-once • July – New dashboard – first HA features flink.apache.org 12
  14. 14. Feature Timeline 2015 • August – Gelly Scala API – DataSet.sample() • September – Cascading on Flink – Scala 2.11 (and 2.10) – Offheap memory – Reworked windowing – Rolling file sink – Elasticsearch sink – High Availability flink.apache.org 13 • October – Reworked Kafka Consumer – Streaming API graduation • November – Flink 0.10 – JDBC backend • December – Flink behind firewalls
  15. 15. git Stats 2015 git log --pretty=oneline --after=1/1/2015 | wc -l 2201 commits in 2015 flink.apache.org 14 Lines of code Commits per day Commits per hour
  16. 16. We are not done yet • Flink Forward 2016 • Roadmap 2016 (pending community discussion) – Autoscaling – Stream SQL – Managed Memory State … much more Let’s make 2016 the Year of Flink! flink.apache.org 15

×