Enviar búsqueda
Cargar
Breaking The Clustering Limits @ AlphaCSP JavaEdge 2007
•
Descargar como PPT, PDF
•
1 recomendación
•
727 vistas
Baruch Sadogursky
Seguir
Tecnología
Denunciar
Compartir
Denunciar
Compartir
1 de 61
Descargar ahora
Recomendados
IO is slow, memory is fast. For many applications, the main performance and scalability bottleneck is disk and network access. This session will cover strategies that can help you utilize your RAM efficiently even in a distributed environment. Discover how a clustering solution like Terracotta can help you reduce overall application latency.
From I/O To RAM
From I/O To RAM
Baruch Sadogursky
A presentation from TheEdge10 about Hadoop and Big data
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the Rescue
Shay Sofer
Persisting Your Objects In The Database World @ AlphaCSP Professional OSS Con...
Persisting Your Objects In The Database World @ AlphaCSP Professional OSS Con...
Baruch Sadogursky
Review of REST and JAX-RS, as presented by Guy Nir at AlphaCSP's JavaEdge conference, 2009.
REST and JAX-RS
REST and JAX-RS
Guy Nir
Clustering In The Wild
Clustering In The Wild
Sergio Bossa
This talk covers a number of important topics for making scalable Apache Spark programs - from RDD re-use to considerations for working with Key/Value data, why avoiding groupByKey is important and more. We also include Python specific considerations, like the difference between DataFrames/Datasets and traditional RDDs with Python. We also explore some tricks to intermix Python and JVM code for cases where the performance overhead is too high.
Improving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVM
Holden Karau
Ray (https://github.com/ray-project/ray) is a framework developed at UC Berkeley and maintained by Anyscale for building distributed AI applications. Over the last year, the broader machine learning ecosystem has been rapidly adopting Ray as the primary framework for distributed execution. In this talk, we will overview how libraries such as Horovod (https://horovod.ai/), XGBoost, and Hugging Face Transformers, have integrated with Ray. We will then showcase how Uber leverages Ray and these ecosystem integrations to simplify critical production workloads at Uber. This is a joint talk between Anyscale and Uber.
Ray and Its Growing Ecosystem
Ray and Its Growing Ecosystem
Databricks
Building Production Class Deep Learning Workflows for the Enterprise
Smart Data Conference: DL4J and DataVec
Smart Data Conference: DL4J and DataVec
Josh Patterson
Recomendados
IO is slow, memory is fast. For many applications, the main performance and scalability bottleneck is disk and network access. This session will cover strategies that can help you utilize your RAM efficiently even in a distributed environment. Discover how a clustering solution like Terracotta can help you reduce overall application latency.
From I/O To RAM
From I/O To RAM
Baruch Sadogursky
A presentation from TheEdge10 about Hadoop and Big data
TheEdge10 : Big Data is Here - Hadoop to the Rescue
TheEdge10 : Big Data is Here - Hadoop to the Rescue
Shay Sofer
Persisting Your Objects In The Database World @ AlphaCSP Professional OSS Con...
Persisting Your Objects In The Database World @ AlphaCSP Professional OSS Con...
Baruch Sadogursky
Review of REST and JAX-RS, as presented by Guy Nir at AlphaCSP's JavaEdge conference, 2009.
REST and JAX-RS
REST and JAX-RS
Guy Nir
Clustering In The Wild
Clustering In The Wild
Sergio Bossa
This talk covers a number of important topics for making scalable Apache Spark programs - from RDD re-use to considerations for working with Key/Value data, why avoiding groupByKey is important and more. We also include Python specific considerations, like the difference between DataFrames/Datasets and traditional RDDs with Python. We also explore some tricks to intermix Python and JVM code for cases where the performance overhead is too high.
Improving PySpark performance: Spark Performance Beyond the JVM
Improving PySpark performance: Spark Performance Beyond the JVM
Holden Karau
Ray (https://github.com/ray-project/ray) is a framework developed at UC Berkeley and maintained by Anyscale for building distributed AI applications. Over the last year, the broader machine learning ecosystem has been rapidly adopting Ray as the primary framework for distributed execution. In this talk, we will overview how libraries such as Horovod (https://horovod.ai/), XGBoost, and Hugging Face Transformers, have integrated with Ray. We will then showcase how Uber leverages Ray and these ecosystem integrations to simplify critical production workloads at Uber. This is a joint talk between Anyscale and Uber.
Ray and Its Growing Ecosystem
Ray and Its Growing Ecosystem
Databricks
Building Production Class Deep Learning Workflows for the Enterprise
Smart Data Conference: DL4J and DataVec
Smart Data Conference: DL4J and DataVec
Josh Patterson
Presented at JAX London 2013 With the proliferation of data sources and growing user bases, the amount of data generated requires new ways for storage and processing. Hadoop opened new possibilities, yet it falls short of instant delivery. Adding stream processing using Nathan Marz’s Storm, can overcome this delay and bridge the gap to real-time aggregation and reporting. On the Batch layer all master data is kept and is immutable. Once the base data is stored a recurring process will index the data. This process reads all master data, parses it and will create new views out of it.
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
jaxLondonConference
The combination of deep learning with Apache Spark has the potential to make a huge impact. Joseph Bradley and Xiangrui Meng share best practices for integrating popular deep learning libraries with Apache Spark. Rather than comparing deep learning systems or specific optimizations, Joseph and Xiangrui focus on issues that are common to many deep learning frameworks when running on a Spark cluster, such as optimizing cluster setup and data ingest (clusters can be configured to avoid task conflicts on GPUs and to allow using multiple GPUs per worker), configuring the cluster (setting up pipelines for efficient data ingest improves job throughput), and monitoring long-running jobs (interactive monitoring facilitates both the work of configuration and checking the stability of deep learning jobs). Joseph and Xiangrui then demonstrate the techniques using Google’s popular TensorFlow library.
Integrating Deep Learning Libraries with Apache Spark
Integrating Deep Learning Libraries with Apache Spark
Databricks
This presentation was delivered by Adarsh Pannu at IBM's Insight Conference in Nov 2015.
Apache Spark: What's under the hood
Apache Spark: What's under the hood
Adarsh Pannu
Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache Spark
Databricks
Introduction to Spark ML Pipelines Workshop slides - companion IJupyter notebooks in Python & Scala are available from my github at https://github.com/holdenk/spark-intro-ml-pipeline-workshop
Introduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines Workshop
Holden Karau
Description: We are amidst the Big Data Zeitgeist era in which data comes at us fast, in myriad forms and formats at intermittent intervals or in a continuous stream, and we need to respond to streaming data immediately. This need has created a notion of writing a streaming application that’s continuous, reacts and interacts with data in real-time. We call this continuous application, which we will discuss. Abstract: We are amidst the Big Data Zeitgeist era in which data comes at us fast, in myriad forms and formats at intermittent intervals or in a continuous stream, and we need to respond to streaming data immediately. This need has created a notion of writing a streaming application that’s continuous, reacts and interacts with data in real-time. We call this continuous application. In this talk we will explore the concepts and motivations behind the continuous application, how Structured Streaming Python APIs in Apache Spark 2.x enables writing continuous applications, examine the programming model behind Structured Streaming, and look at the APIs that support them. Through a short demo and code examples, I will demonstrate how to write an end-to-end Structured Streaming application that reacts and interacts with both real-time and historical data to perform advanced analytics using Spark SQL, DataFrames and Datasets APIs. You’ll walk away with an understanding of what’s a continuous application, appreciate the easy-to-use Structured Streaming APIs, and why Structured Streaming in Apache Spark 2.x is a step forward in developing new kinds of streaming applications.
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
Databricks
In this session, we show how to leverage CORD dataset, containing more than 400000 scientific papers on COVID and related topics, and recent advances in natural language processing and other AI techniques to generate new insights in support of the ongoing fight against this infectious disease. The idea explored in our talk is to apply modern NLP methods, such and named entity recognition (NER) and relation extraction to article’s abstracts (and, possibly, full text), to extract some meaningful insights from the text, and to enable semantically rich search over the paper corpus. We first investigate how to train NER model using Medical NER dataset from Kaggle, and specialized version of BERT (PubMedBERT) as a feature extractor, to allow automatic extraction of such entities as medical condition names, medicine names and pathogens. Entity extraction alone can provide us with some interesting findings, such as how approaches to COVID treatment evolved with time, in terms of mentioned medicines. We demonstrate how to use Azure Machine Learning for training the model. To take this investigation one step further, we also investigate the usage of pre-trained medical models, available as Text Analytics for Health service on the Microsoft Azure cloud. In addition to many entity types, it can also extract relations (such as the dosage of medicine provisioned), entity negation, and entity mapping to some well-known medical ontologies. We investigate the best way to use Azure ML at scale to score large paper collection, and to store the results.
How Machine Learning and AI Can Support the Fight Against COVID-19
How Machine Learning and AI Can Support the Fight Against COVID-19
Databricks
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
DataWorks Summit/Hadoop Summit
Presented at JAX London 2013 Worried about the future of Java? Want to see it keep moving forward? Don't be concerned. The transformation of Java is already underway. Driven by new technologies and new opportunities Java and the JVM are entering uncharted worlds and challenging old approaches. In this session learn about one such expedition in the form of an introductory talk to technology being developed by IBM. This experimental technology is exploring a new way to share data between the JVM and other runtimes.
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
jaxLondonConference
Apache Spark is a fast and general engine for distributed computing & big data processing with APIs in Scala, Java, Python, and R. This tutorial will briefly introduce PySpark (the Python API for Spark) with some hands-on-exercises combined with a quick introduction to Spark's core concepts. We will cover the obligatory wordcount example which comes in with every big-data tutorial, as well as discuss Spark's unique methods for handling node failure and other relevant internals. Then we will briefly look at how to access some of Spark's libraries (like Spark SQL & Spark ML) from Python. While Spark is available in a variety of languages this workshop will be focused on using Spark and Python together.
A really really fast introduction to PySpark - lightning fast cluster computi...
A really really fast introduction to PySpark - lightning fast cluster computi...
Holden Karau
Beyond Shuffling - Tips & Tricks for scaling your Apache Spark programs. This talk walks through a number of common mistakes which can keep our Spark programs from scaling and examines the solutions, as well as general techniques useful for moving from beyond a prof of concept to production. It covers topics like effective RDD re-use, considerations for working with key/value data, and finishes up with a preview of some of the work being done to add code generation to Spark ML.
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Holden Karau
Spark Summit 2016 talk by Costin Leau (Elastic)
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Jen Aman
This presentations has been done at Data Science & Engineering Club Meetup. https://github.com/erenavsarogullari/apache-spark-fundamentals-meetup-talk
Apache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup Talk
Eren Avşaroğulları
Spark and spark streaming internals
Spark and spark streaming internals
Spark and spark streaming internals
Sigmoid
As Spark jobs are used for more mission critical tasks, beyond exploration, it is important to have effective tools for testing. This talk expands on “Effective Testing For Spark Programs” (not required to have been seen) to discuss how to create large scale test jobs without depending on collect & parallelize which limit the sizes of datasets we can work with. Testing Spark Streaming jobs can be especially challenging, as the normal techniques for loading test data don’t work and additional work must be done to collect the results and stop streaming. We will explore the difficulties with testing Streaming Programs, options for setting up integration testing, beyond just local mode, with Spark, and also examine best practices for acceptance tests.
Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016
Holden Karau
Due to Spark, writing big data applications has never been easier…at least until they stop being easy! At Lightbend we’ve helped our customers out of a number of hidden Spark pitfalls. Some crop up often; the ever-persistent OutOfMemoryError, the confusing NoSuchMethodError, shuffle and partition management, etc. Others occur less frequently; an obscure configuration affecting SQL broadcasts, struggles with speculating, a failing stream recovery due to RDD joins, S3 file reading leading to hangs, etc. All are intriguing! In this session we will provide insights into their origins and show how you can avoid making the same mistakes. Whether you are a seasoned Spark developer or a novice, you should learn some new tips and tricks that could save you hours or even days of debugging.
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Spark Summit
Building Production Class Deep Learning Workflows for the Enterprise
Deep Learning: DL4J and DataVec
Deep Learning: DL4J and DataVec
Josh Patterson
we will see internal architecture of spark cluster i.e what is driver, worker,executer and cluster manager, how spark program will be run on cluster and what are jobs,stages and task.
Internals
Internals
Sandeep Purohit
Something really exciting and largely unnoticed is going on in the Spark ecosystem. As data scientists and engineers learn Spark, they’re actually all implicitly learning a much older, more general topic: typed functional programming. While Spark itself was built on an accumulation of powerful computer science concepts from functional programming and other areas, developers are often encountering these ideas in the context of Spark for the first time. It turns out that Spark makes an excellent platform for learning concepts like immutability, higher order and anonymous functions, laziness, and monadic operators. This talk will discuss how Spark can be used as teaching tool, to build skills in areas like typed functional programming. We’ll explore a skill-building curriculum that can be used with a data scientist or engineer who only has experience in imperative, dynamically-typed languages like Python. This curriculum introduces the core concepts of functional programming and type theory, while providing learners the opportunity to immediately apply their skills at massive scale, using the power of Spark’s painless scalability and resilience. Based on the experience of building machine learning teams at x.ai and other data-centric startups, this curriculum is the foundation of building poly-skilled, highly autonomous team members who can build scalable intelligent systems. We’ll work from foundational concepts of Scala and functional programming towards a fully implemented machine learning pipeline, all using Spark and MLlib. Unique new features of Spark like Datasets and Structured Streaming will be particularly useful in this effort. Using this approach, teams can help members in all roles learn how to use sophisticated programming techniques that ensure correctness at scale. With these skills in their toolbox, data scientists and engineers often find that building powerful machine learning systems is intuitive, easy, and even fun.
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Spark Summit
An investigation of how PostgreSQL and its latest capabilities (JSONB data type, GIN indices, Full Text Search) can be used to store, index and perform queries on structured Bibliographic Data such as MARC21/MARCXML, breaking the dependence on proprietary and arcane or obsolete software products. Talk presented at FOSDEM 2016 in Brussels on 31/01/2016. This is a very practical & hands-on presentation with example code which is certainly not optimal ;)
Using PostgreSQL with Bibliographic Data
Using PostgreSQL with Bibliographic Data
Jimmy Angelakos
Inleiding op de clustering van IT-componenten
Clustering van IT-componenten
Clustering van IT-componenten
Richard Claassens CIPPE
Session Presented at 5th IndicThreads.com Conference On Java held on 10-11 December 2010 in Pune, India WEB: http://J10.IndicThreads.com ------------ Rising power dissipation in microprocessor chips is leading to a trend towards increasing the number of cores on a chip (multi-core processors) rather than increasing clock frequency as the primary basis for increasing system performance. Consequently the number of threads in commodity hardware has also exploded. This leads to complexity in designing and configuring high performance Java applications that make effective use of new hardware. In this talk we provide a summary of the changes happening in the multi-core world and subsequently discuss about some of the JVM features which exploit the multi-core capabilities of the underlying hardware. We also explain techniques to analyze and optimize your application for highly concurrent systems. Key topics include an overview of Java Virtual Machine features & configuration, ways to correctly leverage java.util.concurrent package to achieve enhanced parallelism for applications in a multi-core environment, operating system issues, virtualization, Java code optimizations and useful profiling tools and techniques. Takeaways for the Audience Attendees will leave with a better understanding of the new multi-core world, understanding of Java Virtual Machine features which exploit mulit-core and the techniques they can apply to ensure their Java applications run well in mulit-core environment.
Optimizing your java applications for multi core hardware
Optimizing your java applications for multi core hardware
IndicThreads
Más contenido relacionado
La actualidad más candente
Presented at JAX London 2013 With the proliferation of data sources and growing user bases, the amount of data generated requires new ways for storage and processing. Hadoop opened new possibilities, yet it falls short of instant delivery. Adding stream processing using Nathan Marz’s Storm, can overcome this delay and bridge the gap to real-time aggregation and reporting. On the Batch layer all master data is kept and is immutable. Once the base data is stored a recurring process will index the data. This process reads all master data, parses it and will create new views out of it.
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
jaxLondonConference
The combination of deep learning with Apache Spark has the potential to make a huge impact. Joseph Bradley and Xiangrui Meng share best practices for integrating popular deep learning libraries with Apache Spark. Rather than comparing deep learning systems or specific optimizations, Joseph and Xiangrui focus on issues that are common to many deep learning frameworks when running on a Spark cluster, such as optimizing cluster setup and data ingest (clusters can be configured to avoid task conflicts on GPUs and to allow using multiple GPUs per worker), configuring the cluster (setting up pipelines for efficient data ingest improves job throughput), and monitoring long-running jobs (interactive monitoring facilitates both the work of configuration and checking the stability of deep learning jobs). Joseph and Xiangrui then demonstrate the techniques using Google’s popular TensorFlow library.
Integrating Deep Learning Libraries with Apache Spark
Integrating Deep Learning Libraries with Apache Spark
Databricks
This presentation was delivered by Adarsh Pannu at IBM's Insight Conference in Nov 2015.
Apache Spark: What's under the hood
Apache Spark: What's under the hood
Adarsh Pannu
Memory management is at the heart of any data-intensive system. Spark, in particular, must arbitrate memory allocation between two main use cases: buffering intermediate data for processing (execution) and caching user data (storage). This talk will take a deep dive through the memory management designs adopted in Spark since its inception and discuss their performance and usability implications for the end user.
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache Spark
Databricks
Introduction to Spark ML Pipelines Workshop slides - companion IJupyter notebooks in Python & Scala are available from my github at https://github.com/holdenk/spark-intro-ml-pipeline-workshop
Introduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines Workshop
Holden Karau
Description: We are amidst the Big Data Zeitgeist era in which data comes at us fast, in myriad forms and formats at intermittent intervals or in a continuous stream, and we need to respond to streaming data immediately. This need has created a notion of writing a streaming application that’s continuous, reacts and interacts with data in real-time. We call this continuous application, which we will discuss. Abstract: We are amidst the Big Data Zeitgeist era in which data comes at us fast, in myriad forms and formats at intermittent intervals or in a continuous stream, and we need to respond to streaming data immediately. This need has created a notion of writing a streaming application that’s continuous, reacts and interacts with data in real-time. We call this continuous application. In this talk we will explore the concepts and motivations behind the continuous application, how Structured Streaming Python APIs in Apache Spark 2.x enables writing continuous applications, examine the programming model behind Structured Streaming, and look at the APIs that support them. Through a short demo and code examples, I will demonstrate how to write an end-to-end Structured Streaming application that reacts and interacts with both real-time and historical data to perform advanced analytics using Spark SQL, DataFrames and Datasets APIs. You’ll walk away with an understanding of what’s a continuous application, appreciate the easy-to-use Structured Streaming APIs, and why Structured Streaming in Apache Spark 2.x is a step forward in developing new kinds of streaming applications.
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
Databricks
In this session, we show how to leverage CORD dataset, containing more than 400000 scientific papers on COVID and related topics, and recent advances in natural language processing and other AI techniques to generate new insights in support of the ongoing fight against this infectious disease. The idea explored in our talk is to apply modern NLP methods, such and named entity recognition (NER) and relation extraction to article’s abstracts (and, possibly, full text), to extract some meaningful insights from the text, and to enable semantically rich search over the paper corpus. We first investigate how to train NER model using Medical NER dataset from Kaggle, and specialized version of BERT (PubMedBERT) as a feature extractor, to allow automatic extraction of such entities as medical condition names, medicine names and pathogens. Entity extraction alone can provide us with some interesting findings, such as how approaches to COVID treatment evolved with time, in terms of mentioned medicines. We demonstrate how to use Azure Machine Learning for training the model. To take this investigation one step further, we also investigate the usage of pre-trained medical models, available as Text Analytics for Health service on the Microsoft Azure cloud. In addition to many entity types, it can also extract relations (such as the dosage of medicine provisioned), entity negation, and entity mapping to some well-known medical ontologies. We investigate the best way to use Azure ML at scale to score large paper collection, and to store the results.
How Machine Learning and AI Can Support the Fight Against COVID-19
How Machine Learning and AI Can Support the Fight Against COVID-19
Databricks
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
DataWorks Summit/Hadoop Summit
Presented at JAX London 2013 Worried about the future of Java? Want to see it keep moving forward? Don't be concerned. The transformation of Java is already underway. Driven by new technologies and new opportunities Java and the JVM are entering uncharted worlds and challenging old approaches. In this session learn about one such expedition in the form of an introductory talk to technology being developed by IBM. This experimental technology is exploring a new way to share data between the JVM and other runtimes.
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
jaxLondonConference
Apache Spark is a fast and general engine for distributed computing & big data processing with APIs in Scala, Java, Python, and R. This tutorial will briefly introduce PySpark (the Python API for Spark) with some hands-on-exercises combined with a quick introduction to Spark's core concepts. We will cover the obligatory wordcount example which comes in with every big-data tutorial, as well as discuss Spark's unique methods for handling node failure and other relevant internals. Then we will briefly look at how to access some of Spark's libraries (like Spark SQL & Spark ML) from Python. While Spark is available in a variety of languages this workshop will be focused on using Spark and Python together.
A really really fast introduction to PySpark - lightning fast cluster computi...
A really really fast introduction to PySpark - lightning fast cluster computi...
Holden Karau
Beyond Shuffling - Tips & Tricks for scaling your Apache Spark programs. This talk walks through a number of common mistakes which can keep our Spark programs from scaling and examines the solutions, as well as general techniques useful for moving from beyond a prof of concept to production. It covers topics like effective RDD re-use, considerations for working with key/value data, and finishes up with a preview of some of the work being done to add code generation to Spark ML.
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Holden Karau
Spark Summit 2016 talk by Costin Leau (Elastic)
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Jen Aman
This presentations has been done at Data Science & Engineering Club Meetup. https://github.com/erenavsarogullari/apache-spark-fundamentals-meetup-talk
Apache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup Talk
Eren Avşaroğulları
Spark and spark streaming internals
Spark and spark streaming internals
Spark and spark streaming internals
Sigmoid
As Spark jobs are used for more mission critical tasks, beyond exploration, it is important to have effective tools for testing. This talk expands on “Effective Testing For Spark Programs” (not required to have been seen) to discuss how to create large scale test jobs without depending on collect & parallelize which limit the sizes of datasets we can work with. Testing Spark Streaming jobs can be especially challenging, as the normal techniques for loading test data don’t work and additional work must be done to collect the results and stop streaming. We will explore the difficulties with testing Streaming Programs, options for setting up integration testing, beyond just local mode, with Spark, and also examine best practices for acceptance tests.
Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016
Holden Karau
Due to Spark, writing big data applications has never been easier…at least until they stop being easy! At Lightbend we’ve helped our customers out of a number of hidden Spark pitfalls. Some crop up often; the ever-persistent OutOfMemoryError, the confusing NoSuchMethodError, shuffle and partition management, etc. Others occur less frequently; an obscure configuration affecting SQL broadcasts, struggles with speculating, a failing stream recovery due to RDD joins, S3 file reading leading to hangs, etc. All are intriguing! In this session we will provide insights into their origins and show how you can avoid making the same mistakes. Whether you are a seasoned Spark developer or a novice, you should learn some new tips and tricks that could save you hours or even days of debugging.
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Spark Summit
Building Production Class Deep Learning Workflows for the Enterprise
Deep Learning: DL4J and DataVec
Deep Learning: DL4J and DataVec
Josh Patterson
we will see internal architecture of spark cluster i.e what is driver, worker,executer and cluster manager, how spark program will be run on cluster and what are jobs,stages and task.
Internals
Internals
Sandeep Purohit
Something really exciting and largely unnoticed is going on in the Spark ecosystem. As data scientists and engineers learn Spark, they’re actually all implicitly learning a much older, more general topic: typed functional programming. While Spark itself was built on an accumulation of powerful computer science concepts from functional programming and other areas, developers are often encountering these ideas in the context of Spark for the first time. It turns out that Spark makes an excellent platform for learning concepts like immutability, higher order and anonymous functions, laziness, and monadic operators. This talk will discuss how Spark can be used as teaching tool, to build skills in areas like typed functional programming. We’ll explore a skill-building curriculum that can be used with a data scientist or engineer who only has experience in imperative, dynamically-typed languages like Python. This curriculum introduces the core concepts of functional programming and type theory, while providing learners the opportunity to immediately apply their skills at massive scale, using the power of Spark’s painless scalability and resilience. Based on the experience of building machine learning teams at x.ai and other data-centric startups, this curriculum is the foundation of building poly-skilled, highly autonomous team members who can build scalable intelligent systems. We’ll work from foundational concepts of Scala and functional programming towards a fully implemented machine learning pipeline, all using Spark and MLlib. Unique new features of Spark like Datasets and Structured Streaming will be particularly useful in this effort. Using this approach, teams can help members in all roles learn how to use sophisticated programming techniques that ensure correctness at scale. With these skills in their toolbox, data scientists and engineers often find that building powerful machine learning systems is intuitive, easy, and even fun.
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Spark Summit
An investigation of how PostgreSQL and its latest capabilities (JSONB data type, GIN indices, Full Text Search) can be used to store, index and perform queries on structured Bibliographic Data such as MARC21/MARCXML, breaking the dependence on proprietary and arcane or obsolete software products. Talk presented at FOSDEM 2016 in Brussels on 31/01/2016. This is a very practical & hands-on presentation with example code which is certainly not optimal ;)
Using PostgreSQL with Bibliographic Data
Using PostgreSQL with Bibliographic Data
Jimmy Angelakos
La actualidad más candente
(20)
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
A real-time architecture using Hadoop & Storm - Nathan Bijnens & Geert Van La...
Integrating Deep Learning Libraries with Apache Spark
Integrating Deep Learning Libraries with Apache Spark
Apache Spark: What's under the hood
Apache Spark: What's under the hood
Deep Dive: Memory Management in Apache Spark
Deep Dive: Memory Management in Apache Spark
Introduction to Spark ML Pipelines Workshop
Introduction to Spark ML Pipelines Workshop
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
Writing Continuous Applications with Structured Streaming Python APIs in Apac...
How Machine Learning and AI Can Support the Fight Against COVID-19
How Machine Learning and AI Can Support the Fight Against COVID-19
Inferno Scalable Deep Learning on Spark
Inferno Scalable Deep Learning on Spark
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
Packed Objects: Fast Talking Java Meets Native Code - Steve Poole (IBM)
A really really fast introduction to PySpark - lightning fast cluster computi...
A really really fast introduction to PySpark - lightning fast cluster computi...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Elasticsearch And Apache Lucene For Apache Spark And MLlib
Apache Spark Fundamentals Meetup Talk
Apache Spark Fundamentals Meetup Talk
Spark and spark streaming internals
Spark and spark streaming internals
Beyond parallelize and collect - Spark Summit East 2016
Beyond parallelize and collect - Spark Summit East 2016
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Problem Solving Recipes Learned from Supporting Spark: Spark Summit East talk...
Deep Learning: DL4J and DataVec
Deep Learning: DL4J and DataVec
Internals
Internals
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Spark as the Gateway Drug to Typed Functional Programming: Spark Summit East ...
Using PostgreSQL with Bibliographic Data
Using PostgreSQL with Bibliographic Data
Similar a Breaking The Clustering Limits @ AlphaCSP JavaEdge 2007
Inleiding op de clustering van IT-componenten
Clustering van IT-componenten
Clustering van IT-componenten
Richard Claassens CIPPE
Session Presented at 5th IndicThreads.com Conference On Java held on 10-11 December 2010 in Pune, India WEB: http://J10.IndicThreads.com ------------ Rising power dissipation in microprocessor chips is leading to a trend towards increasing the number of cores on a chip (multi-core processors) rather than increasing clock frequency as the primary basis for increasing system performance. Consequently the number of threads in commodity hardware has also exploded. This leads to complexity in designing and configuring high performance Java applications that make effective use of new hardware. In this talk we provide a summary of the changes happening in the multi-core world and subsequently discuss about some of the JVM features which exploit the multi-core capabilities of the underlying hardware. We also explain techniques to analyze and optimize your application for highly concurrent systems. Key topics include an overview of Java Virtual Machine features & configuration, ways to correctly leverage java.util.concurrent package to achieve enhanced parallelism for applications in a multi-core environment, operating system issues, virtualization, Java code optimizations and useful profiling tools and techniques. Takeaways for the Audience Attendees will leave with a better understanding of the new multi-core world, understanding of Java Virtual Machine features which exploit mulit-core and the techniques they can apply to ensure their Java applications run well in mulit-core environment.
Optimizing your java applications for multi core hardware
Optimizing your java applications for multi core hardware
IndicThreads
Session Presented at 5th IndicThreads.com Conference On Java held on 10-11 December 2010 in Pune, India WEB: http://J10.IndicThreads.com ------------ Enterprise applications typically comprise of multi layered stacks including the application modules, application servers, the Java Virtual Machine and the underlying Operating System. Consequently the performance of these applications are a factor of these different layers. In the eventuality of a performance problem, it is often difficult to determine the starting point for diagnosis. The Java Virtual Machine is the ‘engine’ for most of the applications. It is responsible broadly for efficient execution and memory management of applications. End users have difficulty attributing the effect of the JVM on the performance of the application. This is because usually JVM is viewed as a ‘black box’. This talk provides an insight into the key subsystems of the JVM by looking under the hood of a high performance JVM. It ventures onto talk about approaches and techniques for analyzing performance issues. It concludes by introducing the audience to a tool called the “Health Center” which is useful for evaluating and comprehending the JVM behavior of a running application in an unobtrusive, lightweight manner. Takeaways for the Audience A better understanding of key JVM components, approaches and techniques to diagnose performance issues and performance evaluation using the Health Center
Best Practices for performance evaluation and diagnosis of Java Applications ...
Best Practices for performance evaluation and diagnosis of Java Applications ...
IndicThreads
Presentation on J2EE batch process design, tuning and performance.
J2EE Batch Processing
J2EE Batch Processing
Chris Adkin
第四回SCDN
Java one 2010
Java one 2010
scdn
An introduction to real-time java. www.denizoguz.com
Introduction to Real Time Java
Introduction to Real Time Java
Deniz Oguz
Java Core | Modern Java Concurrency | Martijn Verburg & Ben Evans
Java Core | Modern Java Concurrency | Martijn Verburg & Ben Evans
JAX London
Apresentação de Cesario Ramos - 1º encontro PT.JUG.
Lightweight Grids With Terracotta
Lightweight Grids With Terracotta
PT.JUG
Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
ConSanFrancisco123
Use case of the usage of Apache Spark @Windward Ltd. Video lecture on YouTube: https://www.youtube.com/watch?v=rPO6P5YIKUI Showing the domain of the company, A short introduction of Apache Spark, And the Tool Box used @Windward Ltd to form a working production Spark Data Pipeline.
Spark to Production @Windward
Spark to Production @Windward
Demi Ben-Ari
BeJUG meetup presentation
How Java 19 Influences the Future of Your High-Scale Applications .pdf
How Java 19 Influences the Future of Your High-Scale Applications .pdf
Ana-Maria Mihalceanu
Small Presentation about Terracotta Network Attached Memory(NAM).
Terracotta DSO
Terracotta DSO
Khurram Mahmood
Slides from my presentation about Shopzilla's concurrency strategies to the Pasadena Java User's Group on April 26, 2010. This is essentially the same material as covered by my colleague Rodney Barlow in an earlier presentation http://www.slideshare.net/rodneypbarlow/shopzilla-on-concurrency, with a few minor tweaks.
Shopzilla On Concurrency
Shopzilla On Concurrency
Will Gage
This session aims to establish applications running against distributed and scalable system, or as we know cloud computing system. We will introduce you not only briefing of Hazelcast but also deeper kernel of it, and how it works with Spark, the most famous Map-reduce library. Furthermore, we will introduce another in-memory cache called Apache Ignite and compare it with Hazelcast to see what's the difference between them. In the end, we will give a demonstration showing how Hazelcast and Spark work together well to form a cloud-base service which is distributed, flexible, reliable, available, scalable and stable. You can find demo code here: https://github.com/CyberJos/jcconf2016-hazelcast-spark https://cyberjos.blog/java/seminar/jcconf-2016-cloud-computing-applications-hazelcast-spark-and-ignite/
JCConf 2016 - Cloud Computing Applications - Hazelcast, Spark and Ignite
JCConf 2016 - Cloud Computing Applications - Hazelcast, Spark and Ignite
Joseph Kuo
Abstract Concurrency is everywhere. Prior to Java 5, concurrency was difficult and error prone. Since Java 5, it's far more prevalent in our application code, and through time it's been lurking in open-source frameworks and containers. Concurrency is also a fundamental part of Shopzilla's web-site and services ecosystem. Introduction Rod Barlow from Shopzilla will explore a brief history of concurrency, and the key concurrency features and techniques provided by the Java API since Java 5. Topics covered include Immutability, Atomic References, Blocking Queues, Locks and Deadlocks. Also covered is Concurrency in Frameworks, and Shopzilla's Website Concurrency Framework, including Thread Pools, Executors and Futures.
Shopzilla On Concurrency
Shopzilla On Concurrency
Rodney Barlow
How can we use Lambdas in Java 8 in a low latency context? What are some of the performance consideration in using Java 8.
Low latency in java 8 v5
Low latency in java 8 v5
Peter Lawrey
This is Felix' talk at the 2010 edition of MySQL Con in Santa Clara, CA
The Adventure: BlackRay as a Storage Engine
The Adventure: BlackRay as a Storage Engine
fschupp
Sharing 4 years of experience about node.js - A google chrome V8 engine javascript based web server technology. This slide covers about wide range of knowledge about node.js learned from 4 years of production, experiment, test & failures 4년 동안 node.js 서버 프로그래밍을 경험한 내용을 간략하게 정리해 보았습니다. node.js 를 접하시는 분들에게 도움이 되었으면 합니다.
node.js 실무 - node js in practice by Jesang Yoon
node.js 실무 - node js in practice by Jesang Yoon
Jesang Yoon
11g R2
11g R2
afa reg
Spring Boot 3 And Beyond presented by Dan Vega at SpringOne Tour Charles Schwab 2023.
Spring Boot 3 And Beyond
Spring Boot 3 And Beyond
VMware Tanzu
Similar a Breaking The Clustering Limits @ AlphaCSP JavaEdge 2007
(20)
Clustering van IT-componenten
Clustering van IT-componenten
Optimizing your java applications for multi core hardware
Optimizing your java applications for multi core hardware
Best Practices for performance evaluation and diagnosis of Java Applications ...
Best Practices for performance evaluation and diagnosis of Java Applications ...
J2EE Batch Processing
J2EE Batch Processing
Java one 2010
Java one 2010
Introduction to Real Time Java
Introduction to Real Time Java
Java Core | Modern Java Concurrency | Martijn Verburg & Ben Evans
Java Core | Modern Java Concurrency | Martijn Verburg & Ben Evans
Lightweight Grids With Terracotta
Lightweight Grids With Terracotta
Clustered Architecture Patterns Delivering Scalability And Availability
Clustered Architecture Patterns Delivering Scalability And Availability
Spark to Production @Windward
Spark to Production @Windward
How Java 19 Influences the Future of Your High-Scale Applications .pdf
How Java 19 Influences the Future of Your High-Scale Applications .pdf
Terracotta DSO
Terracotta DSO
Shopzilla On Concurrency
Shopzilla On Concurrency
JCConf 2016 - Cloud Computing Applications - Hazelcast, Spark and Ignite
JCConf 2016 - Cloud Computing Applications - Hazelcast, Spark and Ignite
Shopzilla On Concurrency
Shopzilla On Concurrency
Low latency in java 8 v5
Low latency in java 8 v5
The Adventure: BlackRay as a Storage Engine
The Adventure: BlackRay as a Storage Engine
node.js 실무 - node js in practice by Jesang Yoon
node.js 실무 - node js in practice by Jesang Yoon
11g R2
11g R2
Spring Boot 3 And Beyond
Spring Boot 3 And Beyond
Más de Baruch Sadogursky
So, you want to update the software for your user, be it the nodes in your K8s cluster, a browser on user’s desktop, an app in user’s smartphone or even a user’s car. What can possibly go wrong? In this talk, we’ll analyze real-world software update fails and how multiple DevOps patterns, that fit a variety of scenarios, could have saved the developers. Manually making sure that everything works before sending an update and expecting the user to do acceptance tests before they update is most definitely not on the list of such patterns. Join us for some awesome and scary continuous update horror stories and some obvious (and some not so obvious) proven ideas for improvement and best practices you can start following tomorrow.
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
Baruch Sadogursky
So, you want to update the software for your user, be it the nodes in your K8s cluster, a browser on user’s desktop, an app in user’s smartphone or even a user’s car. What can possibly go wrong? In this talk, we’ll analyze real-world software update fails and how multiple DevOps patterns, that fit a variety of scenarios, could have saved the developers. Manually making sure that everything works before sending an update and expecting the user to do acceptance tests before they update is most definitely not on the list of such patterns. Join us for some awesome and scary continuous update horror stories and some obvious (and some not so obvious) proven ideas for improvement and best practices you can start following tomorrow.
DevOps Patterns & Antipatterns for Continuous Software Updates @ DevOps.com A...
DevOps Patterns & Antipatterns for Continuous Software Updates @ DevOps.com A...
Baruch Sadogursky
In this talk, we’ll take you to a scaling journey, from 3 developers to a 100. We’ll talk about the challenges each milestone in this growth brings, both technological and methodological, and how to solve those challenges using the right mix of people, the right selection of tools and the correctly crafted process. The speakers excel in the different aspects of this triangle and went through this journey (more than once) themselves. And the fun and entertaining presentation as a Greek tragedy can’t hurt, can it?
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Oracle Code NY...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Oracle Code NY...
Baruch Sadogursky
Devops is usually viewed from a traditional perspective of a collaboration of Dev, Ops, and QA, driven by the change in Culture, People, and Process. But how do you know where you stand and where to move? As in almost any field, data and metrics give you the gauges and instruments. In this talk, we’ll talk about the key measurements for the DevOps transformation process and provide you with 3 metrics you can start measuring tomorrow.
Data driven devops as presented at QCon London 2018
Data driven devops as presented at QCon London 2018
Baruch Sadogursky
We asked the Fortune 500 software delivery leaders what holds them back. This talk is the analysis of their insights on what bottlenecks they encountered in their DevOps journey.
A Research Study Into DevOps Bottlenecks as presented at Oracle Code LA 2018
A Research Study Into DevOps Bottlenecks as presented at Oracle Code LA 2018
Baruch Sadogursky
You know what to expect by now: funny and puzzling questions about Java 8 and Java 9, JFrog t-shirts are airborne, the usual combo of learning and fun ahead!
Java Puzzlers NG S03 a DevNexus 2018
Java Puzzlers NG S03 a DevNexus 2018
Baruch Sadogursky
Do you always know what’s going on with your product artifacts since the moment they are built by the CI server from Git sources all the way to being deployed by Helm into Kuberenetes? In this talk, we will show how to build a reliable and transparent pipeline from code to cluster using Git, Artifactory, Docker, Kubernetes, and Helm. We’ll show how you such a pipeline can help you answer the big questions: What to deploy, What is deployed, and what is this artifact that I am looking for. This kind of transparency is critical for today’s environments, and Kubernetes with Helm shouldn’t be an exception.
Where the Helm are your binaries? as presented at Canada Kubernetes Meetups
Where the Helm are your binaries? as presented at Canada Kubernetes Meetups
Baruch Sadogursky
By Baruch Sadogursky Devops is usually viewed from a traditional perspective of a collaboration of Dev, Ops and QA, driven by the change in Culture, People and Process. But how do you know where you stand and were to move? As in almost any field, data and metrics give you the gauges and instruments. In this talk we’ll talk about the key measurements for the DevOps transformation process and provide you with 3 metrics you can start measuring tomorrow.
Data driven devops as presented at Codemash 2018
Data driven devops as presented at Codemash 2018
Baruch Sadogursky
By Baruch Sadogursky We asked the Fortune 500 software delivery leaders what holds them back. This talk is the analysis of their insights on what bottlenecks they encountered in their DevOps journey.
A Research Study into DevOps Bottlenecks as presented at Codemash 2018
A Research Study into DevOps Bottlenecks as presented at Codemash 2018
Baruch Sadogursky
By Baruch Sadogursky There are three hard things in computer science: cache invalidation, naming things, and off-by-one errors. This session tackles naming, especially Docker version naming. Labels, tags, checksums...how should you use them to keep track of Docker versions? What about dev versus prod images—how best to distinguish those? What about the “latest” tag? What about cleanup? Could we do more? Versioning often seems like a simple problem, but when you have a tool that gives you as much power and flexibility as Docker does, it often helps to develop guidelines. The presentation examines the tools available for managing Docker images and some simple patterns you can employ in various use cases for CI/CD to keep track of your containers.
Best Practices for Managing Docker Versions as presented at JavaOne 2017
Best Practices for Managing Docker Versions as presented at JavaOne 2017
Baruch Sadogursky
Debugging applications in production is like being the detective in a crime movie. Especially with microservices. Especially with containers. Especially in the cloud. Trying to see what’s going on in a production deployment at scale is impossible without proper tools! Google has spent over a decade deploying containerized Java applications at unprecedented scale and the infrastructure and tools developed by Google have made it uniquely possible to manage, troubleshoot, and debug, at scale. Join this session to see how you can diagnose and troubleshoot production issues w/ out of the box Kubernetes tools, as well as getting insight from the ecosystem with Weave Scope, JFrog Artifactory & Stackdriver tools.
Troubleshooting & Debugging Production Microservices in Kubernetes as present...
Troubleshooting & Debugging Production Microservices in Kubernetes as present...
Baruch Sadogursky
As in a good Greek Tragedy, scaling devops to big teams has 3 stages and usually end badly. In this play (it’s more than a talk!) we’ll present you with Pentagon Inc, and their way to scaling devops from a team of 3 engineers to a team of 100 (spoiler – it’s painful!)
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Devoxx 2017
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Devoxx 2017
Baruch Sadogursky
In this session we will compare and contrast the experience of implementing voice user interface for the two market leader voice activated assistants. Both are extendable, both have Java APIs, but which is better? Two speakers, two laptops, two IDEs writing Java code to implement the same Alexa Skill and Google Home Action and you pick the winner!
Amazon Alexa Skills vs Google Home Actions, the Big Java VUI Faceoff as prese...
Amazon Alexa Skills vs Google Home Actions, the Big Java VUI Faceoff as prese...
Baruch Sadogursky
As in a good Greek Tragedy, scaling devops to big teams has 3 stages and usually end badly. In this play (it’s more than a talk!) we’ll present you with Pentagon Inc, and their way to scaling devops from a team of 3 engineers to a team of 100 (spoiler – it’s painful!)
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at DevOps Days Be...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at DevOps Days Be...
Baruch Sadogursky
Moar puzzlers! The more we work with Java 8, the more we go into the rabbit hole. Did they add all those streams, lambdas, monads, Optionals and CompletableFutures only to confuse us? It surely looks so! And Java 9 that heads our way brings even more of what we like the most, more puzzlers, of course! In this season we as usual have a great batch of the best Java WTF, great jokes to present them and great prizes for the winners!
Java Puzzlers NG S02: Down the Rabbit Hole as it was presented at The Pittsbu...
Java Puzzlers NG S02: Down the Rabbit Hole as it was presented at The Pittsbu...
Baruch Sadogursky
As in a good Greek Tragedy, scaling devops to big teams has 3 stages and usually end badly. In this play (it’s more than a talk!) we’ll present you with Pentagon Inc, and their way to scaling devops from a team of 3 engineers to a team of 100 (spoiler – it’s painful!)
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at The Pittsburgh...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at The Pittsburgh...
Baruch Sadogursky
Developer relations strategy is often an afterthought. This session’s speaker asks whether that’s OK and gets the opinion of DevRel leaders from companies large and small.
Let’s Wing It: A Study in DevRel Strategy
Let’s Wing It: A Study in DevRel Strategy
Baruch Sadogursky
In this talk, Baruch Sadogursky presents the challenges of a high demand SaaS product incident triage at scale, as well as discuss the sources of log items, including the platform, tenants and other types of log sources. He will show practical examples of collector and filters configuration and will take you through a number of real world examples of problems investigations using Artifactory and Sumo Logic.
Log Driven First Class Customer Support at Scale
Log Driven First Class Customer Support at Scale
Baruch Sadogursky
No relationship in DevOps is more important than that between your CI/CD server and your Binary Repository. Jenkins has long been the go-to server for CI/CD, and JFrog Artifactory has long been one of the most popular integrations with it. This webinar focuses on the new features of the integration, leveraging the Jenkins Pipeline DSL for infrastructure-as-code of your favorite artifactory features whether it be generic, maven, gradle or Docker, and will show an end-to-end example of pipelines across multiple technologies and how powerful these new capabilities are.
[Webinar] The Frog And The Butler: CI Pipelines For Modern DevOps
[Webinar] The Frog And The Butler: CI Pipelines For Modern DevOps
Baruch Sadogursky
While Docker has enabled an unprecedented velocity of software production, it is all too easy to spin out of control. A promotion-based model is required to control and track the flow of Docker images as much as it is required for a traditional software development lifecycle. New tools often introduce new paradigms. We will examine the patterns and the antipatterns for Docker image management, and what impact the new tools have on the battle-proven paradigms of the software development lifecycle.
Patterns and antipatterns in Docker image lifecycle as was presented at DC Do...
Patterns and antipatterns in Docker image lifecycle as was presented at DC Do...
Baruch Sadogursky
Más de Baruch Sadogursky
(20)
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
DevOps Patterns & Antipatterns for Continuous Software Updates @ NADOG April ...
DevOps Patterns & Antipatterns for Continuous Software Updates @ DevOps.com A...
DevOps Patterns & Antipatterns for Continuous Software Updates @ DevOps.com A...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Oracle Code NY...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Oracle Code NY...
Data driven devops as presented at QCon London 2018
Data driven devops as presented at QCon London 2018
A Research Study Into DevOps Bottlenecks as presented at Oracle Code LA 2018
A Research Study Into DevOps Bottlenecks as presented at Oracle Code LA 2018
Java Puzzlers NG S03 a DevNexus 2018
Java Puzzlers NG S03 a DevNexus 2018
Where the Helm are your binaries? as presented at Canada Kubernetes Meetups
Where the Helm are your binaries? as presented at Canada Kubernetes Meetups
Data driven devops as presented at Codemash 2018
Data driven devops as presented at Codemash 2018
A Research Study into DevOps Bottlenecks as presented at Codemash 2018
A Research Study into DevOps Bottlenecks as presented at Codemash 2018
Best Practices for Managing Docker Versions as presented at JavaOne 2017
Best Practices for Managing Docker Versions as presented at JavaOne 2017
Troubleshooting & Debugging Production Microservices in Kubernetes as present...
Troubleshooting & Debugging Production Microservices in Kubernetes as present...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Devoxx 2017
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at Devoxx 2017
Amazon Alexa Skills vs Google Home Actions, the Big Java VUI Faceoff as prese...
Amazon Alexa Skills vs Google Home Actions, the Big Java VUI Faceoff as prese...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at DevOps Days Be...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at DevOps Days Be...
Java Puzzlers NG S02: Down the Rabbit Hole as it was presented at The Pittsbu...
Java Puzzlers NG S02: Down the Rabbit Hole as it was presented at The Pittsbu...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at The Pittsburgh...
DevOps @Scale (Greek Tragedy in 3 Acts) as it was presented at The Pittsburgh...
Let’s Wing It: A Study in DevRel Strategy
Let’s Wing It: A Study in DevRel Strategy
Log Driven First Class Customer Support at Scale
Log Driven First Class Customer Support at Scale
[Webinar] The Frog And The Butler: CI Pipelines For Modern DevOps
[Webinar] The Frog And The Butler: CI Pipelines For Modern DevOps
Patterns and antipatterns in Docker image lifecycle as was presented at DC Do...
Patterns and antipatterns in Docker image lifecycle as was presented at DC Do...
Último
Read about the journey the Adobe Experience Manager team has gone through in order to become and scale API-first throughout the organisation.
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
Radu Cotescu
Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Safe Software
Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
apidays
The value of a flexible API Management solution for Open Banking Steve Melan, Manager for IT Innovation and Architecture - State's and Saving's Bank of Luxembourg Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
apidays
Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
debabhi2
Copy of the slides presented by Matt Robison to the SFWelly Salesforce user group community on May 2 2024. The audience was truly international with attendees from at least 4 different countries joining online. Matt is an expert in data cloud and this was a brilliant session.
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
writing some innovation for development and search
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
I've been in the field of "Cyber Security" in its many incarnations for about 25 years. In that time I've learned some lessons, some the hard way. Here are my slides presented at BSides New Orleans in April 2024.
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Rafal Los
The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
apidays
Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
Igalia
ICT role in 21 century education. How to ICT help in education
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
jfdjdjcjdnsjd
The presentation explores the development and application of artificial intelligence (AI) from its inception to its current status in the modern world. The term "artificial intelligence" was first coined by John McCarthy in 1956 to describe efforts to develop computer programs capable of performing tasks that typically require human intelligence. This concept was first introduced at a conference held at Dartmouth College, where programs demonstrated capabilities such as playing chess, proving theorems, and interpreting texts. In the early stages, Alan Turing contributed to the field by defining intelligence as the ability of a being to respond to certain questions intelligently, proposing what is now known as the Turing Test to evaluate the presence of intelligent behavior in machines. As the decades progressed, AI evolved significantly. The 1980s focused on machine learning, teaching computers to learn from data, leading to the development of models that could improve their performance based on their experiences. The 1990s and 2000s saw further advances in algorithms and computational power, which allowed for more sophisticated data analysis techniques, including data mining. By the 2010s, the proliferation of big data and the refinement of deep learning techniques enabled AI to become mainstream. Notable milestones included the success of Google's AlphaGo and advancements in autonomous vehicles by companies like Tesla and Waymo. A major theme of the presentation is the application of generative AI, which has been used for tasks such as natural language text generation, translation, and question answering. Generative AI uses large datasets to train models that can then produce new, coherent pieces of text or other media. The presentation also discusses the ethical implications and the need for regulation in AI, highlighting issues such as privacy, bias, and the potential for misuse. These concerns have prompted calls for comprehensive regulations to ensure the safe and equitable use of AI technologies. Artificial intelligence has also played a significant role in healthcare, particularly highlighted during the COVID-19 pandemic, where it was used in drug discovery, vaccine development, and analyzing the spread of the virus. The capabilities of AI in healthcare are vast, ranging from medical diagnostics to personalized medicine, demonstrating the technology's potential to revolutionize fields beyond just technical or consumer applications. In conclusion, AI continues to be a rapidly evolving field with significant implications for various aspects of society. The development from theoretical concepts to real-world applications illustrates both the potential benefits and the challenges that come with integrating advanced technologies into everyday life. The ongoing discussion about AI ethics and regulation underscores the importance of managing these technologies responsibly to maximize their their benefits while minimizing potential harms.
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
A Principled Technologies deployment guide Conclusion Deploying VMware Cloud Foundation 5.1 on next gen Dell PowerEdge servers brings together critical virtualization capabilities and high-performing hardware infrastructure. Relying on our hands-on experience, this deployment guide offers a comprehensive roadmap that can guide your organization through the seamless integration of advanced VMware cloud solutions with the performance and reliability of Dell PowerEdge servers. In addition to the deployment efficiency, the Cloud Foundation 5.1 and PowerEdge solution delivered strong performance while running a MySQL database workload. By leveraging VMware Cloud Foundation 5.1 and PowerEdge servers, you could help your organization embrace cloud computing with confidence, potentially unlocking a new level of agility, scalability, and efficiency in your data center operations.
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Principled Technologies
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
The Digital Insurer
Webinar Recording: https://www.panagenda.com/webinars/why-teams-call-analytics-is-critical-to-your-entire-business Nothing is as frustrating and noticeable as being in an important call and being unable to see or hear the other person. Not surprising then, that issues with Teams calls are among the most common problems users call their helpdesk for. Having in depth insight into everything relevant going on at the user’s device, local network, ISP and Microsoft itself during the call is crucial for good Microsoft Teams Call quality support. To ensure a quick and adequate solution and to ensure your users get the most out of their Microsoft 365. But did you know that ‘bad calls’ are also an excellent indicator of other problems arising? Precisely because it is so noticeable!? Like the canary in the mine, bad calls can be early indicators of problems. Problems that might otherwise not have been noticed for a while but can have a big impact on productivity and satisfaction. Join this session by Christoph Adler to learn how true Microsoft Teams call quality analytics helped other organizations troubleshoot bad calls and identify and fix problems that impacted Teams calls or the use of Microsoft365 in general. See what it can do to keep your users happy and productive! In this session we will cover - Why CQD data alone is not enough to troubleshoot call problems - The importance of attributing call problems to the right call participant - What call quality analytics can do to help you quickly find, fix-, and prevent problems - Why having retrospective detailed insights matters - Real life examples of how others have used Microsoft Teams call quality monitoring to problem shoot problems with their ISP, network, device health and more.
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
This presentation explores the impact of HTML injection attacks on web applications, detailing how attackers exploit vulnerabilities to inject malicious code into web pages. Learn about the potential consequences of such attacks and discover effective mitigation strategies to protect your web applications from HTML injection vulnerabilities. for more information visit https://bostoninstituteofanalytics.org/category/cyber-security-ethical-hacking/
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
Boston Institute of Analytics
If you are a Domino Administrator in any size company you already have a range of skills that make you an expert administrator across many platforms and technologies. In this session Gab explains how to apply those skills and that knowledge to take your career wherever you want to go.
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Gabriella Davis
JAM, the future of Polkadot.
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Juan lago vázquez
Último
(20)
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Deploy with confidence: VMware Cloud Foundation 5.1 on next gen Dell PowerEdg...
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Breaking The Clustering Limits @ AlphaCSP JavaEdge 2007
1.
2.
Breaking the Clustering
Limits Baruch Sadogursky Consultant, AlphaCSP
3.
Cluster at NASA
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
Grid Types
19.
Data Grids
20.
21.
Computational Grids
22.
23.
24.
Master/Worker
25.
Map/Reduce
26.
27.
28.
29.
30.
Implementations
31.
EHCache
32.
33.
EHCache Example
34.
GlassFish Shoal
35.
36.
JXTA Usage in
Shoal
37.
Oracle Tangosol Coherence
38.
39.
Coherence Data Grid
40.
41.
42.
JBoss POJO Cache
43.
44.
45.
JBoss POJO Cache
Usage
46.
JGroups Configuration
47.
GigaSpaces
48.
49.
50.
GigaSpaces XAP
51.
OpenTerracotta
52.
53.
54.
55.
56.
57.
Terracotta Client/Server
58.
Terracotta Demo
59.
60.
61.
Descargar ahora