#GeodeSummit - Integration & Future Direction for Spring Cloud Data Flow & Geode

What's hot

#GeodeSummit - Redis to Geode Adaptor

PivotalOpenSourceHub

Spark on Mesos

Jen Aman

Operationalizing YARN based Hadoop Clusters in the Cloud

DataWorks Summit/Hadoop Summit

Splice Machine is an open-source database that combines the benefits of modern lambda architectures with the full expressiveness of ANSI-SQL. Like lambda architectures, it employs separate compute engines for different workloads - some call this an HTAP database (Hybrid Transactional and Analytical Platform). This talk describes the architecture and implementation of Splice Machine V2.0. The system is powered by a sharded key-value store for fast short reads and writes, and short range scans (Apache HBase) and an in-memory, cluster data flow engine for analytics (Apache Spark). It differs from most other clustered SQL systems such as Impala, SparkSQL, and Hive because it combines analytical processing with a distributed Multi-Value Concurrency Method that provides fine-grained concurrency which is required to power real-time applications. This talk will highlight the Splice Machine storage representation, transaction engine, cost-based optimizer, and present the detailed execution of operational queries on HBase, and the detailed execution of analytical queries on Spark. We will compare and contrast how Splice Machine executes queries with other HTAP systems such as Apache Phoenix and Apache Trafodian. We will end with some roadmap items under development involving new row-based and column-based storage encodings. Speakers: Monte Zweben, is a technology industry veteran. Monte’s early career was spent with the NASA Ames Research Center as the Deputy Chief of the Artificial Intelligence Branch, where he won the prestigious Space Act Award for his work on the Space Shuttle program. He then founded and was the Chairman and CEO of Red Pepper Software, a leading supply chain optimization company, which merged in 1996 with PeopleSoft, where he was VP and General Manager, Manufacturing Business Unit. In 1998, he was the founder and CEO of Blue Martini Software – the leader in e-commerce and multi-channel systems for retailers. Blue Martini went public on NASDAQ in one of the most successful IPOs of 2000, and is now part of JDA. Following Blue Martini, he was the chairman of SeeSaw Networks, a digital, place-based media company. Monte is also the co-author of Intelligent Scheduling and has published articles in the Harvard Business Review and various computer science journals and conference proceedings. He currently serves on the Board of Directors of Rocket Fuel Inc. as well as the Dean’s Advisory Board for Carnegie-Mellon’s School of Computer Science.

October 2016 HUG: Architecture of an Open Source RDBMS powered by HBase and ...

Yahoo Developer Network

The architectural tradeoffs between the map/reduce paradigm and parallel databases has been a long and open discussion since the dawn of MapReduce over more than a decade ago. At Facebook, we have spent the past several years in independently building and scaling both Presto and Spark to Facebook scale batch workloads, and it is now increasingly evident that there is significant value in coupling Presto’s state-of-art low-latency evaluation with Spark’s robust and fault tolerant execution engine.

Presto on Apache Spark: A Tale of Two Computation Engines

Databricks

Operational Tips for Deploying Spark by Miklos Christine

Spark Summit

Building Efficient Pipelines in Apache Spark

Jeremy Beard

Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu

Jeremy Beard

Simplified Cluster Operation & Troubleshooting

DataWorks Summit/Hadoop Summit

HDInsight for Architects

Ashish Thapliyal

Achieve big data analytic platform with lambda architecture on cloud

Scott Miao

5 Apache Spark Tips in 5 Minutes

Cloudera, Inc.

In this presentation, Dmitriy will describe the strategy and architecture behind the Apache Ignite(TM) (incubating) In-Memory Data Fabric, a high-performance, distributed in-memory data management software layer that boosts application performance and scale by orders of magnitude. We will dive into the technical details of distributed clusters and compute grids as well as distributed data grids, and provide code samples for each. As integral parts of an In-Memory Data Fabric, Dmitriy will also cover distributed streaming, CEP and Hadoop acceleration. This presentation is particularly relevant for software developers and architects who work on the front lines of high-speed, low-latency Fast Data systems, high-performance transactional systems and real-time analytics applications.

IMCSummit 2015 - Day 2 Developer Track - Anatomy of an In-Memory Data Fabric:...

In-Memory Computing Summit

Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters

DataWorks Summit

Mon-Fong Mike Jiang, Kuan-Yu Hubert Fan-Chiang and Tienyu Rebecca Lin 自2011年起，我們就開始使用HBase作為結構化大數據的儲存工具，主要是做為半導體製造設備參數的分析。為了有效進行數據查詢，我們開發Standard Query Language(SQL)的整合介面，最早的方式是(1)自行開發GUI操作介面及(2)透過自行定義SQL語法的方式進行，但是這樣會衍生出很多額外的工作，特別是SQL Parser與對應的HBase API的連結。為了解決此問題，我們解析了Hive QL Parser作為主要的核心，將此部分的原始碼整合進HareDB HBase Client之中，另外，也整合了HBase Coprocessor，可以加速查詢的進行，這個架構我們實際使用在數個半導體製造廠的大數據系統中，也展現了高查詢效率。除此之外，透過整合Kafka來處理串流數據的匯入，同時對於數據分析的呈現也加上Cube建立工具，這些都是實際開發大數據系統時陸續面對的問題與解決方法，我們將分享這一連串的系統開發過程。 hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#

hbaseconasia2017: HareQL：快速HBase查詢工具的發展過程

HBaseCon

Apache sqoop with an use case

Davin Abraham

If you are interested in Big Data, you surely already know Apache Spark or Apache Cassandra, but do you know Apache Zeppelin ? Do you know that it is possible to draw out beautiful graph using an user-friendly interface out of your Spark RDD and Cassandra queries ? In this session, I will introduce Zeppelin by live coding example and highlight its modular architecture which allows you to plug-in any interpreter for the back-end of your choice. Then we'll dig into the Apache Cassandra interpreter and show how to use it as a default front-end to display your Cassandra data About the Speaker DuyHai DOAN Apache Cassandra Evangelist, DataStax DuyHai DOAN is an Apache Cassandra Evangelist at DataStax. He spends his time between technical presentations/meetups on Cassandra, coding on open source projects like Achilles or Apache Zeppelin to support the community and helping all companies using Cassandra to make their project successful. Previously he was working as a freelance Java/Cassandra consultant.

Getting Started with Apache Cassandra and Apache Zeppelin (DuyHai DOAN, DataS...

DataStax

Apache Geode Offheap Storage

PivotalOpenSourceHub

Apache Kylin is a distributed OLAP engine on Hadoop, which provides sub-second level query latency over datasets scaling to petabytes. Kylin’s superior query performance relies on pre-calculated multi-dimension Cube, which is often time-consuming to build. By default, Kylin uses MapReduce Cube Engine built atop of Hadoop MapReduce framework to aggregate huge amounts of source data. The MR Engine has been well-tuned over years and proven to be stable in hundreds of production deployments. Recently, the Kylin team is trying to further speed up the process of cube building by replacing MR with Spark. Kyligence has initiated the new Spark Cube Engine with some benchmarks between Spark and MR over different datasets, and has received some promising results. Hear about their results and experiences on moving Cube building, which is a huge computing task, to Spark.

Apache Kylin: Speed Up Cubing with Apache Spark with Luke Han and Shaofeng Shi

Databricks

Spark Uber Development Kit

DataWorks Summit/Hadoop Summit

What's hot (20)

#GeodeSummit - Redis to Geode Adaptor

Spark on Mesos

Operationalizing YARN based Hadoop Clusters in the Cloud

October 2016 HUG: Architecture of an Open Source RDBMS powered by HBase and ...

Presto on Apache Spark: A Tale of Two Computation Engines

Operational Tips for Deploying Spark by Miklos Christine

Building Efficient Pipelines in Apache Spark

Building Effective Near-Real-Time Analytics with Spark Streaming and Kudu

Simplified Cluster Operation & Troubleshooting

HDInsight for Architects

Achieve big data analytic platform with lambda architecture on cloud

5 Apache Spark Tips in 5 Minutes

IMCSummit 2015 - Day 2 Developer Track - Anatomy of an In-Memory Data Fabric:...

Mercury: Hybrid Centralized and Distributed Scheduling in Large Shared Clusters

hbaseconasia2017: HareQL：快速HBase查詢工具的發展過程

Apache sqoop with an use case

Getting Started with Apache Cassandra and Apache Zeppelin (DuyHai DOAN, DataS...

Apache Geode Offheap Storage

Apache Kylin: Speed Up Cubing with Apache Spark with Luke Han and Shaofeng Shi

Spark Uber Development Kit

Viewers also liked

This talk introduces an open-source solution that integrates cloud native apps running on Cloud Foundry with an open-source hybrid transactions + analytics real-time solution. The architecture is based on the fastest scalable, highly available and fully consistent In-Memory Data Grid (Apache Geode / GemFire), natively integrated to the first open-source massive parallel data warehouse (Greenplum Database) in a hybrid transactional and analytical architecture that is extremely fast, horizontally scalable, highly resilient and open source. This session also features a live demo running on Cloud Foundry, showing a real case of real-time closed-loop analytics and machine learning using the featured solution.

#GeodeSummit: Architecting Data-Driven, Smarter Cloud Native Apps with Real-T...

PivotalOpenSourceHub

#GeodeSummit - Wall St. Derivative Risk Solutions Using Geode

PivotalOpenSourceHub

Today, if events change the decision model, we wait until the next batch model build for new insights. By extending fast “time-to-decisions” into the world of Big Data Analytics to get fast “time-to-insights”, apps will get what used to be batch insights in near real time. The technology enabling this includes smart in-memory data storage, new storage class memory, and products designed to do one or more parts of an analysis pipeline very well. In this talk we describe how Ampool is building on Apache Geode to allow Big Data analysis solutions to work together with a scalable smart storage class memory layer to allow fast and complex end-to-end pipelines to be built -- closing the loop and providing dramatically lower time to critical insights.

#GeodeSummit: Democratizing Fast Analytics with Ampool (Powered by Apache Geode)

PivotalOpenSourceHub

#GeodeSummit: Easy Ways to Become a Contributor to Apache Geode

PivotalOpenSourceHub

The financial sector is an exciting mix of challenges regarding throughput, high availability as well as specific constraints regarding latency and consistency. In the continuous evolution of its platform, Murex relies on open source technologies like Apache Geode and Apache Storm in a "kind of" lambda architecture to ensure storage, near-real time (around the milliseconds) aggregation of thousands of events per second, advanced notification mechanisms and on-demand deployments. This talk will focus on the technical architecture, the underlying principles as well as the technologies used to support this mix of functional and non-functional requirements.

#GeodeSummit: Combining Stream Processing and In-Memory Data Grids for Near-R...

PivotalOpenSourceHub

One of the largest retailers in North America are considering Apache Geode for their new mobile loyalty application, to support their digital transformation effort. They would use Geode to provide operational data services for their mobile cloud service. This retailer needs to replace sluggish response times with sub-second response which will improved conversion rates. They also want to able to close the loop between data science findings and app experience. This way the right customer interaction is suggested when it is needed such as when customers are looking at their mobile app while walking in the store, or sending notifications at the individuals most likely shopping times. The final benefits of using Geode will include faster development cycles, increased customer loyalty, and higher revenue.

#GeodeSummit - Using Geode as Operational Data Services for Real Time Mobile ...

PivotalOpenSourceHub

Wondering how to improve on your production yield, increase asset life and activate reliability centered maintenance? TEKsystems has developed “Golden Batch” recommendation engine to realize your goals of modern manufacturing. This is a Predictive analytics framework built on top of Manufacturing Data Lake for analysis and training of machine learning algorithms, and subsequent processing and detection of streaming data from sensors to detect or predict failures. We’ll present a solution architecture featuring Spring XD for data pipelining, Apache Geode for in-memory processing, Hadoop as a data lake, and R for machine learning.

#GeodeSummit - Modern manufacturing powered by Spring XD and Geode

PivotalOpenSourceHub

In this session we explore a case study of a large-scale government fraud detection program that prevents billions of dollars in fraudulent payments each year leveraging the beta release of the GemFire+Greenplum Connector, which is planned for release in GemFire 9. Topics will include an overview of the system architecture and a review of the new GemFire+Greenplum Connector features that simplify use cases requiring a blend of massively parallel database capabilities and accelerated in-memory data processing.

#GeodeSummit - Large Scale Fraud Detection using GemFire Integrated with Gree...

PivotalOpenSourceHub

Redis, a 2 minutes introduction

Mirko Calvaresi

"Do your teams release software to production weekly, daily or every hour ? Do you practice software development with tools, process and culture that can respond to the speed of market and customer changes? Agility allows you to experiment with new business models, learn from your mistakes and identify patterns that work. Deliver faster, look for feedback, gain knowledge. In every market, speed wins. Cloud Native describes the patterns of high performing organizations delivering software faster, consistently and reliably at scale. Continuous delivery, DevOps, and microservices label the why, how and what of the cloud natives, the true digital enterprises." Speaker: Vijay Rajagopal, Advisory Platform Architect, Pivotal

Pivotal Cloud Foundry: A Technical Overview

VMware Tanzu

Building Services with WSO2 Application Server and WSO2 Microservices Framewo...

Sagara Gunathunga

Cloud Native Runtime Platform

VMware Tanzu

Presented at GR8Conf US 2016 in Minneapolis, MN. You've got your microservices, cloud config & Eureka servers running in development, but how do you take that to production? A single instance of each isn't a viable solution for these critical services. In this talk, we'll walk through how to configure and run the cloud config and Eureka servers in a high availability manner. We will also cover other production tasks like pushing config updates and monitoring all your Hystrix circuit breakers using Turbine. This is not an introductory talk and assumes a basic familiarity with the Spring Cloud Services.

Spring Cloud Into Production

Todd Miller

Devops Recto-Verso @ DevoxxMA

Arnaud Héritier

WSO2ConUS 2015 - Introduction to WSO2 Microservices Server (MSS)

Afkham Azeez

Os microservices tornaram-se o tema mais quente na arquitetura de software, e muito se pode dizer sobre os seus benefícios. Mas é importante compreender que ao começar a decompor o monolito entramos no reino de sistemas distribuídos. Precisamos lidar com cenários de transparência de localização, recuperação de falhas, escalabilidade, autonomia, facilidade de atualização e outros aspectos. Por outro lado, quando tratamos toda essa complexidade, conseguimos definir aplicações que rodam “eternamente” – tornando-se auto-curáveis (self-healing) e muito escaláveis. O projeto Spring Cloud traz ferramentas para facilitar a aplicação desses conceitos na prática, definindo padrões de implementação para sistemas distribuídos com suporte a gerenciamento de configuração, descoberta de serviços, circuit breakers, roteamento, distribuição de sessões, barramento de controle, dentre outras técnicas. Pode também ser combinado com a plataforma Netflix OSS, que oferece componentes especialmente úteis em um ambiente de microservices, como Eureka (registro), Ribbon (localização), Hystrix (tolerância a falhas), Zulu (roteamento). Nesta palestra será demonstrado como usar na prática essas ferramentas, bem como os desafios e lições aprendidas – tudo com base em casos reais de uso de uma arquitetura de microservices implementada com Spring Cloud e Netflix OSS.

QCon SP 2016 - Construindo Microservices Auto-curáveis com Spring Cloud e Net...

Rodrigo Cândido da Silva

Spring Cloud Servicesの紹介 #pcf_tokyo

Toshiaki Maki

Apache Geode Meetup, London

Apache Geode

Build your first Internet of Things app today with Open Source

Apache Geode

Viewers also liked (19)