How to run Real Time processing on Big Data / Ron Zavner (GigaSpaces)

Copyright © GigaSpaces 2015. All rights reserved.Copyright © GigaSpaces 2015. All rights reserved.
How to run Real Time
processing on Big Data
Ron Zavner
Technical Director, EMEA Gigaspaces

Copyright © GigaSpaces 2015. All rights reserved. 2
About GigaSpaces
GigaSpaces provides software middleware for deployment,
management and scaling of mission-critical applications on
cloud environments.
GigaSpaces serves more than 500 large enterprises & ISVs,
over 50 of which are Fortune-listed.
Direct customers
300+
Fortune / Organizations
40+ / 500+
Cloud Customers
75+
ISVs
25+

Copyright © GigaSpaces 2015. All rights reserved.
Agenda
• The challenge of real time analytics today
• Introduction to In Memory Data Grids
• Meet XAP
• XAP on Big Data (plus advanced features)
• Use cases + Demo
3

In today’s reality
information is gathered from numerous sources

• Number of transactions grows exponentially
• Tolerance for system response time reducessignificantly

In order to gain a competitive edge, organizations need
real time processing of large data sets

Alongside this challenge, there are many other challenges for
management of enterprise applications
Such as:

Current tier based architecture cannot meet these challenges

Lets go over these challenges
and see why

Peak Loads Challenges
Your messaging tier
can only scale so far…
Result:
Costly over-provisioning
Massive over-provisioning
of resources only to meet
peak loads
Your database can
only scale so far…

Performance Challenges
Disk throughput limitations
and bottlenecks
Network bottlenecks
Result:
Higher latency for your
business transactions,
which can be very costly

Business Continuity Challenges
Too many moving parts
Result:
Application becomes
more error-prone and
harder to troubleshoot

TCO Reduction Challenges
Result:
Having many moving
parts means higher
operational costs
(purchase, update,
maintain, etc).

Real-Time Business Insight Challenges
Result:
If performance is not
ideal, than real time
response isn’t either.
In today’s reality, real time
event processing of
constantly growing data
sets gives your company
the competitive edge it
needs.

Tier-based architecture cannot solve today's data &
application processing issues

Meets All
These Challenges

XAP scales the Data Tier using its In-Memory Data Grid
so you can access your data in real time

XAP scales the Data Tier using its In-Memory Data Grid
The database goes
to the background
Partition your data
and store it in memory

Same goes for the Messaging Tier
Partitioned, co-located
in-memory messaging

XAP scales the entire application – so you get extreme processing of your
big data and get real time insights
Business logic, data &
messaging co-located
& partitioned into
processing units

XAP ensures High Availability
Hot backup for each
partition for high
availability

Scale the Web Tier
Host your web
application on the
XAP infrastructure

XAP enables auto-scaling of the entire application on demand
Auto-scale out & in
based on real-time
performance & load

Result:
Real time processing &
analytics of your big data
with XAP IMC

What can XAP do
for you?
Scaling the Data Tier
Multi-site deployment &
DR across remote sites
Batch processing of
large data sets
Online transaction
processing
Real time querying and analysis
of large datasets
Real time processing of
large event stream
Scaling the Web Tier

Some features…
Complex Queries Support
Scala Support
Data Consistency Level
Increase developer productivity and application
scalability using new syntax for complex nested
conditions
Comprehensive Scala support including Tasks,
Objects, APIs and Queries
Update Multiple Entries to different
partition with timeout to cope with locks
Optimize XAP to your business requirements
setting the Data Consistency Level
Optimized Cross Partition Update
IPv6 LRMI Filters
Support for IPv6 Network with XAP
deployments
Web GUI can serve hundreds of nodes with
good UX
Allow for encryption and compression for
selected LRMI connection
Scalable Web GUI
Better support for customized user
credentials
Extended Custom Security

and more…
27
Microsoft Linq provider
Support for queries written using Linq API
and syntax
MongoDB External Data Source
Bi directional data and metadata exchange
between MongoDB and XAP
Unique constrains in Index
Support for unique validation of objects in
index in each partition
Change API Enhancements
Additional Change API returns the value
that had been changed
Advanced Projections support
Partial queries on nested objects
Immutable Objects Support
Zone controlled deployment
Support for mapping primary nodes into
specific zone

28
Aggregation
Framework
Custom
Change
Operations
Enhanced
Initial
Data Load
Management &
Deployment
Automation
Query
Analysis
Memory Xtend
for SSD
Global HTTP
Session Sharing
and more…

Aggregations – Java
29

Aggregations – .Net (using LINQ)
30

Change Operation
31

Custom Change Operation – Java
32

The In-Memory Computing Platform
Extreme scaling solution across multiple verticles

Multi-Site Operations
By Synchronization of dynamic data
across remote sites
Multi-Site Data Replication Across Remote Data Centers
Real Time Data Processing & Analysis
By providing a secured private data cloud
Out of the box
Disaster Recovery Planning
Data is up-to-date across all sites
Near real-time replication of massive data streams – synchronize
data across your different sites in a consistent, failure-proof,
scalable way.

Selected Customers

E-Commerce
Industry
“GigaSpaces XAP performed beautifully, easily withstanding Kohl’s Black Friday load. In fact, we are quite
confident that GigaSpaces could handle Kohl’s growth for years to come”
Handle peak loads without over-
provisioning for maximum traffic
(following 2009 system crash resulting in
loss of millions of $$)
Challenge
Implemented inventory management
on top of XAP within 4 months
Solution
* Kohl’s was N. America’s best performing e-commerce website on Black Friday 2010
Results

Banking
Industry
“GigaSpaces technology allows us to increase customer satisfaction by facilitating better, user-friendly
services, as well as new services, which ultimately enhances our bottom line”
Scalable transaction processing for
trading platform; increase performance
and efficiency of banking processes.
Challenge
GigaSpaces provides infrastructure for
Avanza’s core services, including
trading, customer and data processing,
and storage.
Solution
* Zero system cost for data storage (vs. half system capacity before)
* Massive cost reduction
* Major performance improvement
Results

Premium Edition
XAP In-Memory
Lite Edition
Extreme app scaling & RT insights
Try for free!
Try and let us know what you think…

Check us out:
Email us:
Call us:
Follow us:
www.gigaspaces.com
www.getcloudify.org
info@gigaspaces.com
646-421-2830
@GigaSpaces, @CloudifySource

How to run Real Time processing on Big Data / Ron Zavner (GigaSpaces)

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (8)

Similar a How to run Real Time processing on Big Data / Ron Zavner (GigaSpaces)

Similar a How to run Real Time processing on Big Data / Ron Zavner (GigaSpaces) (20)

Más de Ontico

Más de Ontico (20)

Último

Último (20)

How to run Real Time processing on Big Data / Ron Zavner (GigaSpaces)