Más contenido relacionado La actualidad más candente (20) Similar a Leveraging Big Data with Hadoop, NoSQL and RDBMS (20) Más de Aerospike, Inc. (10) Leveraging Big Data with Hadoop, NoSQL and RDBMS2. 2Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Sustainable Differentiators
SPEED AT SCALE
• Index in RAM, Data in Flash: Persistence
• Key-value store
• Open Source
SMART CLIENTS
• Ease of use for programmers
• Patented functionality that accelerates development by shielding
developers from low level programming
CLUSTERING
• Availability
• Patented clustering algorithms that solve the hardest problems of
distributed computing
TCO
• Optimized for Flash
• Demonstrated 10 to 1 price performance savings
• Battle tested, almost no downtime
4. 4Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Polyglot Architecture
Transactions
(Compliance, Legacy)
(Oracle, DB/2,
SQLServer, MariaDB,
Postgres)
Operational
In-memory
(Fast, Scalable, HA)
(Aerospike, Cassandra
with Cache, Coherence)
Operational Analytics
Relational
Ad-Hoc
(Exadata, Aster,
Teradata)
Hadoop Big Data
(Cloudera, Hortonworks,
MapR)
Column
(Vertica, RedShift,
SnowFlake, Oracle)
Graph
(Neo4J, Objectivity,
... )
Streaming
(Spark, Heron)
… fast analytics but
still analytics
App server architecture
In-memory SQL
(VoltDB, MemSQL, …)
Polyglot Analytics systems (multiple are required because of
performance reasons –data layout matters)
Operational
In-memory
Read Write
5. 5Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Old Style Architecture Has Significant Limitations
Challenges
• Complex
• Maintainability
• Durability
• Consistency
• Scalability
• Cost ($)
• Data Lag
Caching Layer
Operational Database
Real-time
Consumer Facing
Pricing /
Inventory/Billing
Real-time
Decisioning
Streaming
Data
Legacy Database
(Mainframe)
RDBMS
Databas
e
Transactional
Systems
Enterprise Environment
Legacy RDBMS
HDFS BASED
6. 6Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
High Performance NoSQL Enables Real-time Applications
Real-time
Consumer Facing
Pricing /
Inventory/Billing
Real-time
Decisioning
Streaming
Data
Aerospike
Connectors
Legacy Database
(Mainframe)
RDBMS
Databas
e
Transactional
Systems
Enterprise Environment
XDR
Aerospike
Legacy RDBMS
HDFS BASED
Next Generation Operational Database
Business transactions of
500/sec X 2500 reads/writes/
calculations per business
transaction
= 1.25M Server
Transactions/sec
Speed at Scale, Predictable Performance, Highest Availability & Lowest TCO
7. 7Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Business Challenge
• Meet SLAs of 750 ms per business transaction
• Differentiate between fraudulent and legitimate orders in real-
time
• Support next-generation Machine Learning
• Stop loss of business due to latency
• Support hundreds of DB reads/writes per credit card
transaction
• Increase operational data 10x
Prevent Only Fraudulent Transactions
8. 8Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Selected Aerospike in-Memory NoSQL
• Built for Flash
• Predictable Low latency at
High Throughput
• Immediate consistency, no data loss
• Cross data center (XDR) support
• 20 Server Cluster
• Dell 730xd w/ 4NVMe SSDs
Prevent Only Fraudulent Transactions
Credit Card Processing System
Fraud Detection & Protection App
Rules
Rule 1
Rule 2
Rule 3
Historical Data
Rule 1-Passed
Rule 2-Passed
Rule 3-Failed
Account Behavior
Static Data
Account
Statistics
9. 9Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Intra-day System of Record
Challenge
• DB2 (RDBMS) stores positions for 10 Million customers
• Must update stock prices, show balances on 300 positions,
process 250M transactions, 2M updates/day
• Risk, System of Engagement and Mobile
• Data inconsistencies, long restarts, growing
number of servers, restarts take 1 hour
10. 10Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Intra-day System of Record
Selected Aerospike in-Memory NoSQL
• Built for Flash
• Predictable Low latency at High Throughput
• Immediate consistency, no data loss
• Hot standby implementation for extra redundancy
• Cross data center (XDR) support
• 10 Server Cluster
IBM DB2
(Mainframe)
Real-Time App Record App
Finance App
Real-Time
Data Feed
Start of the Day
Data Loading
End of Day
Reconciliation
Account
Positions
11. 11Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
AdTech – Predictive Analytics at Scale
Challenge
• Low read latency (milliseconds)
• 100K to 5M operations/second
• Ensure 100% uptime
• Provide global data replication
Performance achieved
• 1 to 6 billion cookies tracked
• 5.0M auctions per second
• 100ms ad rendering, 50ms real-time bidding,
1ms database access
• 1.5KB median object size
Selected Aerospike NoSQL over competition
• 10X fewer nodes
• 10X better TCO
• 20X better read latency
• High throughput at low latency
Ads is Displayed
Publishers
Ad Networks & SSPs
Ad Exchanges
Demand Side
Platform
Data Management
Platforms
Brands Agencies Buyers
0 ms 100 ms
12. 12Proprietary & Confidential || © 2015 Aerospike Inc. All rights reserved.[ ]
Telco – Real-Time Billing and Charging Systems
Challenge
• Edge access to regulate traffic
• Accessible using provisioning applications
(self-serve and through support personnel)
Need for extremely high availability,
reliably, low latency
• > TBs of data
• 10-100M objects
• 10-200K TPS
Selected Aerospike in-Memory NoSQL
• Clustered system
• Predictable low latency at high throughput
• Highly-available and reliable on failure
• Cross data center (XDR) support
SOURCE
DEVICE/USER DESTINATIONReal-Time
Auth. QoS Billing
Request Execute
Request
Real-Time ChecksConfig Module App
Update Device
User Setting
Hot-Standby
XDR
Notas del editor High performance with Aerospike’s hybrid approach to memory also means that Aerospike can scale up to store more data per server and process the same load with 10x fewer servers than a DRAM only system.
Here a customer priced a 10TB system (2x replication) that had to handle a total of 500k TPS.
A 14 node cluster with Aerospike cost only $236k compared to the 186 server cluster they would have had to purchase, install, operate and maintain with the competition.
The prices of the servers were based the price of a Dell R320 and 4 Intel S3700 800 GB in January of 2014. These drives do not need to be overprovisioned. The major issue with the cost on the RAM based servers is the use of 6 x 32 GB DIMM. 32 GB DIMMs are very expensive, on the SSD servers, we could use 6 x 8 GB DIMMs (48 GB RAM). While it is possible to use smaller DIMMs, you will need to have a larger chassis with more CPU. The result is the price of the server is roughly the same (per GB). Key Points:
All of the above challenges
If you use Relational, must use cache and you compromise the value of RDBMS = Consistency & durability issues as well