The increased requirements of modern analytical workloads – querying billions of rows on demand, in real time and in unforeseen ways – is a challenge for traditional databases because they’re optimized for transactional workloads (e.g., point and range queries with indexes). A transactional query may return every column in a single row whereas an analytical query may aggregate a single column in every row. Thus, it is far more efficient to store data by column rather than by row. In addition, the use of distributed data and massively parallel processing enables columnar databases to support scalable, high-performance analytics. In this webinar, we will use the architecture of MariaDB AX to explain how columnar storage and massively parallel processing work, and how they enable columnar databases to query billions of rows in real time, and with the full power of SQL – a challenge for Apache Hadoop/Hive.

1. How columnar databases support
modern analytics
Shane Johnson
Senior Director of Product Marketing

2. Introducing MariaDB AX
How columnar databases
support modern analytics

3. MariaDB Server
InnoDB
MariaDB TX MariaDB AX
Optimized for OLTP
Row-based storage
Optimized for OLAP
Columnar storage

4. MariaDB AX and columnar database use cases
– agenda –
MariaDB AX
Introduction
Architecture
Use cases
Financial services
Healthcare
Telecommunications
Digital advertising

5. A database platform for
modern analytics and data
warehousing.
MariaDB AX and columnar database use cases
– introduction –
Distributed data
Columnar storage
Parallel processing
Data adapters
Connectors (Spark & Kafka)
Open source
Standard SQL

6. MariaDB AX architecture
How columnar databases
support modern analytics

7. MariaDB Server
InnoDB
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00

8. MariaDB Server
InnoDB
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00
SELECT AVG(spend)
FROM tbl_purchases

9. MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00

10. MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00
SELECT AVG(spend)
FROM tbl_purchases

11. MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00
1. high compression (65-95%)
2. supports spare columns (NULL)
3. supports many columns
4. no need for indexes

12. MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
customer month year spend
1 01 2018 100.00
2 02 2018 50.00
3 03 2018 75.00
10000000 03 2018 75.00
1. columns stored as segments (file)
2. segments have extents (logical)
3. extents have 8 million rows

14. Performance Module (PM)
User Module (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage

15. S1 (UM + PM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage

16. S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
S2 (PM)
Rows 1 to 300,000

17. S4 (PM)S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
S3 (PM)
ColumnStore
Storage
ColumnStore
Storage
Rows 1 to 100,000 Rows 100,001 to 200,000 Rows 200,001 to 300,000

18. S4 (PM)S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
S3 (PM)
ColumnStore
Storage
ColumnStore
Storage
T1: 00,001 to 30,000
T2: 30,001 to 60,000
T4: 100,001 to 130,000
T5: 130,001 to 160,000
T7: 200,001 to 230,000
T8: 230,001 to 260,000

19. S6 (PM)S4 (PM)
S2 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
S5 (PM)
ColumnStore
Storage
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
S3 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
Query
throughput
Query
latency

20. S2 (PM)
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
Storage
SQL
S4 (PM)
ColumnStore
Storage
S6 (PM)
ColumnStore
Storage

21. S1 (UM)
S2 (PM)
ColumnStore
Storage
Import (CLI)
S4 (PM)
ColumnStore
Storage
S6 (PM)
ColumnStore
Storage
File

22. S2 (PM)
ColumnStore
Storage
S4 (PM)
ColumnStore
Storage
S6 (PM)
ColumnStore
Storage
Import (CLI)
File (2/3)
Import (CLI)
File (1/3)
Import (CLI)
File (3/3)

23. Application/Service/Script
S2 (PM)
ColumnStore
Storage
Bulk data
adapter
S4 (PM)
ColumnStore
Storage
S6 (PM)
ColumnStore
Storage

24. Spark job/task
S2 (PM)
ColumnStore
Storage
Spark
Connector
S4 (PM)
ColumnStore
Storage
S6 (PM)
ColumnStore
Storage

25. Bulk adapters
S1 (UM)
MariaDB Server
InnoDB
ColumnStore
ColumnStore
C
Java
Python
Stream adapters
Kafka
MaxScale CDCSpark
Storage
SQL
Import (CSV)S2 (PM)

26. MariaDB AX
customer use cases
How columnar databases
support modern analytics

27. MariaDB AX and columnar database use cases
– financial services –
Drivers
Become customer-centric
Facilitate regulatory compliance
Create competitive advantages
Goals
Improve customer satisfaction
Mitigate financial risks
Predict market changes
Use cases
Fraud detection: identify patterns + detect anomalies in financial transactions
Compliance archiving: store financial trade history for long-term retention
Investment forecasting: analyze financial markets + securities to predict ROI

28. MariaDB AX and columnar database use cases
– OTC Markets Group –
Data
10TB of rolling data (5 years)
10,000 U.S. and global securities
100,000 trades
24 million quotes
Use cases
Subscribers analyze quote and trading
data
Regulatory agencies build compliance
reports on demand

29. MariaDB AX and columnar database use cases
– healthcare –
Drivers
Digital transformation
Electronic health records (EHRs)
Value-base care (VBC)
Goals
Improved population health
Better patient experiences
Reduced cost of care
Use cases
Population health mgt: analyze claims/surveys to recommend interventins
Evidence-based medicine: improve diagnostic accuracy by analyzing EHRs
Precision medicine: identify targeted treatments by analyzing genomes

30. MariaDB AX and columnar database use cases
– Institute for Health Metrics and Evaluation –
Data
30TB of data
100 billion data points
Multi-billion row tables
Use cases
Enable the public to analyze global
health population data via online data
visualization tools

32. MariaDB AX and columnar database use cases
– telecommunications –
Drivers
Improve sales, marketing and
operational efficiency
Goals
High customer retention
Better network optimizations
New services and revenue
Use cases
Churn prevention: analyze customer plans/usage to create retention programs
Cross-selling: identify opportunities by analyzing call detail records (CDRs)
Network optimization: analyze traffic/cell tower data to optimize capacity

33. MariaDB AX and columnar database use cases
– Pinger, Inc. –
About
30 million texts
3 million phone calls
1.5 billions logs a day
24 months’ worth of data
Use cases
To support customer behavioural
analysis based on historical data and
usage

34. MariaDB AX and columnar database use cases
– digital advertising –
Drivers
Granularity of data
Demographic and behavioral
Social and location
Goals
Deliver the right ad to the right person at
the right time, in the right location and
through the right medium
Use cases
Audience segmentation: improve ad relevance via fine-grained visitor profiles
Ad placement: choose where to show ads based on click and conversion data
Real-time bidding: analyze big request/response history to optimize prices

35. MariaDB AX and columnar database use cases
– digital advertising vendor –
About
300 million impressions a month
70 million rows a day
60TB of uncompressed data
Use cases
Enable customers to create a custom
report on up to 30 columns on
demand

36. How columnar databases
support modern analytics
Questions?

37. Thank you