Más contenido relacionado La actualidad más candente (20) Similar a Kyligence Cloud 4 - An Overview (20) Más de SamanthaBerlant (8) Kyligence Cloud 4 - An Overview1. Kyligence Cloud 4 – An Overview
George Demarest
Head of Marketing
george.demarest@kyligence.io
Li Kang
VP, North America
li.kang@kyligence.io
2. Agenda
Introduction to Kyligence
Announcement: Kyligence Cloud 4
Kyligence Innovations
Use Cases and Successes
Demonstration
3. © Kyligence Inc. 2021, Confidential.
Three Pillars of Modern Cloud Analytics
Distributed Compute Columnar Storage Precomputation
4. © Kyligence Inc. 2021, Confidential.
MPP Queries – Slow and Costly in the Cloud
Common to Greenplum, Snowflake, Synapse, Redshift
Data Volume /
$$$
Resource
Consumption
Runtime Computation
MPP Engines : Runtime Computation
Data
Concurrency
5. © Kyligence Inc. 2021, Confidential.
OLAP Precomputation - Faster and Cheaper in the Cloud
Q: How many laptops were sold in Los Angeles in June?
An Aggregate Index (a.k.a. Cube)
Precomputation – Compute Once, Query Many Times
Data
Data Volume /
$$$
Resource
Consumption
Runtime Computation
Concurrency
Precomputation
6. © Kyligence Inc. 2021, Confidential.
Apache Kylin
Top Level Project
The only open-source OLAP
technology on big data platform
AwardWinning
InfoWorld’s Bossies 2015 & 2016
(Best of Open-Source Software Awards)
Sub-Second Interactive Query
Large scale, high concurrency, multi-
dimensional, sub-second query latency
1,000+ Organizations
Adopted by thousands of
organizations globally
7. © Kyligence Inc. 2021, Confidential.
Kyligence: Kylin + Intelligence
• Founded in 2016 by the creators of Apache Kylin
• Enhances Kylin with AI-assisted auto-modeling, Unified Semantic
Layer for leading BI tools
• CRN Top-10 big data startups in 2018
• Global Presence: San Jose, Seattle, New York, Shanghai, Beijing
• VCs: Fidelity International, Shunwei Capital, Broadband Capital,
Redpoint, Cisco, Coatue
2016
Pre-A
Redpoint
Cisco
2017
Series A
CBC
Shunwei
2018
Series B
8Roads
2019
Series C
Coatue
8. © Kyligence Inc. 2021, Confidential.
Kyligence in Action
Migrated 1,200+ Cognos
cubes to 2 Kyligence cubes
360º real-time view of spend
insights for finance teams
Migrated 2nd largest SSAS
deployment to Kyligence/Azure
Top 10 Global
Investment Bank
US FinTech
SaaS Provider
World’s Largest
Credit/Debit Card Network
Results:
95% queries < 2s
100+ concurrent users
Results:
500X improved efficiency
10X faster + more stable analytics
Results:
Single, 100+ TB distributed
Kyligence cube
9. © Kyligence Inc. 2021, Confidential.
Trusted by Global Fortune 500
Banking & Finance
Telecom
Technology
CPG & Retail
10. © Kyligence Inc. 2021, Confidential.
Announced Jan. 21
Kyligence Cloud 4
The First Cloud-Native OLAP Platform
Cloud-native architecture Elasticity, separate compute from storage
Cloud immediacy Deploys in minutes, on AWS/Azure marketplace
Enhanced AI engine Auto-modeling, self-tuning cubes/indexes
Unified Semantic Model MDX, SQL, REST access transparent to BI users
11. © Kyligence Inc. 2021, Confidential.
Kyligence Cloud Architecture
Unified Semantic Layer
Data as a Service
Machine Learning
Data as a Service
SaaS & Apps
CRM HCM
SCM
Low Latency Queries
Any BI Tool
Any Cloud
Any Data Platform
Data Warehouse
Streaming Data
Data Lake
AI Augmented Engine
Precomputation Layer
Distributed Cubes
& Table Indexes
Multi-dimensional
Modeling
Security &
Governance
Finance
Marketing
Sales
12. © Kyligence Inc. 2021, Confidential.
Cloud Native Architecture
Separation of compute and
storage
• Spark compute cluster
• Shared cloud object storage
• Scale and optimize separately for best
results and price performance
• Continuity: build cluster separate from – and
doesn’t disrupt – query cluster
Removed legacy Hadoop requirement
• MapReduce, HDFS, HBase
Spark Cubing Persisted in Parquet format,
Optimized for Cloud Storage
High performance components
• Cubing with Spark, stored as Parquet
• All tasks are executed in memory with Spark
- cubing, querying, modeling, aggregation
• Cubes and indexes saved in Parquet
13. © Kyligence Inc. 2021, Confidential.
Precomputing Everything Isn’t Practical
Precomputation
Total Indexes = 2 ^ Number of dimensions - 1
14. © Kyligence Inc. 2021, Confidential.
Precomputing Everything Isn’t Practical
Challenge
• 50 dimensions => 1.2T Indexes
• Today, 100+ dimensions are possible
• Cost is always critical
Solution
• AI-brokered compromise
15. © Kyligence Inc. 2021, Confidential.
AI-Assisted Cubing and Auto-Modeling
Model Training
Pre-aggregation
Data Processing
Auto-Generate
Cube & Index
AI-Augmented
Engine
Self-training
Group wisdom
Queries across the org
ingested as input
Benefits
- Continuous model improvement without
human intervention
- Greater user satisfaction
16. © Kyligence Inc. 2021, Confidential.
Intelligent Query Routing
With Smart Pushdown™
Goal:
• Cubes: maximize query hits on
precomputed results
• Table Indexes: precompute
detailed/ad-hoc queries if possible
• Intelligent pushdown if not
Total Freedom for BI users to slice
and dice, drill down, roll up, etc.
Seconds to Minutes
Smart Pushdown™
Kyligence
millions of queries/day
17. © Kyligence Inc. 2021, Confidential.
Unified Semantic Layer
From technical details to
business contexts
Consistent semantics across
all major BI tools
Centralized access control
Advanced business
definitions and logic
18. © Kyligence Inc. 2021, Confidential.
Case Study: UnionPay – Modernize Traditional OLAP
World’s largest debit/credit card issuer
Results
500x improvement in maintenance efficiency
10x faster with greater stability
Time to insight: less than 4 hours
Significant TCO reduction
Solution
Replaced Cognos backend with Kyligence
Consolidated 1,200 cubes to 2
Retained Cognos reporting frontend
Challenge
1,200+ existing Cognos cubes to manage
1,000+ ETL jobs to maintain
Time to insight: over 4 days
Backend migrated to
2 Kyligence cubes
19. © Kyligence Inc. 2021, Confidential.
Case Study: UnionPay – Modernize Traditional OLAP
World’s largest debit/credit card issuer
Blog
eBook
20. © Kyligence Inc. 2021, Confidential.
Case Study: US Wealth Management – OLAP Cloud Migration
Challenges
Microsoft SSAS reaching practical limits
Many TB OLAP Cube, 350B rows
Query latencies unacceptable for dashboards
MDX queries 20 secs – 60+ secs
SQL queries 5 secs – 10+ secs
Hampered analyst productivity
Slow load times
Duplicate/derived cubes: duplicate effort
Solution
Redeploy to Azure/Kyligence
Same data sources, same BI tools
80+% query latencies < 1 sec
21. © Kyligence Inc. 2021, Confidential.
Case Study: US AI Fintech Vendor – Self-Service Cloud Analytics
SaaS vendors sit on top of a large
amount of transaction data
Provide business insights and industry
benchmarks as an add-on product
subscription
Challenges are performance,
concurrency, and cost
Fixed cloud infrastructure cost while
# of users/queries grow
Not possible with Cloud DWs
22. © Kyligence Inc. 2021, Confidential.
Use Case – Cloud Data Warehouse Enhancements
Improved query performances
Cost reduction by 10x
Support 100s of users is very expensive on Snowflake
Every query / compute causes charges
Excel pivot table support
Target prospects
Snowflake customers
Large dataset and/or large # of users
Prefer to use Excel pivot table
24. © Kyligence Inc. 2021, Confidential.
Kyligence Cloud 4
The First Cloud-Native OLAP Platform
Cloud-native architecture Elasticity, separate compute from storage
Cloud immediacy Deploys in minutes, on AWS/Azure marketplace
Enhanced AI engine Auto-modeling, self-tuning cubes/indexes
Unified Semantic Model MDX, SQL, REST access transparent to BI users
25. © Kyligence Inc. 2021, Confidential.
Contact Us
Kyligence Inc
http://kyligence.io
info@kyligence.io
Twitter: @Kyligence
Apache Kylin
http://kylin.apache.org
dev@kylin.apache.org
Twitter: @ApacheKylin
28. © Kyligence Inc. 2021, Confidential.
Kyligence Pivot to Snowflake Architecture
• Leverage MDX for semantic consistency
• Seamless user experience
• Zero client configuration
• Calculate once, consume many
• Supports AWS and Azure
Database Events Files IoT
Unified Semantic Layer
MDX
Engine
Access
Control
Enterprise
Security
Query
Pushdown