db tech showcase 2019 https://www.db-tech-showcase.com/dbts/tokyo にて、登壇させていただいた際の資料です。お話しできなかったAppendix もあるのでぜひチェックしてみてください。 スピーカー Jisun Kim・Daisuke Masubuchi 「Azure Data Explorer」（ADX）は、Azure管理用の内部ツールを商用に公開したバージョンで、毎日1兆以上のイベントと数百TBから数千TBを超えるデータを処理します。Preview公開後は、大手の石油会社や韓国のオンラインゲーム企業がこの基盤を利用してデータ分析をはじめました。 長年韓国のゲーム会社でデータアナリストをしてきたJisunと、日本のゲーム会社様をお手伝いしている増渕から、ADXを使用したゲームのリアルタイム分析がどのように変わるのか、パイプライン作成、クエリの活用、分析の技術的なヒントを実際のツールとデモを行いました。

1. 韓国オンラインゲームから学ぶアドホックなビックデータ分析
Data Analytics in Gaming
with Azure Data Explorer (Kusto)
Jisun Kim
Global Black Belt, Microsoft
Daisuke Masubuchi
Global Black Belt, Microsoft

2. Whoweare?
Global Black Belt, Gaming Technology Sales Professional is technical sales role within our
enterprise sales organization with a special focus on advanced IT workloads in gaming industry.
We're passionate about helping our gaming customers and partners in China, Japan, Korea and
other Asian countries to start Azure journey with successful launch
ジサン
• データ分析
• 車の運転
• MMORPG
• ヨガ
• 日本食！
ぶっちーさん
• ビール
• ブロスタ
• カレー
• Webサーバー
• Bot開発

3. GamingIndustryinKorea
韓国ゲーム業界の特徴
MMORPG
Free to Play
Online Service Oriented
Huge Log Data
PC games
Mobile games Massive CCU
Hardcore Gamers

4. BigDataAnalyticsinGaming
オンラインゲームのデータ分析の役目
Data Analyst

5. AzureDataExplorerinGamingAnalytics
Fast & Low latency
早い。低レイテンシー
データの収集（インジェスチョン）に最適化

6. 今年注目のAzure新サービス「AzureDataExplorer」
General Availability: Azure Data Explorer
Posted on Thursday, February 7, 2019
Ask questions and iteratively explore data on the fly to improve p
roducts, enhance customer experiences, monitor devices, and bo
ost operations.
Quickly identify patterns, anomalies, and trends in your data. Exp
lore new questions and get answers in minutes. Run as many que
ries as you need, thanks to the optimized cost structure.
2⽉に⼀般提供開始したAzure Data Explorer
質問によってすばやく反復的にデータを調査
製品の改良、カスタマー エクスペリエンスの強化、
デバイスの監視、操作の向上を実現できます。
データのパターン、異常、および傾向を特定する
新たな問題を調査し、数分で回答を得る

7. AzureDataExplorer
Fastandfullymanagedbigdataanalyticsservice
フルマネージド
Fully managed for efficiency
Focus on insights, not the infra-
structure for fast time to value
No infrastructure to manage;
provision the service, choose the
SKU for your workload, and create
database.
ストリーミングデータ
Optimized for streaming data
Get near-instant insights
from fast-flowing data
Scale linearly up to 200 MB per second
per node with highly performant, low
latency ingestion.
ノードごとに200MB/Sec
リニアにスケールアウト可能
フレキシブルなBigデータ分析
Designed for data exploration
Run ad-hoc queries using the
intuitive query language
Returns results from 1 Billion records <
1 second without modifying the data
or metadata
10億レコードを対象にしたクエリーも、
1秒以内に応答します。（Demoあり）

8. Telemetry Analytics for internal Analytics Data Platform for products
AI IOT
Interactive Analytics Big Data Platform
2015 - 2016
Starting with 1st party validation
Building modern analytics
Vision of analytics platform for MSFT
2019
Analytics engine for 3rd party offers
Unified platform across OMS/AI
Expanded scenarios for IOT timeseries
Bridged across client/server security
2017
GA - February 2019
AzureDataExplorer-Evolution
OMS ASC Defender

9. 5 HB
Data ingested per year
13.1 B
Total queries
600 K
Cores
3.7 K
Cluster
460 PB
Total data size
46
Regions in Azure
42 K
Nodes
265 K
Databases
22.6K
Explorer distinct users
AzureDataExplorer-BytheNumbers

10. © Microsoft Corporation
AzureDataExploreroverview
1. Capability for many data types,
formats, and sources
Structured (numbers), semi-structured (JSONXML),
and free text
2. Batch or streaming ingestion
Use managed ingestion pipeline or
queue a request for pull ingestion
3. Compute and storage isolation
• Independent scale out / scale in
• Persistent data in Azure Blob Storage
• Caching for low-latency on compute
4. Multiple options to support
data consumption
Use out-of-the box tools such as Kusto Explorer and
connectors
or use APIs/SDKs for custom solution
Data Lake
/ Blob
IoT
Ingested Data
Engine
Data
Management
Azure Data Explorer
Azure Storage
Event Hub
IoT Hub
Customer Data
Lake
Kafka Sync
Logstash Plugin
Event Grid
Azure Portal
Power BI
ADX Web UI
ODBC / JDBC Apps
Apps (Via API)
Logstash Plugin
Apps (Via API)
Create,
Manage
Stream
Batch
Grafana
Query,
Control Commands
Azure OSS Applications
Active Data
Connections
Kusto Explorer

11. © Microsoft Corporation
Intuitivequerying
Simple and powerful
• Rich rational query language (filter, aggregate, join,
calculated columns, and more)
• Built-in full-text search, time series, user analytics, and
machine learning operators
• Out-of-the box visualization (render)
• Easy-to-use syntax + Microsoft IntelliSense
• Highly recognizable hierarchical schema entities
Comprehensive
• Built for querying over structured, semi-structured and
unstructured data simultaneously
Extensible
• In-line Python
• SQL

13. [Requirement]
• Stream & batch ingestion with Logstash & Azure Log Analytics
• Send alert for anomaly to Slack in timely manner
CustomerStory:CaseA

14. [Requirement]
• Experience overall Azure Data Service
• ETL Process(orchestration) is the highest priority
• Initial data ingestion & daily batch process
• Comfortable for co-working with business dept.
[Compete]
• AWS, Google
CustomerStory:CaseB

15. ConnecttoAzureDataExplorer
Link to our WebSite
Azure Data Explorer

17. © Microsoft Corporation
Kusto は Azure内部や Windowsで使われてきたクエリー言語
KQL
• The Kusto Query Language
• Kusto Engine ：Kusto を処理する機構
Where to use Kusto
• Azure Log Analytics
• Azure Application Insight
• Windows Defender Advanced Thread Protection
• Azure Security Center
サンプルですぐに試せるます！
ぜひチェックしてみてください。
• Log Analytics https://aka.ms/LADemo
• Application Insights https://aka.ms/AIAnalyticsDemo
• ATP https://aka.ms/WinDefATP

18. © Microsoft Corporation
画面の操作イメージ
スキーマから
DB, TBL, 項目
などを選択
クエリーを記述して実行
データを展開 実行結果を表示

19. © Microsoft Corporation
他のソリューションとの組み合わせ例
VisualizationExploration ModelingAnalyticsIntegration /
Ingestion
Storage
Apache Kafka for
HDInsight
Azure Data Factory
Azure SQL Data
Warehouse
Azure Analysis
Services
Power BI
Azure Data Lake Storage
Azure Databricks
SparkRAzure Data Explorer

20. © Microsoft Corporation
Azure Data Lake Storage Gen2
ADLS Gen2 adds a high performance HDFS Endpoint to Azure Blob Storage and inherits the rich feature set of Azure
Blob Storage *
Object Tiering and Lifecycle Polic
y Management
AAD Integration, RBAC, Storage A
ccount Security
HA/DR support through ZRS and
RA-GRS
Common Blob Storage Fo
undation
Blob API Gen2 API
Server Backups, Archive Storag
e, Semi-structured Data
Unstructured O
bject Data
Hadoop File System, File and F
older Hierarchy, Granular ACLS
Atomic File Transactions
File Data

21. AzureDataExplorerコスト試算ツール
(http://aka.ms/adx.cost)