Azure data bricks ml

1. Azure Databricks
@vishwasnarayan5 Vishwas N
https://hacksterdude.web.app/

2. About me
I am a Podcaster : http://tiny.cc/vnrpodcast
I love to talking to techies
A Bibliophile
Passionate about Image dataset - Computer Vision
Now exploring Azure cloud

3. Spark SQL
+ Data
Frames
Streaming
MLLIb
Machine
Learning
Graph X
Graph
Computation
Eco - System
Spark Core API

4. Azure DataBricks Platform
DataBricks WorkSpace DataBricks WorkFlows
DataBricks RunTime
Data Bricks I/O(DBIO) DataBricks Serverless
DataBricks Enterprise Security
…….

5. NoSQL
Storage
SQL
H
a
d
o
o
p
In Memory
MPP
Machine
Learning
ETL
Queuing
Service
Move Data to/From
SQL Databases
Streaming
Streaming
Distributed
System
Coordination
B
A
T
C
H
Real Time
Full
Text
Search
Hadoop EcoSystem : This is it

6. Spark Cluster Architecture
Source : https://spark.apache.org/docs/latest/cluster-overview.html

7. Spark Core Engine
Spark SQL
Interactive
Queries
Spark Structured
Streaming
Stream processing
Spark MLlib
Machine
Learning
Yarn Mesos
Standalone
Scheduler
Spark
MLlib
Machine
Learning
Spark
Streaming
Stream
processing
GraphX
Graph
Computation

10. What is it all about?

11. Where MLFlow Makes
a Lot of Influence