SlideShare a Scribd company logo
1 of 8
Big Data @ Zulily
By Echo Li, Data Engineer,
eli@zulily.com
Data Services, BI and Big Data Analytics, Zulily
Where we are
Powerful and Flexible
2
BIDS Data Platform
CUSTOMER INTERACTION POINTS
WEBSTORE MEMBER ENGAGEMENT
EVENT MANAGEMENT VENDOR
MANAGEMENT
SUPPLY CHAIN ERP & BACK OFFICE
Site Mobile
Orders &
Payments
Content
Mgmt.
CRM
Relevancy
(personalization)
Messaging
Offers &
Promotions
Item
Master
Catalog &
Event
Workflow
Mgmt.
Planning PortalEDI / Data
Exchange
Purchase
Orders
Workflow
& Tools
Order
Mgmt.
Fulfillment
Mgmt.
Transportation
Warehouse &
Inventory
Mgmt.
Financial
(SAP
Enterprise)
Business
Intelligence
HRIS
Warehouse
Automation
Initiatives:
• Capacity & Scale
• Data driven decision
making
– Data for Everyone
• Better customer
experience through
Personalization &
Targeting
How We Do It
…powered by Hortonworks Data Platform & Google Cloud
Tableau (Visualization & Reporting) Data Services (ZATA API)
Google BigQuery
Big Data Platform - Google Compute Engine
Hortonworks Data Platform 2.1 on Google Cloud
HDFS YARN HIVE/TEZ AMBARI
Google Cloud Storage
Platform Tools (zulily Build)
ZuSync (ETL) ZuScheduler (Scheduling) ZuMon (Data Monitoring)
Customer Data Mart Merch DataMart Supply Chain DataMart
Clickstream/Web Analytics
Data Processing Pipeline & Analytics
2014 zulily Proprietary and Confidential
4
Operational
Systems
External APIs
(Google, FB, Yahoo, Bing etc)
Hadoop Processing in Cloud
Real Time
ZuSync
Landing
Zone(LZ)
Staging(stg)
AtomicData
Store(ADS)
Aggregated
Dataset
Tier 1 ETLWF Tier 2 ETLWF
Cust ADS
Order ADS
Clickstrea
m
Big Query Tables
Our Journey…
5
Data Platform V1.0
Technology Stack:
• SQL Server
Challenges:
• Scale & Only supported
structured relational data
Advantages:
• Simple
• All data in same data store
• Makes it easy for
visualization, analytics and
reporting
Data Platform V2.0
Technology Stack:
• SQL Server, Apache Hadoop
Challenges:
• Lack of single data store
• Unable to mash up data
across structured and
unstructured data
• Difficult to scale visualization
with large scale data
Advantage:
• Ability to process
unstructured data at scale
• Tableau allows us to have
single visualization layer on
top of all data
Modern Data Platform V3.0
Technology Stack:
• Hadoop, Google Cloud Platform, Big
Query
Challenges:
• New Pricing Model which is good and
bad
• Requires new data processing
methodology(especially for structured
data)
Advantages:
• Supports Scale, high Speed
• Single Data Platform for structured and
unstructured data
• Enables scenarios which were difficult
to achieve in V1.0 or V2.0
• Enterprise Hadoop capabilities enable
management, monitoring and workflow
definition which are critical
Use Cases…
Use Case#1: Site & Event Funnel Analysis
Google
Cloud
Storage
Hadoop/GCE
Web
Servers
zulily
data
API
BigQuery
Funnel Analysis
ZATA(DATA API)
Reporting &
Analysis
(Powered by
Tableau)
Benefits
Increase Revenue
Improve marketing strategy and
targeting
Improve business decisions
Hadoop
/GCE
Use Case #3: Supply Chain Visibility
zulily
Sync
Others
Carriers
Google
Cloud
Storage
Order
Visibility
BigQuery
In Transit
Shipment
PO
zulily SCS
PO Shipment
EDI
Flat File
Vendor
Data
Exch.
Benefits
End to end
order visibility
Manage by exception
Reduce shipping costs
As our Journey Continues… we need more talents !!!
Please check out our career page:
http://www.zulily.com/careers

More Related Content

What's hot

Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
Cortana analytics ou comment office 365 peut rendre vos données plus intellig...Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
Nicolas Georgeault
 

What's hot (20)

From ingest to insights with AWS
From ingest to insights with AWSFrom ingest to insights with AWS
From ingest to insights with AWS
 
8 ways qlik integrates with salesforce.com
8 ways qlik integrates with salesforce.com8 ways qlik integrates with salesforce.com
8 ways qlik integrates with salesforce.com
 
Understanding Cortana Intelligence Suite & Power BI Demo
Understanding Cortana Intelligence Suite & Power BI DemoUnderstanding Cortana Intelligence Suite & Power BI Demo
Understanding Cortana Intelligence Suite & Power BI Demo
 
Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
Cortana analytics ou comment office 365 peut rendre vos données plus intellig...Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
Cortana analytics ou comment office 365 peut rendre vos données plus intellig...
 
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
Achieving Massive Concurrency & Sub-second Query Latency on Cloud Warehouses ...
 
Why Finance Should Consider Agile Modern Data Delivery Platform
Why Finance Should Consider Agile Modern Data Delivery PlatformWhy Finance Should Consider Agile Modern Data Delivery Platform
Why Finance Should Consider Agile Modern Data Delivery Platform
 
Systems of Intelligence: The Biggest Change in Enterprise Applications in 50 ...
Systems of Intelligence: The Biggest Change in Enterprise Applications in 50 ...Systems of Intelligence: The Biggest Change in Enterprise Applications in 50 ...
Systems of Intelligence: The Biggest Change in Enterprise Applications in 50 ...
 
Why HR Should Consider Agile Modern Data Delivery Platform
Why HR Should Consider Agile Modern Data Delivery PlatformWhy HR Should Consider Agile Modern Data Delivery Platform
Why HR Should Consider Agile Modern Data Delivery Platform
 
Finance Analytics
Finance AnalyticsFinance Analytics
Finance Analytics
 
Customer Event Hub – a modern Customer 360° view with DataStax Enterprise (DSE)
Customer Event Hub – a modern Customer 360° view with DataStax Enterprise (DSE) Customer Event Hub – a modern Customer 360° view with DataStax Enterprise (DSE)
Customer Event Hub – a modern Customer 360° view with DataStax Enterprise (DSE)
 
Hybrid IT: Legg Mason
Hybrid IT: Legg MasonHybrid IT: Legg Mason
Hybrid IT: Legg Mason
 
DesignMind Data Analytics Consulting
DesignMind Data Analytics Consulting DesignMind Data Analytics Consulting
DesignMind Data Analytics Consulting
 
Why Business Intelligence Should Consider Agile Modern Data Delivery Platform
Why Business Intelligence Should Consider Agile Modern Data Delivery PlatformWhy Business Intelligence Should Consider Agile Modern Data Delivery Platform
Why Business Intelligence Should Consider Agile Modern Data Delivery Platform
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
 
Neumann Data Platform
Neumann Data PlatformNeumann Data Platform
Neumann Data Platform
 
Dynamics Day 2015: Dynamics AX and Enterprise Cloud Computing
Dynamics Day 2015: Dynamics AX and Enterprise Cloud ComputingDynamics Day 2015: Dynamics AX and Enterprise Cloud Computing
Dynamics Day 2015: Dynamics AX and Enterprise Cloud Computing
 
Informatica Cloud Winter 2016 Release Webinar
Informatica Cloud Winter 2016 Release WebinarInformatica Cloud Winter 2016 Release Webinar
Informatica Cloud Winter 2016 Release Webinar
 
Snowplow: where we came from and where we are going - March 2016
Snowplow: where we came from and where we are going - March 2016Snowplow: where we came from and where we are going - March 2016
Snowplow: where we came from and where we are going - March 2016
 
Datamensional Business Intelligence and Data Services
Datamensional Business Intelligence and Data ServicesDatamensional Business Intelligence and Data Services
Datamensional Business Intelligence and Data Services
 
Dynamics Day 2017 Adelaide - IoT, Machine Learning and Big Data actionable in...
Dynamics Day 2017 Adelaide - IoT, Machine Learning and Big Data actionable in...Dynamics Day 2017 Adelaide - IoT, Machine Learning and Big Data actionable in...
Dynamics Day 2017 Adelaide - IoT, Machine Learning and Big Data actionable in...
 

Similar to Big data at zulily

Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Denodo
 
Power BI - 2016 - Public
Power BI - 2016 - PublicPower BI - 2016 - Public
Power BI - 2016 - Public
Julian Payne
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine Learning
Provectus
 
SendGrid Improves Email Delivery with Hybrid Data Warehousing
SendGrid Improves Email Delivery with Hybrid Data WarehousingSendGrid Improves Email Delivery with Hybrid Data Warehousing
SendGrid Improves Email Delivery with Hybrid Data Warehousing
Amazon Web Services
 

Similar to Big data at zulily (20)

Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
Accelerate Self-Service Analytics with Virtualization and Visualisation (Thai)
 
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
 
Making the Most of Power BI with SQL Server 2014 and Azure
Making the Most of Power BI with SQL Server 2014 and AzureMaking the Most of Power BI with SQL Server 2014 and Azure
Making the Most of Power BI with SQL Server 2014 and Azure
 
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenariosThe Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
The Practice of Big Data - The Hadoop ecosystem explained with usage scenarios
 
Building Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureBuilding Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft Azure
 
Bringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to SalesforceBringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to Salesforce
 
Power BI - 2016 - Public
Power BI - 2016 - PublicPower BI - 2016 - Public
Power BI - 2016 - Public
 
Accelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and VisualizationAccelerate Self-Service Analytics with Data Virtualization and Visualization
Accelerate Self-Service Analytics with Data Virtualization and Visualization
 
Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...
Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...
Hybrid Analytics in Healthcare: Leveraging Power BI and Office 365 to Make Sm...
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
 
Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and TableauAnalyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
 
Skilwise Big data
Skilwise Big dataSkilwise Big data
Skilwise Big data
 
Feature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine LearningFeature Store as a Data Foundation for Machine Learning
Feature Store as a Data Foundation for Machine Learning
 
Retail & CPG
Retail & CPGRetail & CPG
Retail & CPG
 
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, ClouderaMongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
MongoDB IoT City Tour STUTTGART: Hadoop and future data management. By, Cloudera
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
 
SendGrid Improves Email Delivery with Hybrid Data Warehousing
SendGrid Improves Email Delivery with Hybrid Data WarehousingSendGrid Improves Email Delivery with Hybrid Data Warehousing
SendGrid Improves Email Delivery with Hybrid Data Warehousing
 
Derfor skal du bruge en DataLake
Derfor skal du bruge en DataLakeDerfor skal du bruge en DataLake
Derfor skal du bruge en DataLake
 
2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoption2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoption
 
OAC Workshop - Detroit 2019
OAC Workshop -  Detroit 2019OAC Workshop -  Detroit 2019
OAC Workshop - Detroit 2019
 

Recently uploaded

Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Bertram Ludäscher
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
HyderabadDolls
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
ZurliaSoop
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
nirzagarg
 
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
gajnagarg
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
nirzagarg
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
wsppdmt
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
nirzagarg
 
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
gajnagarg
 

Recently uploaded (20)

Top Call Girls in Balaghat 9332606886Call Girls Advance Cash On Delivery Ser...
Top Call Girls in Balaghat  9332606886Call Girls Advance Cash On Delivery Ser...Top Call Girls in Balaghat  9332606886Call Girls Advance Cash On Delivery Ser...
Top Call Girls in Balaghat 9332606886Call Girls Advance Cash On Delivery Ser...
 
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...Reconciling Conflicting Data Curation Actions:  Transparency Through Argument...
Reconciling Conflicting Data Curation Actions: Transparency Through Argument...
 
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
SAC 25 Final National, Regional & Local Angel Group Investing Insights 2024 0...
 
Charbagh + Female Escorts Service in Lucknow | Starting ₹,5K To @25k with A/C...
Charbagh + Female Escorts Service in Lucknow | Starting ₹,5K To @25k with A/C...Charbagh + Female Escorts Service in Lucknow | Starting ₹,5K To @25k with A/C...
Charbagh + Female Escorts Service in Lucknow | Starting ₹,5K To @25k with A/C...
 
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
Sealdah % High Class Call Girls Kolkata - 450+ Call Girl Cash Payment 8005736...
 
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With OrangePredicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
Predicting HDB Resale Prices - Conducting Linear Regression Analysis With Orange
 
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
Jual Obat Aborsi Surabaya ( Asli No.1 ) 085657271886 Obat Penggugur Kandungan...
 
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
Top profile Call Girls In Purnia [ 7014168258 ] Call Me For Genuine Models We...
 
20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf20240412-SmartCityIndex-2024-Full-Report.pdf
20240412-SmartCityIndex-2024-Full-Report.pdf
 
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
Top profile Call Girls In Vadodara [ 7014168258 ] Call Me For Genuine Models ...
 
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
Gulbai Tekra * Cheap Call Girls In Ahmedabad Phone No 8005736733 Elite Escort...
 
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Hapur [ 7014168258 ] Call Me For Genuine Models We ...
 
Ranking and Scoring Exercises for Research
Ranking and Scoring Exercises for ResearchRanking and Scoring Exercises for Research
Ranking and Scoring Exercises for Research
 
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
如何办理英国诺森比亚大学毕业证(NU毕业证书)成绩单原件一模一样
 
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
5CL-ADBA,5cladba, Chinese supplier, safety is guaranteed
 
Statistics notes ,it includes mean to index numbers
Statistics notes ,it includes mean to index numbersStatistics notes ,it includes mean to index numbers
Statistics notes ,it includes mean to index numbers
 
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
Top profile Call Girls In Bihar Sharif [ 7014168258 ] Call Me For Genuine Mod...
 
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
Digital Advertising Lecture for Advanced Digital & Social Media Strategy at U...
 
Digital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham WareDigital Transformation Playbook by Graham Ware
Digital Transformation Playbook by Graham Ware
 
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
Top profile Call Girls In Latur [ 7014168258 ] Call Me For Genuine Models We ...
 

Big data at zulily

  • 1. Big Data @ Zulily By Echo Li, Data Engineer, eli@zulily.com Data Services, BI and Big Data Analytics, Zulily
  • 2. Where we are Powerful and Flexible 2 BIDS Data Platform CUSTOMER INTERACTION POINTS WEBSTORE MEMBER ENGAGEMENT EVENT MANAGEMENT VENDOR MANAGEMENT SUPPLY CHAIN ERP & BACK OFFICE Site Mobile Orders & Payments Content Mgmt. CRM Relevancy (personalization) Messaging Offers & Promotions Item Master Catalog & Event Workflow Mgmt. Planning PortalEDI / Data Exchange Purchase Orders Workflow & Tools Order Mgmt. Fulfillment Mgmt. Transportation Warehouse & Inventory Mgmt. Financial (SAP Enterprise) Business Intelligence HRIS Warehouse Automation Initiatives: • Capacity & Scale • Data driven decision making – Data for Everyone • Better customer experience through Personalization & Targeting
  • 3. How We Do It …powered by Hortonworks Data Platform & Google Cloud Tableau (Visualization & Reporting) Data Services (ZATA API) Google BigQuery Big Data Platform - Google Compute Engine Hortonworks Data Platform 2.1 on Google Cloud HDFS YARN HIVE/TEZ AMBARI Google Cloud Storage Platform Tools (zulily Build) ZuSync (ETL) ZuScheduler (Scheduling) ZuMon (Data Monitoring) Customer Data Mart Merch DataMart Supply Chain DataMart Clickstream/Web Analytics
  • 4. Data Processing Pipeline & Analytics 2014 zulily Proprietary and Confidential 4 Operational Systems External APIs (Google, FB, Yahoo, Bing etc) Hadoop Processing in Cloud Real Time ZuSync Landing Zone(LZ) Staging(stg) AtomicData Store(ADS) Aggregated Dataset Tier 1 ETLWF Tier 2 ETLWF Cust ADS Order ADS Clickstrea m Big Query Tables
  • 5. Our Journey… 5 Data Platform V1.0 Technology Stack: • SQL Server Challenges: • Scale & Only supported structured relational data Advantages: • Simple • All data in same data store • Makes it easy for visualization, analytics and reporting Data Platform V2.0 Technology Stack: • SQL Server, Apache Hadoop Challenges: • Lack of single data store • Unable to mash up data across structured and unstructured data • Difficult to scale visualization with large scale data Advantage: • Ability to process unstructured data at scale • Tableau allows us to have single visualization layer on top of all data Modern Data Platform V3.0 Technology Stack: • Hadoop, Google Cloud Platform, Big Query Challenges: • New Pricing Model which is good and bad • Requires new data processing methodology(especially for structured data) Advantages: • Supports Scale, high Speed • Single Data Platform for structured and unstructured data • Enables scenarios which were difficult to achieve in V1.0 or V2.0 • Enterprise Hadoop capabilities enable management, monitoring and workflow definition which are critical
  • 6. Use Cases… Use Case#1: Site & Event Funnel Analysis Google Cloud Storage Hadoop/GCE Web Servers zulily data API BigQuery Funnel Analysis ZATA(DATA API) Reporting & Analysis (Powered by Tableau) Benefits Increase Revenue Improve marketing strategy and targeting Improve business decisions
  • 7. Hadoop /GCE Use Case #3: Supply Chain Visibility zulily Sync Others Carriers Google Cloud Storage Order Visibility BigQuery In Transit Shipment PO zulily SCS PO Shipment EDI Flat File Vendor Data Exch. Benefits End to end order visibility Manage by exception Reduce shipping costs
  • 8. As our Journey Continues… we need more talents !!! Please check out our career page: http://www.zulily.com/careers

Editor's Notes

  1. Relational data is doubling every few months Non relational data is growing even faster It’s not about clicks its about impressions It’s not about who visited the site but who did not Data was fragmented across different stores limiting analytics More people more need for faster data