SlideShare a Scribd company logo
1 of 27
Copyright © 2019 Impetus Technologies, Inc.
You are prohibited from making a copy or modification of, or from redistributing, rebroadcasting, or
re-encoding of this content without the prior written consent of Impetus.
This presentation may include images from other products and services. These images are used for
illustrative purposes only. Unless explicitly stated there is no implied endorsement or sponsorship of
these products by Impetus. All copyrights and trademarks are property of their respective owners.
Automate and Optimize
Data Warehouse Migration
to Snowflake
IMP
Our Mission
Enabling a unified, clear, and present view of your business
Contents
Cloud migration strategy
How Impetus can help
Automated workload transformation to Snowflake
Demo
Q&A
Poll
Speakers
Barry Tuthill
VP Field Operations
Impetus Technologies
Sanjay Sharma
Principal Architect
Impetus Technologies
Cloud Migration Strategy
Key Considerations
Business
• BAU
• Existing investments
• Risk management
• Cost and scale
• People, processes, etc.
• Interoperability
Key Considerations
Technical
• Architecture (Tech stack)
• Performance
• Migration blueprint
• Automation
• Parallel-run period
Top 5 Concerns
Existing investments
Migrate as is vs. total re-engineering
Extent of automation
Risk and uncertainty
Performance and cost
Typical Migration Checklist
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
Migration Checklist
Reuse – Embrace what you already have
Automate – Leverage automation for faster time-to-value
Optimize – Meet the performance SLA and budget on cloud
Certify – Thoroughly validate before you put them into production
How Impetus Can Help
End-to-end Automated Migration
Assessment and recommendations for migration
Optimized Schema (DDL) transformation
Data migration – one time and incremental pipelines setup
Workload logic (DML) conversion to SnowSQL
Stored Procedure conversion to Snow JavaScript procedures, Spark or Python wrappers
Scheduling/ orchestration logic conversion to Shell or Python or Cloud native wrappers
Agile DevOps, Continuous integration and delivery process setup
Validation and certification
Automating the Complete Migration Lifecycle
1
2
3
4
Assess
Transform
Validate
Execute
Assessment
Automated profiling
Identification of workloads and dependencies
Creation of optimized schema
Grouping of workloads
Transformation
90% automated code conversion to SnowSQL
Automated data migration to an optimized schema
Automated handling of data types, views, UDFs and stored procedures
Creation of patterns for the target platform
Auto-generation of patterns for newer migrations
Query-editing for optimized fixes and performance tuning
Validation and Execution
Auto-generation of validation/ reconciliation scripts
Execution through cloud ready orchestration/ execution engine
Integrated DevOps and Agile processes setup
Accelerated decommissioning methodology to retire legacy footprint
Multi cloud ready with security and governance
Automated Workload Transformation to Snowflake
ENTERPRISE
DATA WAREHOUSE
VALIDATE
Query Logs
Script Files
Data
Query
BIG DATA
WAREHOUSE
ASSESS EXECUTETRANSFORM
Data - S3/Spark, Snowpipe, ETL
SQL,
Stored
Procedures,
Scheduling/
Workflow
LOGIC TRANSFORMATION
SnowSQL
Snow
JavaScript
procedures/
Spark/ Python /
Wrappers
Ingestion,
Scheduler/
workflow setup
Demo
Business Benefits
Accelerate time-to-value
Meet performance SLAs
Decrease risk and uncertainty
Automate end-to-end migration
Save up to 80% in cost, time, and effort
Maximize ROI
Poll Results and Feedback
Q&A
Thank You
Visit www.impetus.com or get in touch with us at inquiry@impetus.com

More Related Content

What's hot

Azure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overviewAzure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overviewGeorge Walters
 
Azure SQL Database Managed Instance
Azure SQL Database Managed InstanceAzure SQL Database Managed Instance
Azure SQL Database Managed InstanceJames Serra
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMatillion
 
Microsoft Azure Technical Overview
Microsoft Azure Technical OverviewMicrosoft Azure Technical Overview
Microsoft Azure Technical Overviewgjuljo
 
Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeKent Graziano
 
Databricks Platform.pptx
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptxAlex Ivy
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
Introducing Azure SQL Database
Introducing Azure SQL DatabaseIntroducing Azure SQL Database
Introducing Azure SQL DatabaseJames Serra
 
Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglyTyler Wishnoff
 
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar SeriesMigrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar SeriesAmazon Web Services
 
Demystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWDemystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWKent Graziano
 
Snowflake Data Governance
Snowflake Data GovernanceSnowflake Data Governance
Snowflake Data Governancessuser538b022
 
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020Timothy McAliley
 
Building Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureBuilding Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureDmitry Anoshin
 
DevOps + DataOps = Digital Transformation
DevOps + DataOps = Digital Transformation DevOps + DataOps = Digital Transformation
DevOps + DataOps = Digital Transformation Delphix
 
Preparing for a future Microservices journey using DDD & Wardley Maps
Preparing for a future Microservices journey using DDD & Wardley MapsPreparing for a future Microservices journey using DDD & Wardley Maps
Preparing for a future Microservices journey using DDD & Wardley MapsSusanne Kaiser
 

What's hot (20)

Azure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overviewAzure SQL Database Managed Instance - technical overview
Azure SQL Database Managed Instance - technical overview
 
Azure SQL Database Managed Instance
Azure SQL Database Managed InstanceAzure SQL Database Managed Instance
Azure SQL Database Managed Instance
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
 
Microsoft Azure Technical Overview
Microsoft Azure Technical OverviewMicrosoft Azure Technical Overview
Microsoft Azure Technical Overview
 
Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with Snowflake
 
Databricks Platform.pptx
Databricks Platform.pptxDatabricks Platform.pptx
Databricks Platform.pptx
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
Snowflake Datawarehouse Architecturing
Snowflake Datawarehouse ArchitecturingSnowflake Datawarehouse Architecturing
Snowflake Datawarehouse Architecturing
 
Introducing Azure SQL Database
Introducing Azure SQL DatabaseIntroducing Azure SQL Database
Introducing Azure SQL Database
 
Cloud Migration: A How-To Guide
Cloud Migration: A How-To GuideCloud Migration: A How-To Guide
Cloud Migration: A How-To Guide
 
Snowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the UglySnowflake: The Good, the Bad, and the Ugly
Snowflake: The Good, the Bad, and the Ugly
 
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar SeriesMigrate your Data Warehouse to Amazon Redshift - September Webinar Series
Migrate your Data Warehouse to Amazon Redshift - September Webinar Series
 
Demystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFWDemystifying Data Warehousing as a Service - DFW
Demystifying Data Warehousing as a Service - DFW
 
Snowflake Data Governance
Snowflake Data GovernanceSnowflake Data Governance
Snowflake Data Governance
 
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020
Data Migration to Azure SQL and Azure SQL Managed Instance - June 19 2020
 
Building Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft AzureBuilding Modern Data Platform with Microsoft Azure
Building Modern Data Platform with Microsoft Azure
 
DevOps + DataOps = Digital Transformation
DevOps + DataOps = Digital Transformation DevOps + DataOps = Digital Transformation
DevOps + DataOps = Digital Transformation
 
Preparing for a future Microservices journey using DDD & Wardley Maps
Preparing for a future Microservices journey using DDD & Wardley MapsPreparing for a future Microservices journey using DDD & Wardley Maps
Preparing for a future Microservices journey using DDD & Wardley Maps
 
Azure purview
Azure purviewAzure purview
Azure purview
 
Elastic Data Warehousing
Elastic Data WarehousingElastic Data Warehousing
Elastic Data Warehousing
 

Similar to Automate and Optimize Data Warehouse Migration to Snowflake

The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...Impetus Technologies
 
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarKeys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarImpetus Technologies
 
AWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAmazon Web Services
 
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationLeverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationCapgemini
 
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...Amazon Web Services
 
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Amazon Web Services
 
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarAutomated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarImpetus Technologies
 
tero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppttero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.pptTero Peltola
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationAmazon Web Services
 
Deep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudDeep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudNGA Human Resources
 
Creating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationCreating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationTom Laszewski
 
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Amazon Web Services
 
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014Amazon Web Services
 
Digital transformation slideshare
Digital transformation   slideshareDigital transformation   slideshare
Digital transformation slideshareShivamPatsariya1
 
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesSuccessful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesCprime
 
Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Oliver Wells
 
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceAWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceTom Laszewski
 
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionAccenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionTom Laszewski
 

Similar to Automate and Optimize Data Warehouse Migration to Snowflake (20)

The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
The fastest way to convert etl analytics and data warehouse to AWS- Impetus W...
 
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinarKeys to establish sustainable DW and analytics on the cloud -Impetus webinar
Keys to establish sustainable DW and analytics on the cloud -Impetus webinar
 
AWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud MigrationsAWS Summit Singapore - Secrets to Successful Cloud Migrations
AWS Summit Singapore - Secrets to Successful Cloud Migrations
 
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud MigrationLeverage the Power of SAP HANA with Microsoft Azure Cloud Migration
Leverage the Power of SAP HANA with Microsoft Azure Cloud Migration
 
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
AWS Summit Singapore Webinar Edition | Move it! Migrating to AWS (Level 200) ...
 
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
Migrating Thousands of Workloads to AWS at Enterprise Scale – Chris Wegmann, ...
 
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus WebinarAutomated EDW Assessment and Actionable Recommendations - Impetus Webinar
Automated EDW Assessment and Actionable Recommendations - Impetus Webinar
 
tero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppttero-peltola-serverlessMeetup-10.11.2022.ppt
tero-peltola-serverlessMeetup-10.11.2022.ppt
 
Where to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio MigrationWhere to Begin? Application Portfolio Migration
Where to Begin? Application Portfolio Migration
 
Deep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR CloudDeep Dive. How to Transition SAP Payroll to HR Cloud
Deep Dive. How to Transition SAP Payroll to HR Cloud
 
Creating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organizationCreating an Operating Model to enable a high frequency organization
Creating an Operating Model to enable a high frequency organization
 
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
Demystifying Cloud Economics - How to Build an Investment Case for Scale Migr...
 
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014(ENT306) Application Portfolio Migration | AWS re:Invent 2014
(ENT306) Application Portfolio Migration | AWS re:Invent 2014
 
original.pptx
original.pptxoriginal.pptx
original.pptx
 
Digital transformation slideshare
Digital transformation   slideshareDigital transformation   slideshare
Digital transformation slideshare
 
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life ExamplesSuccessful Atlassian Cloud Migrations and Optimizations: Real Life Examples
Successful Atlassian Cloud Migrations and Optimizations: Real Life Examples
 
Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS) Ritech Solutions - Go For Launch Overview (AWS)
Ritech Solutions - Go For Launch Overview (AWS)
 
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive GuidanceAWS Cloud Center Excellence Quick Start Prescriptive Guidance
AWS Cloud Center Excellence Quick Start Prescriptive Guidance
 
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout SessionAccenture 2014 AWS re:Invent Enterprise Migration Breakout Session
Accenture 2014 AWS re:Invent Enterprise Migration Breakout Session
 
Migration Planning
Migration PlanningMigration Planning
Migration Planning
 

More from Impetus Technologies

Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Impetus Technologies
 
Building a mature foundation for life in the cloud
Building a mature foundation for life in the cloudBuilding a mature foundation for life in the cloud
Building a mature foundation for life in the cloudImpetus Technologies
 
Best practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarBest practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarImpetus Technologies
 
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarInstantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarImpetus Technologies
 
Solving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarSolving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarImpetus Technologies
 
Anomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleAnomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleImpetus Technologies
 
Keys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataKeys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataImpetus Technologies
 
Build Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesBuild Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesImpetus Technologies
 
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Impetus Technologies
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Impetus Technologies
 
Streaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkStreaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkImpetus Technologies
 
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationImpetus Technologies
 
The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...Impetus Technologies
 
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Impetus Technologies
 
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxAnomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxImpetus Technologies
 

More from Impetus Technologies (16)

Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...Eliminate cyber-security threats using data analytics – Build a resilient ent...
Eliminate cyber-security threats using data analytics – Build a resilient ent...
 
Building a mature foundation for life in the cloud
Building a mature foundation for life in the cloudBuilding a mature foundation for life in the cloud
Building a mature foundation for life in the cloud
 
Best practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus WebinarBest practices to build a sustainable data lake on cloud - Impetus Webinar
Best practices to build a sustainable data lake on cloud - Impetus Webinar
 
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinarInstantly convert Teradata ETL and EDW to Spark- Impetus webinar
Instantly convert Teradata ETL and EDW to Spark- Impetus webinar
 
Solving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinarSolving the EDW transformation conundrum - Impetus webinar
Solving the EDW transformation conundrum - Impetus webinar
 
Anomaly detection with machine learning at scale
Anomaly detection with machine learning at scaleAnomaly detection with machine learning at scale
Anomaly detection with machine learning at scale
 
Keys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of DataKeys to Formulating an Effective Data Management Strategy in the Age of Data
Keys to Formulating an Effective Data Management Strategy in the Age of Data
 
Build Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in MinutesBuild Spark-based ETL Workflows on Cloud in Minutes
Build Spark-based ETL Workflows on Cloud in Minutes
 
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
Planning your Next-Gen Change Data Capture (CDC) Architecture in 2019 - Strea...
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
 
Streaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache SparkStreaming Analytics for IoT with Apache Spark
Streaming Analytics for IoT with Apache Spark
 
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live ImplementationAnomaly Detection - Real World Scenarios, Approaches and Live Implementation
Anomaly Detection - Real World Scenarios, Approaches and Live Implementation
 
The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...The structured streaming upgrade to Apache Spark and how enterprises can bene...
The structured streaming upgrade to Apache Spark and how enterprises can bene...
 
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...Apache spark empowering the real time data driven enterprise - StreamAnalytix...
Apache spark empowering the real time data driven enterprise - StreamAnalytix...
 
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptxAnomaly Detection and Spark Implementation - Meetup Presentation.pptx
Anomaly Detection and Spark Implementation - Meetup Presentation.pptx
 
Importance of Big Data Analytics
Importance of Big Data AnalyticsImportance of Big Data Analytics
Importance of Big Data Analytics
 

Recently uploaded

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreternaman860154
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilV3cube
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 

Recently uploaded (20)

Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
Developing An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of BrazilDeveloping An App To Navigate The Roads of Brazil
Developing An App To Navigate The Roads of Brazil
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 

Automate and Optimize Data Warehouse Migration to Snowflake

  • 1. Copyright © 2019 Impetus Technologies, Inc. You are prohibited from making a copy or modification of, or from redistributing, rebroadcasting, or re-encoding of this content without the prior written consent of Impetus. This presentation may include images from other products and services. These images are used for illustrative purposes only. Unless explicitly stated there is no implied endorsement or sponsorship of these products by Impetus. All copyrights and trademarks are property of their respective owners.
  • 2. Automate and Optimize Data Warehouse Migration to Snowflake IMP
  • 3. Our Mission Enabling a unified, clear, and present view of your business
  • 4. Contents Cloud migration strategy How Impetus can help Automated workload transformation to Snowflake Demo Q&A
  • 6. Speakers Barry Tuthill VP Field Operations Impetus Technologies Sanjay Sharma Principal Architect Impetus Technologies
  • 8. Key Considerations Business • BAU • Existing investments • Risk management • Cost and scale • People, processes, etc. • Interoperability
  • 9. Key Considerations Technical • Architecture (Tech stack) • Performance • Migration blueprint • Automation • Parallel-run period
  • 10. Top 5 Concerns Existing investments Migrate as is vs. total re-engineering Extent of automation Risk and uncertainty Performance and cost
  • 12. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 13. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 14. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 15. Migration Checklist Reuse – Embrace what you already have Automate – Leverage automation for faster time-to-value Optimize – Meet the performance SLA and budget on cloud Certify – Thoroughly validate before you put them into production
  • 17. End-to-end Automated Migration Assessment and recommendations for migration Optimized Schema (DDL) transformation Data migration – one time and incremental pipelines setup Workload logic (DML) conversion to SnowSQL Stored Procedure conversion to Snow JavaScript procedures, Spark or Python wrappers Scheduling/ orchestration logic conversion to Shell or Python or Cloud native wrappers Agile DevOps, Continuous integration and delivery process setup Validation and certification
  • 18. Automating the Complete Migration Lifecycle 1 2 3 4 Assess Transform Validate Execute
  • 19. Assessment Automated profiling Identification of workloads and dependencies Creation of optimized schema Grouping of workloads
  • 20. Transformation 90% automated code conversion to SnowSQL Automated data migration to an optimized schema Automated handling of data types, views, UDFs and stored procedures Creation of patterns for the target platform Auto-generation of patterns for newer migrations Query-editing for optimized fixes and performance tuning
  • 21. Validation and Execution Auto-generation of validation/ reconciliation scripts Execution through cloud ready orchestration/ execution engine Integrated DevOps and Agile processes setup Accelerated decommissioning methodology to retire legacy footprint Multi cloud ready with security and governance
  • 22. Automated Workload Transformation to Snowflake ENTERPRISE DATA WAREHOUSE VALIDATE Query Logs Script Files Data Query BIG DATA WAREHOUSE ASSESS EXECUTETRANSFORM Data - S3/Spark, Snowpipe, ETL SQL, Stored Procedures, Scheduling/ Workflow LOGIC TRANSFORMATION SnowSQL Snow JavaScript procedures/ Spark/ Python / Wrappers Ingestion, Scheduler/ workflow setup
  • 23. Demo
  • 24. Business Benefits Accelerate time-to-value Meet performance SLAs Decrease risk and uncertainty Automate end-to-end migration Save up to 80% in cost, time, and effort Maximize ROI
  • 25. Poll Results and Feedback
  • 26. Q&A
  • 27. Thank You Visit www.impetus.com or get in touch with us at inquiry@impetus.com

Editor's Notes

  1. This could be the poll questions How to leverage existing investments? Low hanging fruits Interdependent workloads
  2. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  3. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  4. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  5. Devise a strategy for existing inventory of workloads Identify inter-dependencies Migration blueprint Code transformation and optimization
  6. Data types – configuration driven mapping from source data warehouse to Snowflake data types
  7. Automated legacy data warehouse inventory and profiling Identification of workloads (metadata, data, etc.) and dependencies Creation of optimized schema (clustering keys, Parquet format/file size for S3 uploads, etc.) Grouping of workloads into migration units
  8. Up to 90% automated code conversion to SnowSQL Automated data migration to an optimized schema Automated handling of data types, views, intervals, loops, UDFs, procedures Creation of patterns for the target platform (ingestion, data sync, recon, lineage, security, orchestration, etc.) Auto-generation of patterns for newer migrations Query-editing for optimized fixes and performance tuning
  9. Data types – configuration driven mapping from source data warehouse to Snowflake data types