SlideShare una empresa de Scribd logo
1 de 10
Descargar para leer sin conexión
What is ETL?
                        Extraction, Transformation, Loading

Simple Example of ETL


                                Customer      Customer
                                   ID          Name


                               105           Sainsbury
        Master Data

                               102           Tesco


                               109           Waitrose


                               101           Asda



                                                              By
                                                              Karthikeyan Selvaraj
Let’s say the master data table here is a flat file ie excel file which is in your computer .
                   We need to bring this table into SAP BI platform




                                                                        Customer Customer
SAP BI Platform                                                            ID     Name

                                                                        105         Sainsbury

                                                                        102         Tesco

                                                                        109         Waitrose

                                                                        101         Asda




                                                                       By
                                                                       Karthikeyan Selvaraj
The first step is to extract the master data table ie excel file into BI-data warehouse
The components needed for extracting the data into BI data warehouse are
1. DataSource
2. InfoPackage

1. DataSource



    DataSource
                                                  DataSource: It defines about the data.
                                                 For eg: Once I finish this presentation, I
  What type of                                    will choose a location to save this ppt
  data?                                          and I also define in what version I want
  Where the                                      to save this ppt similarly, In datasource
  data is                                             we will define about the data.
  located?




                                                                       By
                                                                       Karthikeyan Selvaraj
The first step is to extract the master data table ie excel file into BI-data warehouse
 The components needed for extracting the data into BI data warehouse are
 1. DataSource
 2. InfoPackage

  2. InfoPackage

    What is InfoPackage?
    In simple words we can define InfoPackage, It is like a key to open and enter into a
    room.
    It helps to bring the data from a legacy system or SAP system. For our scenario it
    helps to bring the data from our computer into BI datawarehouse.

        Customer      Customer                            DataSource
Excel      ID          Name
File    105          Sainsbury
                                                        What type of
        102          Tesco                              data?
        109          Waitrose                           Where the
        101          Asda                               data is
                                                        located?
                                     InfoPackage                       By
           Computer                                   BI Datawarehouse Karthikeyan Selvaraj
Now we have moved the master data table into BI datawarehouse by executing the
 InfoPackage
 Once the data comes into BI, It is stored in a table called PSA (Persistent Staging Area)
 The data that comes inside from any source system will be stored temporarily in PSA.

Excel
File
  Customer    Customer                 DataSource
                                                                     PSA
     ID        Name
 105         Sainsbury                                     Customer     Customer
                                   What type of               ID         Name
 102         Tesco                 data?
                                                           105         Sainsbury
 109         Waitrose              Where the
                                   data is                 102         Tesco
 101         Asda                  located?                109         Waitrose

                         InfoPackage                       101         Asda


                                                  BI Datawarehouse
        Computer
                                                                       By
                                                                       Karthikeyan Selvaraj
Transformation of Data
The first part of ETL ie Extraction is done successfully. Now we need to transform the data
so that it can be made more optimized for reporting.
In order to do that, we define fields of the table as Info Objects. In our master data table
we have two fields ie Customer ID and Customer Name so in BI we define them as Info
Objects.
Info Objects are divided into three types
1. Characteristics – sorting keys such as company code, product ID, etc.
2. Key Figures – quantity, amount or number of items. Data that can be manipulated.
3. Units – currency, measure this all comes under unit.
 Customer ID and Customer name are characteristic Info Objects.
           PSA
 Customer     Customer                                     Customer ID
    ID         Name                                         Info Object
105           Sainsbury                                   Customer Name
                                                            Info Object
102           Tesco
109           Waitrose
101           Asda
                                                                         By
                                       Characteristic Info Object        Karthikeyan Selvaraj
Transformation of Data
The attribute for Customer ID is Customer name
In database we define the attributes for primary key similarly we need to define the
attributes for master data field ie for Customer ID.
Once that is done we do the mapping ie transformation. We map the fields of the
DataSource to the fields of the Info Objects


                                                           InfoProvider
            DataSource


           Customer ID                                     Customer ID
                                                            Info Object
                                 Transformation
             Customer                                    Customer Name
              Name                                         Info Object




                                                                        By
                                                                        Karthikeyan Selvaraj
Loading
Once the mapping is done, data has to be transferred from DataSource (PSA Table) to
InfoProvider ( Info Objects)
This is done by a process called Data Transfer Process (DTP).
How?: We create the DTP in InfoProvider layer and activate it. After activation we execute
the DTP (Data Transfer Process). Now the Data from the PSA Table are transferred to their
respective InfoObjects.

                                                           InfoProvider
            DataSource


           Customer ID                                    Customer ID
                                                           Info Object
                                 Transformation
             Customer                                   Customer Name
              Name                                        Info Object




                                     DTP
                                                                       By
                                                                       Karthikeyan Selvaraj
Loading
Data are moved to their respective InfoObjects as per their mapping and it’s ready for
reporting from the InfoProvider Layer.

                                     InfoProvider



                      Customer ID            Customer Name
                       Info Object             Info Object

                           105                  Sainsbury
                           102                      Tesco
                           109                  Waitrose
                           101                      Asda




                                                                    By
                                                                    Karthikeyan Selvaraj
Thank You

            By
            Karthikeyan Selvaraj

Más contenido relacionado

La actualidad más candente

Informatica PowerCenter
Informatica PowerCenterInformatica PowerCenter
Informatica PowerCenterRamy Mahrous
 
Scaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksScaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksDatabricks
 
Talend Open Studio Data Integration
Talend Open Studio Data IntegrationTalend Open Studio Data Integration
Talend Open Studio Data IntegrationRoberto Marchetto
 
Data warehouse 21 snowflake schema
Data warehouse 21 snowflake schemaData warehouse 21 snowflake schema
Data warehouse 21 snowflake schemaVaibhav Khanna
 
Etl overview training
Etl overview trainingEtl overview training
Etl overview trainingMondy Holten
 
What is Informatica Powercenter
What is Informatica PowercenterWhat is Informatica Powercenter
What is Informatica PowercenterBigClasses Com
 
ETL VS ELT.pdf
ETL VS ELT.pdfETL VS ELT.pdf
ETL VS ELT.pdfBOSupport
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMatillion
 
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...Edureka!
 
OLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEOLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEZalpa Rathod
 
Introduction to Data Engineer and Data Pipeline at Credit OK
Introduction to Data Engineer and Data Pipeline at Credit OKIntroduction to Data Engineer and Data Pipeline at Credit OK
Introduction to Data Engineer and Data Pipeline at Credit OKKriangkrai Chaonithi
 
ETL Process & Data Warehouse Fundamentals
ETL Process & Data Warehouse FundamentalsETL Process & Data Warehouse Fundamentals
ETL Process & Data Warehouse FundamentalsSOMASUNDARAM T
 
Cloud-native Semantic Layer on Data Lake
Cloud-native Semantic Layer on Data LakeCloud-native Semantic Layer on Data Lake
Cloud-native Semantic Layer on Data LakeDatabricks
 
Microsoft Azure Data Factory Data Flow Scenarios
Microsoft Azure Data Factory Data Flow ScenariosMicrosoft Azure Data Factory Data Flow Scenarios
Microsoft Azure Data Factory Data Flow ScenariosMark Kromer
 
Etl process in data warehouse
Etl process in data warehouseEtl process in data warehouse
Etl process in data warehouseKomal Choudhary
 

La actualidad más candente (20)

ETL Technologies.pptx
ETL Technologies.pptxETL Technologies.pptx
ETL Technologies.pptx
 
Informatica PowerCenter
Informatica PowerCenterInformatica PowerCenter
Informatica PowerCenter
 
Scaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with DatabricksScaling and Modernizing Data Platform with Databricks
Scaling and Modernizing Data Platform with Databricks
 
Talend Open Studio Data Integration
Talend Open Studio Data IntegrationTalend Open Studio Data Integration
Talend Open Studio Data Integration
 
Data warehouse 21 snowflake schema
Data warehouse 21 snowflake schemaData warehouse 21 snowflake schema
Data warehouse 21 snowflake schema
 
Etl overview training
Etl overview trainingEtl overview training
Etl overview training
 
What is Informatica Powercenter
What is Informatica PowercenterWhat is Informatica Powercenter
What is Informatica Powercenter
 
ETL VS ELT.pdf
ETL VS ELT.pdfETL VS ELT.pdf
ETL VS ELT.pdf
 
Master the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - SnowflakeMaster the Multi-Clustered Data Warehouse - Snowflake
Master the Multi-Clustered Data Warehouse - Snowflake
 
Etl techniques
Etl techniquesEtl techniques
Etl techniques
 
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...
Talend Components | tMap, tJoin, tFileList, tInputFileDelimited | Talend Onli...
 
OLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSEOLAP & DATA WAREHOUSE
OLAP & DATA WAREHOUSE
 
Introduction to Data Engineer and Data Pipeline at Credit OK
Introduction to Data Engineer and Data Pipeline at Credit OKIntroduction to Data Engineer and Data Pipeline at Credit OK
Introduction to Data Engineer and Data Pipeline at Credit OK
 
Tableau Prep.pptx
Tableau Prep.pptxTableau Prep.pptx
Tableau Prep.pptx
 
ETL Process & Data Warehouse Fundamentals
ETL Process & Data Warehouse FundamentalsETL Process & Data Warehouse Fundamentals
ETL Process & Data Warehouse Fundamentals
 
Datastage ppt
Datastage pptDatastage ppt
Datastage ppt
 
Cloud-native Semantic Layer on Data Lake
Cloud-native Semantic Layer on Data LakeCloud-native Semantic Layer on Data Lake
Cloud-native Semantic Layer on Data Lake
 
Oracle database introduction
Oracle database introductionOracle database introduction
Oracle database introduction
 
Microsoft Azure Data Factory Data Flow Scenarios
Microsoft Azure Data Factory Data Flow ScenariosMicrosoft Azure Data Factory Data Flow Scenarios
Microsoft Azure Data Factory Data Flow Scenarios
 
Etl process in data warehouse
Etl process in data warehouseEtl process in data warehouse
Etl process in data warehouse
 

Similar a ETL Process

Lezlee Coulter SQl Server Portfolio
Lezlee Coulter SQl Server PortfolioLezlee Coulter SQl Server Portfolio
Lezlee Coulter SQl Server Portfoliolacndar1
 
Best-Fit-Engineering Deployments of Logical Data Warehouses
Best-Fit-Engineering Deployments of Logical Data WarehousesBest-Fit-Engineering Deployments of Logical Data Warehouses
Best-Fit-Engineering Deployments of Logical Data WarehousesDenodo
 
AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019Intel® Software
 
Keynote Presentation
Keynote PresentationKeynote Presentation
Keynote PresentationSplunk
 
Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Cana Ko
 
Introduction to Data Vault Modeling
Introduction to Data Vault ModelingIntroduction to Data Vault Modeling
Introduction to Data Vault ModelingKent Graziano
 
Big Data's Impact on the Enterprise
Big Data's Impact on the EnterpriseBig Data's Impact on the Enterprise
Big Data's Impact on the EnterpriseCaserta
 
Datawarehousing & DSS
Datawarehousing & DSSDatawarehousing & DSS
Datawarehousing & DSSDeepali Raut
 
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...Cloudera, Inc.
 
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, SisenseDatabase Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense✔ Eric David Benari, PMP
 
Summit 2011 ods edw technical
Summit 2011 ods edw technicalSummit 2011 ods edw technical
Summit 2011 ods edw technicalGreg Turmel
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Cathrine Wilhelmsen
 
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Hortonworks
 
Fulfilling Real-Time Analytics on Oracle BI Applications Platform
Fulfilling Real-Time Analytics on Oracle BI Applications PlatformFulfilling Real-Time Analytics on Oracle BI Applications Platform
Fulfilling Real-Time Analytics on Oracle BI Applications PlatformPerficient, Inc.
 
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...Kenneth de Brucq
 

Similar a ETL Process (20)

Lezlee Coulter SQl Server Portfolio
Lezlee Coulter SQl Server PortfolioLezlee Coulter SQl Server Portfolio
Lezlee Coulter SQl Server Portfolio
 
Best-Fit-Engineering Deployments of Logical Data Warehouses
Best-Fit-Engineering Deployments of Logical Data WarehousesBest-Fit-Engineering Deployments of Logical Data Warehouses
Best-Fit-Engineering Deployments of Logical Data Warehouses
 
Kaizentric Presentation
Kaizentric PresentationKaizentric Presentation
Kaizentric Presentation
 
AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019AIDC NY: BODO AI Presentation - 09.19.2019
AIDC NY: BODO AI Presentation - 09.19.2019
 
Keynote Presentation
Keynote PresentationKeynote Presentation
Keynote Presentation
 
Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831Talk IT_ Oracle_김태완_110831
Talk IT_ Oracle_김태완_110831
 
Introduction to Data Vault Modeling
Introduction to Data Vault ModelingIntroduction to Data Vault Modeling
Introduction to Data Vault Modeling
 
Big Data's Impact on the Enterprise
Big Data's Impact on the EnterpriseBig Data's Impact on the Enterprise
Big Data's Impact on the Enterprise
 
Lançamento ERwin 08/02
Lançamento ERwin 08/02Lançamento ERwin 08/02
Lançamento ERwin 08/02
 
Datawarehousing & DSS
Datawarehousing & DSSDatawarehousing & DSS
Datawarehousing & DSS
 
Msbi
MsbiMsbi
Msbi
 
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...
Limitless Data, Rapid Discovery, Powerful Insight: How to Connect Cloudera to...
 
Data Warehousing
Data WarehousingData Warehousing
Data Warehousing
 
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, SisenseDatabase Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense
Database Camp 2016 @ United Nations, NYC - Amir Orad, CEO, Sisense
 
Summit 2011 ods edw technical
Summit 2011 ods edw technicalSummit 2011 ods edw technical
Summit 2011 ods edw technical
 
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
Getting Started: Data Factory in Microsoft Fabric (Microsoft Fabric Community...
 
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
 
ITReady DW Day2
ITReady DW Day2ITReady DW Day2
ITReady DW Day2
 
Fulfilling Real-Time Analytics on Oracle BI Applications Platform
Fulfilling Real-Time Analytics on Oracle BI Applications PlatformFulfilling Real-Time Analytics on Oracle BI Applications Platform
Fulfilling Real-Time Analytics on Oracle BI Applications Platform
 
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...
Dell Solutions Tour 2015 - Dell Blueprints – referansearkitekturer eller nøkk...
 

Último

Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity PlanDatabarracks
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfMounikaPolabathina
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rick Flair
 
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Manik S Magar
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 

Último (20)

Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
How to write a Business Continuity Plan
How to write a Business Continuity PlanHow to write a Business Continuity Plan
How to write a Business Continuity Plan
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
What is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdfWhat is DBT - The Ultimate Data Build Tool.pdf
What is DBT - The Ultimate Data Build Tool.pdf
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...Rise of the Machines: Known As Drones...
Rise of the Machines: Known As Drones...
 
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate AgentsRyan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
Ryan Mahoney - Will Artificial Intelligence Replace Real Estate Agents
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!Anypoint Exchange: It’s Not Just a Repo!
Anypoint Exchange: It’s Not Just a Repo!
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 

ETL Process

  • 1. What is ETL? Extraction, Transformation, Loading Simple Example of ETL Customer Customer ID Name 105 Sainsbury Master Data 102 Tesco 109 Waitrose 101 Asda By Karthikeyan Selvaraj
  • 2. Let’s say the master data table here is a flat file ie excel file which is in your computer . We need to bring this table into SAP BI platform Customer Customer SAP BI Platform ID Name 105 Sainsbury 102 Tesco 109 Waitrose 101 Asda By Karthikeyan Selvaraj
  • 3. The first step is to extract the master data table ie excel file into BI-data warehouse The components needed for extracting the data into BI data warehouse are 1. DataSource 2. InfoPackage 1. DataSource DataSource DataSource: It defines about the data. For eg: Once I finish this presentation, I What type of will choose a location to save this ppt data? and I also define in what version I want Where the to save this ppt similarly, In datasource data is we will define about the data. located? By Karthikeyan Selvaraj
  • 4. The first step is to extract the master data table ie excel file into BI-data warehouse The components needed for extracting the data into BI data warehouse are 1. DataSource 2. InfoPackage 2. InfoPackage What is InfoPackage? In simple words we can define InfoPackage, It is like a key to open and enter into a room. It helps to bring the data from a legacy system or SAP system. For our scenario it helps to bring the data from our computer into BI datawarehouse. Customer Customer DataSource Excel ID Name File 105 Sainsbury What type of 102 Tesco data? 109 Waitrose Where the 101 Asda data is located? InfoPackage By Computer BI Datawarehouse Karthikeyan Selvaraj
  • 5. Now we have moved the master data table into BI datawarehouse by executing the InfoPackage Once the data comes into BI, It is stored in a table called PSA (Persistent Staging Area) The data that comes inside from any source system will be stored temporarily in PSA. Excel File Customer Customer DataSource PSA ID Name 105 Sainsbury Customer Customer What type of ID Name 102 Tesco data? 105 Sainsbury 109 Waitrose Where the data is 102 Tesco 101 Asda located? 109 Waitrose InfoPackage 101 Asda BI Datawarehouse Computer By Karthikeyan Selvaraj
  • 6. Transformation of Data The first part of ETL ie Extraction is done successfully. Now we need to transform the data so that it can be made more optimized for reporting. In order to do that, we define fields of the table as Info Objects. In our master data table we have two fields ie Customer ID and Customer Name so in BI we define them as Info Objects. Info Objects are divided into three types 1. Characteristics – sorting keys such as company code, product ID, etc. 2. Key Figures – quantity, amount or number of items. Data that can be manipulated. 3. Units – currency, measure this all comes under unit. Customer ID and Customer name are characteristic Info Objects. PSA Customer Customer Customer ID ID Name Info Object 105 Sainsbury Customer Name Info Object 102 Tesco 109 Waitrose 101 Asda By Characteristic Info Object Karthikeyan Selvaraj
  • 7. Transformation of Data The attribute for Customer ID is Customer name In database we define the attributes for primary key similarly we need to define the attributes for master data field ie for Customer ID. Once that is done we do the mapping ie transformation. We map the fields of the DataSource to the fields of the Info Objects InfoProvider DataSource Customer ID Customer ID Info Object Transformation Customer Customer Name Name Info Object By Karthikeyan Selvaraj
  • 8. Loading Once the mapping is done, data has to be transferred from DataSource (PSA Table) to InfoProvider ( Info Objects) This is done by a process called Data Transfer Process (DTP). How?: We create the DTP in InfoProvider layer and activate it. After activation we execute the DTP (Data Transfer Process). Now the Data from the PSA Table are transferred to their respective InfoObjects. InfoProvider DataSource Customer ID Customer ID Info Object Transformation Customer Customer Name Name Info Object DTP By Karthikeyan Selvaraj
  • 9. Loading Data are moved to their respective InfoObjects as per their mapping and it’s ready for reporting from the InfoProvider Layer. InfoProvider Customer ID Customer Name Info Object Info Object 105 Sainsbury 102 Tesco 109 Waitrose 101 Asda By Karthikeyan Selvaraj
  • 10. Thank You By Karthikeyan Selvaraj