SlideShare una empresa de Scribd logo
1 de 17
Descargar para leer sin conexión
By: RAVI RANJAN




                  DATA
              WAREHOUSE
                  By: Ravi Ranjan
DEFINITION
 Data Warehouse
 A collection of corporate
 information, derived directly
 from operational systems
 and some external data
 sources. Its specific purpose
 is to support business
 decisions, not business
 operations.
THE PURPOSE OF DATA WAREHOUSING

     Realize    the value of data
          Data / information is an asset
          Methods to realize the
          value, (Reporting, Analysis, etc.)


     Make    better decisions
         Turn data into information
         Create competitive advantage
         Methods to support the decision making
          process, (EIS, DSS, etc.)
Data Warehouse Components

• Staging Area
      • A preparatory repository where transaction data
        can be transformed for use in the data warehouse
• Data Mart
      • Traditional dimensionally modeled set of dimension
        and fact tables
      • Per Kimball, a data warehouse is the union of a set
        of data marts
• Operational Data Store (ODS)
      • Modeled to support near real-time reporting needs.
DATA WAREHOUSE FUNCTIONALITY


Relational
Databases
                            Optimized Loader
               Extraction
ERP
Systems        Cleansing
                            Data Warehouse
                            Engine         Analyze
Purchased                                      Query
Data



Legacy
Data            Metadata Repository
EVOLUTION ARCHITECTURE OF DATA WAREHOUSE


                                      GO TO
 Top-Down Architecture               DIAGRAM

                                      GO TO
 Bottom-Up Architecture              DIAGRAM

                                      GO TO
 Enterprise Data Mart Architecture   DIAGRAM

                                      GO TO
 Data Stage/Data Mart Architecture   DIAGRAM
VERY LARGE DATA BASES

  WAREHOUSES ARE VERY LARGE DATABASES

 Terabytes   -- 10^12 bytes: Wal-Mart -- 24 Terabytes

 Petabytes -- 10^15 bytes: Geographic Information
                             Systems
 Exabytes -- 10^18 bytes:  National Medical Records

 Zettabytes   -- 10^21 bytes: Weather images

 Zottabytes   -- 10^24 bytes: Intelligence Agency Videos
COMPLEXITIES OF CREATING A DATA WAREHOUSE

     Incomplete errors
        Missing Fields
        Records or Fields That, by Design, are not
         Being Recorded

     Incorrecterrors
        Wrong Calculations, Aggregations
        Duplicate Records
        Wrong Information Entered into Source
         System
SUCCESS & FUTURE OF DATA WAREHOUSE

 The    Data Warehouse has successfully supported the
    increased needs of the State over the past eight years.
   The need for growth continues however, as the desire for
    more integrated data increases.
 The   Data Warehouse has software and tools in place to
    provide the functionality needed to support new
    enterprise Data Warehouse projects.
 The   future capabilities of the Data Warehouse can be
    expanded to include other programs and agencies.
DATA WAREHOUSE PITFALLS


 You are going to spend much time
 extracting, cleaning, and loading data
 Youare going to find problems with systems feeding the
 data warehouse
 Youwill find the need to store/validate data not being
 captured/validated by any existing system
 Large scale data warehousing can become an exercise
 in data homogenizing
DATA WAREHOUSE PITFALLS…

 The  time it takes to load the warehouse will expand
  to the amount of the time in the available window...
  and then some
 You are building a HIGH maintenance system

 You will fail if you concentrate on resource
  optimization to the neglect of project, data, and
  customer management issues and an understanding
  of what adds value to the customer
BEST PRACTICES


 Complete     requirements and design

 Prototyping    is key to business understanding

 Utilizing   proper aggregations and detailed data

 Training    is an on-going process

 Build   data integrity checks into your system.
Ppt
Top-Down Architecture




                      BACK TO
                    ARCHITECTURE
Bottom-Up Architecture




                           BACK TO
                         ARCHITECTURE
Enterprise Data Mart Architecture




                                 BACK TO
                               ARCHITECTURE
Data Stage/Data Mart Architecture




                                BACK TO
                              ARCHITECTURE

Más contenido relacionado

La actualidad más candente

Introduction to Data Warehouse
Introduction to Data WarehouseIntroduction to Data Warehouse
Introduction to Data WarehouseSOMASUNDARAM T
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architecturepcherukumalla
 
OLAP OnLine Analytical Processing
OLAP OnLine Analytical ProcessingOLAP OnLine Analytical Processing
OLAP OnLine Analytical ProcessingWalid Elbadawy
 
Data warehouse architecture
Data warehouse architecture Data warehouse architecture
Data warehouse architecture janani thirupathi
 
DATA WAREHOUSING
DATA WAREHOUSINGDATA WAREHOUSING
DATA WAREHOUSINGKing Julian
 
Dimensional Modeling
Dimensional ModelingDimensional Modeling
Dimensional ModelingSunita Sahu
 
Data warehousing - Dr. Radhika Kotecha
Data warehousing - Dr. Radhika KotechaData warehousing - Dr. Radhika Kotecha
Data warehousing - Dr. Radhika KotechaRadhika Kotecha
 
Dw & etl concepts
Dw & etl conceptsDw & etl concepts
Dw & etl conceptsjeshocarme
 
Date warehousing concepts
Date warehousing conceptsDate warehousing concepts
Date warehousing conceptspcherukumalla
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architectureuncleRhyme
 
Etl - Extract Transform Load
Etl - Extract Transform LoadEtl - Extract Transform Load
Etl - Extract Transform LoadABDUL KHALIQ
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Miningidnats
 

La actualidad más candente (20)

Introduction to Data Warehouse
Introduction to Data WarehouseIntroduction to Data Warehouse
Introduction to Data Warehouse
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architecture
 
OLAP OnLine Analytical Processing
OLAP OnLine Analytical ProcessingOLAP OnLine Analytical Processing
OLAP OnLine Analytical Processing
 
DATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MININGDATA WAREHOUSING AND DATA MINING
DATA WAREHOUSING AND DATA MINING
 
Data warehouse architecture
Data warehouse architecture Data warehouse architecture
Data warehouse architecture
 
Data warehouse
Data warehouse Data warehouse
Data warehouse
 
DATA WAREHOUSING
DATA WAREHOUSINGDATA WAREHOUSING
DATA WAREHOUSING
 
Data preprocessing
Data preprocessingData preprocessing
Data preprocessing
 
Introduction to ETL and Data Integration
Introduction to ETL and Data IntegrationIntroduction to ETL and Data Integration
Introduction to ETL and Data Integration
 
Dimensional Modeling
Dimensional ModelingDimensional Modeling
Dimensional Modeling
 
Oltp vs olap
Oltp vs olapOltp vs olap
Oltp vs olap
 
Data warehouse
Data warehouseData warehouse
Data warehouse
 
Data warehousing
Data warehousingData warehousing
Data warehousing
 
Data warehouse
Data warehouseData warehouse
Data warehouse
 
Data warehousing - Dr. Radhika Kotecha
Data warehousing - Dr. Radhika KotechaData warehousing - Dr. Radhika Kotecha
Data warehousing - Dr. Radhika Kotecha
 
Dw & etl concepts
Dw & etl conceptsDw & etl concepts
Dw & etl concepts
 
Date warehousing concepts
Date warehousing conceptsDate warehousing concepts
Date warehousing concepts
 
Data warehouse architecture
Data warehouse architectureData warehouse architecture
Data warehouse architecture
 
Etl - Extract Transform Load
Etl - Extract Transform LoadEtl - Extract Transform Load
Etl - Extract Transform Load
 
Data Warehousing and Data Mining
Data Warehousing and Data MiningData Warehousing and Data Mining
Data Warehousing and Data Mining
 

Similar a Ppt

Data warehousing
Data warehousingData warehousing
Data warehousingVarun Jain
 
Oracle: Fundamental Of Dw
Oracle: Fundamental Of DwOracle: Fundamental Of Dw
Oracle: Fundamental Of Dworacle content
 
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAINING
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAININGDATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAINING
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAININGDatawarehouse Trainings
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?RTTS
 
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Hortonworks
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureJames Serra
 
Introducing the Big Data Ecosystem with Caserta Concepts & Talend
Introducing the Big Data Ecosystem with Caserta Concepts & TalendIntroducing the Big Data Ecosystem with Caserta Concepts & Talend
Introducing the Big Data Ecosystem with Caserta Concepts & TalendCaserta
 
professional informatica trainer
professional informatica trainerprofessional informatica trainer
professional informatica trainervibrantuser
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World DistilledRTTS
 
Dw Concepts
Dw ConceptsDw Concepts
Dw Conceptsdataware
 
Anexinet Big Data Solutions
Anexinet Big Data SolutionsAnexinet Big Data Solutions
Anexinet Big Data SolutionsMark Kromer
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
 
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data Virtuality
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data VirtualityBeyond the Data Lake - Matthias Korn, Technical Consultant at Data Virtuality
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data VirtualityDataconomy Media
 
Take Action: The New Reality of Data-Driven Business
Take Action: The New Reality of Data-Driven BusinessTake Action: The New Reality of Data-Driven Business
Take Action: The New Reality of Data-Driven BusinessInside Analysis
 
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS Amazon Web Services LATAM
 

Similar a Ppt (20)

Data warehousing
Data warehousingData warehousing
Data warehousing
 
Oracle: Fundamental Of DW
Oracle: Fundamental Of DWOracle: Fundamental Of DW
Oracle: Fundamental Of DW
 
Oracle: Fundamental Of Dw
Oracle: Fundamental Of DwOracle: Fundamental Of Dw
Oracle: Fundamental Of Dw
 
DWBASIC.ppt
DWBASIC.pptDWBASIC.ppt
DWBASIC.ppt
 
Dwh basics datastage online training
Dwh basics datastage online trainingDwh basics datastage online training
Dwh basics datastage online training
 
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAINING
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAININGDATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAINING
DATASTAGE AND QUALITY STAGE 9.1 ONLINE TRAINING
 
What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?What is a Data Warehouse and How Do I Test It?
What is a Data Warehouse and How Do I Test It?
 
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
Webinar - Accelerating Hadoop Success with Rapid Data Integration for the Mod...
 
Building an Effective Data Warehouse Architecture
Building an Effective Data Warehouse ArchitectureBuilding an Effective Data Warehouse Architecture
Building an Effective Data Warehouse Architecture
 
Introducing the Big Data Ecosystem with Caserta Concepts & Talend
Introducing the Big Data Ecosystem with Caserta Concepts & TalendIntroducing the Big Data Ecosystem with Caserta Concepts & Talend
Introducing the Big Data Ecosystem with Caserta Concepts & Talend
 
professional informatica trainer
professional informatica trainerprofessional informatica trainer
professional informatica trainer
 
the Data World Distilled
the Data World Distilledthe Data World Distilled
the Data World Distilled
 
Dw Concepts
Dw ConceptsDw Concepts
Dw Concepts
 
Anexinet Big Data Solutions
Anexinet Big Data SolutionsAnexinet Big Data Solutions
Anexinet Big Data Solutions
 
Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)Data Lakehouse, Data Mesh, and Data Fabric (r1)
Data Lakehouse, Data Mesh, and Data Fabric (r1)
 
DW 101
DW 101DW 101
DW 101
 
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data Virtuality
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data VirtualityBeyond the Data Lake - Matthias Korn, Technical Consultant at Data Virtuality
Beyond the Data Lake - Matthias Korn, Technical Consultant at Data Virtuality
 
The BI Sandbox
The BI SandboxThe BI Sandbox
The BI Sandbox
 
Take Action: The New Reality of Data-Driven Business
Take Action: The New Reality of Data-Driven BusinessTake Action: The New Reality of Data-Driven Business
Take Action: The New Reality of Data-Driven Business
 
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS
Innovation Track AWS Cloud Experience Argentina - Data Lakes & Analytics en AWS
 

Último

Young adult book quiz by SJU quizzers.ppt
Young adult book quiz by SJU quizzers.pptYoung adult book quiz by SJU quizzers.ppt
Young adult book quiz by SJU quizzers.pptSJU Quizzers
 
Inside Look: Brooke Monk's Exclusive OnlyFans Content Production
Inside Look: Brooke Monk's Exclusive OnlyFans Content ProductionInside Look: Brooke Monk's Exclusive OnlyFans Content Production
Inside Look: Brooke Monk's Exclusive OnlyFans Content Productionget joys
 
5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life
5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life
5 Moments of Everyday Self-Loathing That Perfectly Describe Your LifeSalty Vixen Stories & More
 
Carowinds 2024: Thrills, Spills & Surprises
Carowinds 2024: Thrills, Spills & SurprisesCarowinds 2024: Thrills, Spills & Surprises
Carowinds 2024: Thrills, Spills & Surprisescarawinds99
 
Holi:: "The Festival of Colors in India"
Holi:: "The Festival of Colors in India"Holi:: "The Festival of Colors in India"
Holi:: "The Festival of Colors in India"IdolsArts
 
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ..."Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...RAGHURAMYC
 
Taylor Swift quiz( with answers) by SJU quizzers
Taylor Swift quiz( with answers) by SJU quizzersTaylor Swift quiz( with answers) by SJU quizzers
Taylor Swift quiz( with answers) by SJU quizzersSJU Quizzers
 

Último (7)

Young adult book quiz by SJU quizzers.ppt
Young adult book quiz by SJU quizzers.pptYoung adult book quiz by SJU quizzers.ppt
Young adult book quiz by SJU quizzers.ppt
 
Inside Look: Brooke Monk's Exclusive OnlyFans Content Production
Inside Look: Brooke Monk's Exclusive OnlyFans Content ProductionInside Look: Brooke Monk's Exclusive OnlyFans Content Production
Inside Look: Brooke Monk's Exclusive OnlyFans Content Production
 
5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life
5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life
5 Moments of Everyday Self-Loathing That Perfectly Describe Your Life
 
Carowinds 2024: Thrills, Spills & Surprises
Carowinds 2024: Thrills, Spills & SurprisesCarowinds 2024: Thrills, Spills & Surprises
Carowinds 2024: Thrills, Spills & Surprises
 
Holi:: "The Festival of Colors in India"
Holi:: "The Festival of Colors in India"Holi:: "The Festival of Colors in India"
Holi:: "The Festival of Colors in India"
 
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ..."Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...
"Quest for Knowledge: An Exciting Journey Through 40 Brain-Bending Questions ...
 
Taylor Swift quiz( with answers) by SJU quizzers
Taylor Swift quiz( with answers) by SJU quizzersTaylor Swift quiz( with answers) by SJU quizzers
Taylor Swift quiz( with answers) by SJU quizzers
 

Ppt

  • 1. By: RAVI RANJAN DATA WAREHOUSE By: Ravi Ranjan
  • 2. DEFINITION Data Warehouse A collection of corporate information, derived directly from operational systems and some external data sources. Its specific purpose is to support business decisions, not business operations.
  • 3. THE PURPOSE OF DATA WAREHOUSING  Realize the value of data  Data / information is an asset  Methods to realize the value, (Reporting, Analysis, etc.)  Make better decisions  Turn data into information  Create competitive advantage  Methods to support the decision making process, (EIS, DSS, etc.)
  • 4. Data Warehouse Components • Staging Area • A preparatory repository where transaction data can be transformed for use in the data warehouse • Data Mart • Traditional dimensionally modeled set of dimension and fact tables • Per Kimball, a data warehouse is the union of a set of data marts • Operational Data Store (ODS) • Modeled to support near real-time reporting needs.
  • 5. DATA WAREHOUSE FUNCTIONALITY Relational Databases Optimized Loader Extraction ERP Systems Cleansing Data Warehouse Engine Analyze Purchased Query Data Legacy Data Metadata Repository
  • 6. EVOLUTION ARCHITECTURE OF DATA WAREHOUSE GO TO Top-Down Architecture DIAGRAM GO TO Bottom-Up Architecture DIAGRAM GO TO Enterprise Data Mart Architecture DIAGRAM GO TO Data Stage/Data Mart Architecture DIAGRAM
  • 7. VERY LARGE DATA BASES WAREHOUSES ARE VERY LARGE DATABASES  Terabytes -- 10^12 bytes: Wal-Mart -- 24 Terabytes  Petabytes -- 10^15 bytes: Geographic Information Systems  Exabytes -- 10^18 bytes: National Medical Records  Zettabytes -- 10^21 bytes: Weather images  Zottabytes -- 10^24 bytes: Intelligence Agency Videos
  • 8. COMPLEXITIES OF CREATING A DATA WAREHOUSE  Incomplete errors  Missing Fields  Records or Fields That, by Design, are not Being Recorded  Incorrecterrors  Wrong Calculations, Aggregations  Duplicate Records  Wrong Information Entered into Source System
  • 9. SUCCESS & FUTURE OF DATA WAREHOUSE  The Data Warehouse has successfully supported the increased needs of the State over the past eight years.  The need for growth continues however, as the desire for more integrated data increases.  The Data Warehouse has software and tools in place to provide the functionality needed to support new enterprise Data Warehouse projects.  The future capabilities of the Data Warehouse can be expanded to include other programs and agencies.
  • 10. DATA WAREHOUSE PITFALLS  You are going to spend much time extracting, cleaning, and loading data  Youare going to find problems with systems feeding the data warehouse  Youwill find the need to store/validate data not being captured/validated by any existing system  Large scale data warehousing can become an exercise in data homogenizing
  • 11. DATA WAREHOUSE PITFALLS…  The time it takes to load the warehouse will expand to the amount of the time in the available window... and then some  You are building a HIGH maintenance system  You will fail if you concentrate on resource optimization to the neglect of project, data, and customer management issues and an understanding of what adds value to the customer
  • 12. BEST PRACTICES  Complete requirements and design  Prototyping is key to business understanding  Utilizing proper aggregations and detailed data  Training is an on-going process  Build data integrity checks into your system.
  • 14. Top-Down Architecture BACK TO ARCHITECTURE
  • 15. Bottom-Up Architecture BACK TO ARCHITECTURE
  • 16. Enterprise Data Mart Architecture BACK TO ARCHITECTURE
  • 17. Data Stage/Data Mart Architecture BACK TO ARCHITECTURE

Notas del editor

  1. Legacy data is historical dataThe working information of a staff member Working hours or time-off hours within the fiscal period, up to the current dateWorking Hours = Overtime, etc.Time-Off Hours = Vacation, Sick Leave, etc.
  2. DataStage database, toolA tool set for designing, developing, and runnin.gapplications that populate one or more tables in a data warehouse