1. Data Warehousing
&
Data Mining
By Mandar Kulkarni
PRN 10030141129
MBA-IT
SICSR

2. Contents
• Data warehousing
• Understanding data warehousing
• Data warehouse architecture
• Data Mining
• Data mining techniques

3. Warehouse?
Real time example?

4. Data Warehousing

5. Samsung
Mumbai
Delhi
Sales per item type per branch Sales
for first quarter. Manager
Chennai
Banglore

6. • Now, the sales manager wants to know the
sales of first quarter.?
• Solution
– Extract information from each database store it at
a single place, and process using operational
systems.!

7. Solution
Mumbai
Report
Delhi
Query & Sales
Data Analysis tools Manager
Warehouse
Chennai
Banglore

8. Operational Systems
• Running the business real time
• Routine tasks
• Decision Support Systems(DSS)
– Help in taking actions!
• Used by people who deal with customers,
products
• They are increasingly used by customers

9. Data Warehouse
• A single, complete and consistent store of
data obtained from a variety of different
sources made available to end users in a what
they can understand and use in a business
context.
• A process of transforming data into
information and making it available to users in
a timely enough manner to make a difference

10. Definition
• Integrated, Subject-Oriented, Time-Variant,
Nonvolatile database that provides support
for decision making

11. Data warehouse architecture

12. Source
Data Information
Management & Control Delivery
External
Metadata
Production
MDDB
Data Warehouse
DBMS
Internal
Report /
Query
Archived Data Marts
Data Staging
Data
Mining

13. Components
• Source Data
• Data Staging (Data Extraction, cleaning And Loading )
– Talend is the first open source ETL tool
• Data Storage
• Information Delivery (EIS)
• Management and control

14. OLAP
• Online Analytical Processing Tools
• DSS tools that use multidimensional data
analysis techniques
– Support for a DSS data store
– Data extraction and integration filter
– Specialized presentation interface
• Oracle OLAP 11G

15. Multidimensional analysis

16. OLAP architecture

17. 12 Rules of Data Warehouse
1. Data Warehouse and Operational
Environments are Separated
2. Data is integrated
3. Contains historical data over a long period of
time
4. Data is a snapshot data captured at a given
point in time
5. Data is subject-oriented

18. 6.Mainly read-only with periodic batch updates
7.Development Life Cycle has a data driven
approach versus the traditional process-driven
approach
8.Data contains several levels of detail
-Current, Old, Lightly Summarized, Highly
Summarized

19. 9.Environment is characterized by Read-only
transactions to very large data sets
10.System that traces data sources, transformations,
and storage
11.Metadata is a critical component
– Source, transformation, integration, storage, relationships,
history, etc
12.Contains a chargeback mechanism for resource
usage that enforces optimal use of data by end users

20. OLTP v/s Data warehousing
OLTP Data Warehousing
• Application Oriented • Subject Oriented
• Used to Run Business • Used to analyze business
• Detailed data • Summarized and refined
• Current up-to date • Snapshot Data
• Isolated data • Integrated Data
• Ad-Hoc Access
• Repetitive Access
• Performance relaxed
• Performance Sensitive
• Large volume accessed at a
• Few records accessed time
• Read/Update Access • Mostly Read

21. Data Warehouse summary
• Integrated platform for OLAP and DSS
• Helps optimize business operations
• Easy access to multidimensional data

22. Data Mining

23. Why Data Mining?
Wealth generation
Analyzing trends
Strategic decision making
Security

24. Data Mining
• Look for hidden patterns and trends in data
that is not immediately apparent from
summarizing the data
• No Query…
• …But an “Interestingness criteria”

25. Data Mining
+ =
Interestingness Hidden
Data criteria patterns

26. Data Mining Type
of
Patterns
+ =
Interestingness Hidden
Data criteria patterns

27. Data Mining
Type of data Type of
Interestingness criteria
+ =
Interestingness Hidden
Data criteria patterns

28. Type of Data
• Tabular (Ex: Transaction data)
– Relational
– Multi-dimensional
• Tree (Ex: XML data)
• Graphs
• Sequence (Ex: DNA, activity logs)
• Text, Multimedia …

29. Type of Interestingness
• Frequency
• Rarity
• Correlation
• Length of occurrence (for sequence and temporal data)
• Consistency
• Repeating / periodicity
• “Abnormal” behavior
• Other patterns of interestingness…

30. Data Mining vs Statistical Inference
Statistics:
Statistical
Conceptual Reasoning
Model
(Hypothesis)
“Proof”
(Validation of Hypothesis)

31. Data Mining vs Statistical Inference
Data mining:
Mining
Algorithm
Based on
Data Interestingness
Pattern
(model, rule,
hypothesis)
discovery

32. Used for..
• Data mining is used for
– Frequent Item-sets
– Associations
– Classifications
– Clustering

33. Techniques
• Algorithms
– Apriori algorithm
– Decision tree
• SLIQ
– Supervised Learning in QUEST
– IBM
• “GROUP BY”
mysql> select sum(sal),deptno from emp group by deptno;

34. Data Mining Summary
• Helps in pattern analysis and thus taking
actions –real time and future based.
• Analyzing trends and clusters in business
operations.

35. References
• http://www.datawarehousing.com/
• http://www.dw-institute.com/
• http://www.almaden.ibm.com/cs/quest/index.html

36. Thank you
Any Questions?