Data Preprocessing

Introduction to Data Mining Ch. 2 Data Preprocessing Heon Gyu Lee ( [email_address] ) http://dblab.chungbuk.ac.kr/~hglee DB/Bioinfo., Lab. http://dblab.chungbuk.ac.kr Chungbuk National University

Why Data Preprocessing? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

What is Data? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Attributes Objects

Types of Attributes ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Discrete and Continuous Attributes ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Quality ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Noise ,[object Object],[object Object],Two Sine Waves Two Sine Waves + Noise

Missing Values ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Duplicate Data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Major Tasks in Data Preprocessing ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Data Cleaning

Data Cleaning : How to Handle Missing Data? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Cleaning : How to Handle Noisy Data? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Cleaning : Binning Methods ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Cleaning : Regression x y y = x + 1 X1 Y1 Y1’

Data Cleaning : Cluster Analysis

Data Integration ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Integration : Handling Redundancy in Data Integration ,[object Object],[object Object],[object Object],[object Object],[object Object]

Data Integration : Correlation Analysis (Numerical Data) ,[object Object],[object Object],[object Object],[object Object]

Data Integration : Correlation Analysis (Categorical Data) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Chi-Square Calculation: An Example ,[object Object],[object Object],1500 1200 300 Sum(col.) 1050 1000(840) 50(210) Not like science fiction 450 200(360) 250(90) Like science fiction Sum (row) Not play chess Play chess

Data Transformation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Transformation : Normalization ,[object Object],[object Object],[object Object],[object Object],[object Object],Where j is the smallest integer such that Max(| ν ’ |) < 1

Data Reduction Strategies ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Reduction : Aggregation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Reduction : Aggregation Standard Deviation of Average Monthly Precipitation Standard Deviation of Average Yearly Precipitation Variation of Precipitation in Australia

Data Reduction : Sampling ,[object Object],[object Object],[object Object],[object Object]

Data Reduction : Types of Sampling ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Reduction : Dimensionality Reduction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Dimensionality Reduction : PCA ,[object Object],x 2 x 1 e

Dimensionality Reduction : PCA ,[object Object],[object Object],x 2 x 1 e

Data Reduction : Feature Subset Selection ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Reduction : Feature Creation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Data Reduction : Mapping Data to a New Space Two Sine Waves Two Sine Waves + Noise Frequency ,[object Object],[object Object]

Data Reduction : Discretization Using Class Labels ,[object Object],3 categories for both x and y 5 categories for both x and y

Data Reduction : Discretization Without Using Class Labels Data Equal interval width Equal frequency K-means

Data Reduction : Attribute Transformation ,[object Object],[object Object],[object Object]

Data Preprocessing

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Data Preprocessing

Similar a Data Preprocessing (20)

Más de Object-Frontier Software Pvt. Ltd

Más de Object-Frontier Software Pvt. Ltd (9)

Último

Último (20)

Data Preprocessing