DATA PREPROCESSING

1. Missing Values

   Anotações:

   • Use the attributes mean to fill in the missing value
   • Use the attribute mean for all sample belonging to the same class as the given tupe
   • Use the most probable values to fill in the missing values
   1. Data Cleaning

      Anotações:

      • Fill in missing values & correct inconsistencies in the data
   2. Ignore the tuple

      Anotações:

      • Class label is missing & not effective
   3. Fill in the missing value manuvally

      Anotações:

      • time consuming & may not be feasible
   4. Use global constant to fill in the missing value

      Anotações:

      • replace values & same constant
2. Noisy Data
   1. Binning

      Anotações:

      • It smooth sorted data value by consulting its neighborhood
      • smoothing by bin means smoothing by bin medians smoothing by bin boundaries
   2. Clustering

      Anotações:

      • Similar values organized into groups & outliers may be detected by clustering
   3. Combained computer & human Inspection

      Anotações:

      • Outlier may be identified through a combination of computer and human inspection
   4. Regression

      Anotações:

      • Data can be smoothed by fitting the data to function
      • Linear regression
      • Multiple linear regression
3. Inconsistent Data

   Anotações:

   • Data inconsistencies may be corrected manually using external reference
   • Knowledge engineering tools may also be used to detect the violation of  known data constrains

Anotações:

• It can be applied to obtain a reduced representation of the data set yet closely maintains the integrity of the original data

1. Dimensionality Reduction
2. Data Compression
   1. wavelet transforms
   2. Principal components analysis
3. Numerosity Reduction
4. Data cube Aggregation
5. Strategies
   1. Data cube aggregation
   2. Dimension reduction

      Anotações:

      • step wise forward selection step wise backward elimination combination of  forward selection & backward elimination decision tree induction
   3. Numerosity reduction
      1. Histograms
      2. Clustering
      3. Sampling

         Anotações:

         • SRSWOR of size n SRSWR of size n Cluster sample Stratisfied sample
   4. Discretization & hierarchy Generation
      1. For numeric data
         1. Bining
         2. Histogram & Analysis
         3. Cluster Analysis
         4. Entropy based Discretization
         5. Segmantation by Natural Partitioning
      2. For categorical data
         1. Portion of a hierarchy by explicit data grouping
         2. Partial ordering of attributes explicity at the schema level
         3. Set of attributes,but not their partial orderies

1. For Categorical Data
2. For Numeric Data

1. Data Integration

   Anotações:

   • It can help improve accuracy & speed of the subsequent mining process
   • Reduce and avoid Redundancies & inconsistencies
   • Detection and resolution of data value conflicts
2. Data Transformation
   1. Smoothing

      Anotações:

      • Remove the noise from data
   2. Attribute construction

      Anotações:

      • new attributes are constructed and added to help the mining process
   3. Aggregation

      Anotações:

      • Aggregation operation are applied to the data
   4. Generlization

      Anotações:

      • Primitive data are replaced by high-level concept through the use of concept hierarchies
   5. Normalization

      Anotações:

      • Attribute data are scaled with in small specified range such as  -1.0 to 1.0 or 0.0 to 1.0

1. Ease of Mining Process
2. To Improve the Quality of Data