DATA PREPROCESSING

1. Missing Values

   Nota:

   • Use the attributes mean to fill in the missing value
   • Use the attribute mean for all sample belonging to the same class as the given tupe
   • Use the most probable values to fill in the missing values
   1. Data Cleaning

      Nota:

      • Fill in missing values & correct inconsistencies in the data
   2. Ignore the tuple

      Nota:

      • Class label is missing & not effective
   3. Fill in the missing value manuvally

      Nota:

      • time consuming & may not be feasible
   4. Use global constant to fill in the missing value

      Nota:

      • replace values & same constant
2. Noisy Data
   1. Binning

      Nota:

      • It smooth sorted data value by consulting its neighborhood
      • smoothing by bin means smoothing by bin medians smoothing by bin boundaries
   2. Clustering

      Nota:

      • Similar values organized into groups & outliers may be detected by clustering
   3. Combained computer & human Inspection

      Nota:

      • Outlier may be identified through a combination of computer and human inspection
   4. Regression

      Nota:

      • Data can be smoothed by fitting the data to function
      • Linear regression
      • Multiple linear regression
3. Inconsistent Data

   Nota:

   • Data inconsistencies may be corrected manually using external reference
   • Knowledge engineering tools may also be used to detect the violation of  known data constrains

Nota:

• It can be applied to obtain a reduced representation of the data set yet closely maintains the integrity of the original data

1. Dimensionality Reduction
2. Data Compression
   1. wavelet transforms
   2. Principal components analysis
3. Numerosity Reduction
4. Data cube Aggregation
5. Strategies
   1. Data cube aggregation
   2. Dimension reduction

      Nota:

      • step wise forward selection step wise backward elimination combination of  forward selection & backward elimination decision tree induction
   3. Numerosity reduction
      1. Histograms
      2. Clustering
      3. Sampling

         Nota:

         • SRSWOR of size n SRSWR of size n Cluster sample Stratisfied sample
   4. Discretization & hierarchy Generation
      1. For numeric data
         1. Bining
         2. Histogram & Analysis
         3. Cluster Analysis
         4. Entropy based Discretization
         5. Segmantation by Natural Partitioning
      2. For categorical data
         1. Portion of a hierarchy by explicit data grouping
         2. Partial ordering of attributes explicity at the schema level
         3. Set of attributes,but not their partial orderies

1. For Categorical Data
2. For Numeric Data

1. Data Integration

   Nota:

   • It can help improve accuracy & speed of the subsequent mining process
   • Reduce and avoid Redundancies & inconsistencies
   • Detection and resolution of data value conflicts
2. Data Transformation
   1. Smoothing

      Nota:

      • Remove the noise from data
   2. Attribute construction

      Nota:

      • new attributes are constructed and added to help the mining process
   3. Aggregation

      Nota:

      • Aggregation operation are applied to the data
   4. Generlization

      Nota:

      • Primitive data are replaced by high-level concept through the use of concept hierarchies
   5. Normalization

      Nota:

      • Attribute data are scaled with in small specified range such as  -1.0 to 1.0 or 0.0 to 1.0

1. Ease of Mining Process
2. To Improve the Quality of Data