Machine Learning

Anotações:

• Training data includes the desired solution(labels)

1. Common Tasks
   1. Classification
   2. Prediction
2. Common Algorithms
   1. k-Nearest Neighbors
   2. Linear Regression
   3. Logistic Regression
   4. Support Vector Machines(SVMs)
   5. Decision Tree
   6. Random Forests
   7. Neural Networks

Anotações:

• Tries to learn without a labeled data

1. Common Tasks
   1. Clustering
      1. K-Means
      2. Hierarchical Cluster Anaysis(HCA)
      3. Expectation Maximization
   2. Visualization and Dimensionality Reduction
      1. Principal Component Analysis(PCA)
      2. Kernel PCA
      3. Locally-Linear Embedding(LLE)
      4. t-Distributed Stochastic Neighbor Embedding(t-SNE)
   3. Association Rule Learning
      1. Apriori
      2. Eclat
   4. Anomaly Dectection

Anotações:

• Some labeled data and lots of unlabelled data

Anotações:

• Use reinforcements and penalties to train the data. Often used to make robots learn how to walk. 

Anotações:

• AKA offline, the system is trained and then is used.

Anotações:

• Learns while it runs. It can be pre trained and then getting updated while it runs. It is also called incremental learning because it can be trained offline but in an incremental fashion.

1. Out-of-core

   Anotações:

   • Used to train huge datasets. It chops the data in smaller batches and test it untill it's acceptable.

Anotações:

• Train by heart. Use data create similarity data to find similarities.

Anotações:

• Create a model from the examples and use that model to make predictions.

1. Test Functions
   1. Utility Function
   2. Cost Function

1. Insufficient Training Data
2. Non Representative Training Data

   Anotações:

   • The data must represent what you want to generalize.
   1. Sampling Noise

      Anotações:

      • Sample too small. (Non representative data as a result of chance)
   2. Sampling Bias

      Anotações:

      • Large number of data but sampling is flawed.
      1. Nonresponsive Bias
3. Poor Quality Data

   Anotações:

   • Data that is full of errors, outliers, and noise(e.g., due to poor quality measurements)
   1. Celan up the Outliers
   2. Treat the Instances that are Missing Features
4. Irrelevant Features

   Anotações:

   • The proccess of feature engineering is used to prevent this problem.
   1. Feature Engineering
      1. Feature Selection
      2. Features Extraction

         Anotações:

         • Combine existing features to produce a more useful one. See Dimensionality and reduction algorithms.
      3. New Features
5. Overfitting the Training Data
   1. Overgeneralizing
   2. Noise Attributes

      Anotações:

      • Introducing uninformative attributes can cause noise and to include unwanted patterns.

Anotações:

• Performs well on training data end poorly on test data.

1. Possible Solutions
   1. Simplify the Model

      Anotações:

      • Use a model with fewer parameters, reducing the number of attributes
      1. Regularization

         Anotações:

         • Constrain the model to make it simpler.
   2. Gather More Training Data
   3. Reduce noise in Training Data

Anotações:

• Performs bad in training and test data.

1. Reasons
   1. Model is Too Simple
2. Possible Solutions
   1. More Powerful Model with More Parameters
   2. Feeding better Features
   3. Reducing Regularization

1. Test on Production

   Anotações:

   • Not a good option.
2. Use Test Set

   Anotações:

   • A common rule is 80% for training data and 20% for test data.
   1. Generalization Error

      Anotações:

      • Error rate on new cases. Training error is low and Generalization error is high you have a overfitting problem.
3. Validation Test Set

   Anotações:

   • Used to tune the Hyperparameters and lastly test in the test set.
4. Cross Validation

   Anotações:

   • Split the training set into complementary subsets, and each model is trained against a different combination of these subsets and validated against the remaining parts. Once the model type and hyperparameters have been selected, a final model training is made using the full training set, and the generalization error is measured on the test set. Used to avoid "wasting" too many training data into validation tests sets.

Anotações:

• Is a parameter whose value is set before the learning process begins.

1. Regularization

Anotações:

• Parameters derived via training. Parameters related to the model.e.g., the parameters Theta0 and Theta1 of a linear model.