Machine Learning

Annotations:

• Most common ML application

1. Types
   1. Regression

      Annotations:

      • Output y belongs to R, set of real numbers 
   2. Classification

      Annotations:

      • Output y belongs to a set of specific, possible outcomes e.g. {yes, no}, {0,1,2,....9}
2. Learning task

   Annotations:

   • Given value of an input x, make a good prediction of output y, denoted by y hat. x: scalar or vector x = (x1, x2, .... xp) 'p' features y: scalar or vector
   1. Supervised learning

      Annotations:

      • Given a training se of N data points, learn a prediction function f:x->y such that given a new x, f can accurately predict the corresponding y.
      1. Linear model
         1. Error function
         2. Hyper-parameters
            1. Lambda - regularization coeff
               1. Model selection
                  1. For different values of hyper-param (HP) - train the model - compute the perf in valid set
                  2. Pick val of HP that has best valid perf
                  3. Compute test perf for model with chosen value of HP
            2. M - deg of polynomial
         3. Overfitting
            1. Sol 3: Model selection for based on M
            2. Sol 2: Regularization
            3. Sol 1: Add more data points
            4. Checking for it: Use separate test set
         4. Classification
            1. M1: Linear model
               1. Closed form solution
            2. M2: k-Nearest Neighbour (k-NN)

               Annotations:

               • Average of classification values of k closest neighbours
               1. k - #nearest neighbours
   2. Unsupervised learning