1 Introduction
1.1 What is machine learning
Traditional rule-based symbolic approches (expert system)
Data-driven machine learning methods
Data collection \(\rightarrow\) Feature extraction \(\rightarrow\) Model training.
1.2 Basic concepts of machine learning
Consider a machine learning problem as a system with input and output.
Input \(\rightarrow\) Machine learning \(\rightarrow\) Output.
- Classification vs. Regression
Whether the output is continuous, discrete, or a structured object.
- Supervised vs. Unsupervised learning
Whether the output is known.
- Simple vs. Complex models
Linear models vs. Nonlinear models.
- Parametric vs. Nonparametric models
Parametric models take a presumed functional form and are completely determined by a fixed set of model parameters.
- Overfitting vs. Underfitting
data = signal + noise.
Underfitting occurs when the learning performance is not satisfactory even in the training data.
Overfitting occurs when we notice a nearly perfect performance in the training data but a fairly poor performance in another unseen evaluation data.
- Bias-Variance trade-off
Bias - Underfitting.
Variance - Overfitting.
\(\text{learning error} = \text{bias}^2 + \text{variance}\).