【COMP337 LEC 5-6】
LEC 5 Perceptron ：
Binary Classification Algorithm
8
感应器是 单个神经元的模型
突触连接的强度取决于接受外部刺激的反应
X input
W weights
a = x1*w1+x2*w2....... > / < threshold
Bias
MaxIter is a hyperparameter 超参数 which has to be chosen experimentally
• If we make many passes over the training data, then the algorithm is likely to overfit.
• If we make few passes might lead to underfitting
意思就是这个值，需要不断地手工修改枚举法，保证即不overfit，也不underfitting
LEC 6 Perceptron ：
Geometric Interpretation 几何解释
Hyperplane 超平面
Linear separability
1. When a dataset is linearly separable, there can exist more than one hyperplanes that separates the dataset into positive/negative groups.
当线性分离的时候，存在多个hyperplanes
not unique
2. 所以当不是线性分离的时候，就不会存在hyperplane
Take the average over all weight vectors during the training (Averaged Perceptron algorithm)
Multiclass Classification
Multiclass classifiers
• k-NN
• Naive Bayes
Binary classifiers
• Perceptron
• Logistic regression
How to turn a binary classifier to a multiclass classifier?
Given binary classification algorithm A we want to design a meta-algorithm that use A to make k-class predictions.
两种方法：
One-vs.-one approach
There might be ambiguity if some classes got the same number of votes (if the binary classifier A can produce a confidence score, it can be used to break ties)
One-vs.-rest approach
Drawbacks
1. the scale of the confidence scores may differ between the binary classifiers
2. the binary classifiers are trained on unbalanced datasets: usually the set ofnegative objects will be much larger than the set of positive objects