Pattern Recognition, 4th Edition

3.3. The Perceptron Algorithm

Our major concern now is to compute the unknown parameters wⁱ, i = 0,…, l, defining the decision hyperplane. In this section, we assume that the two classes ω¹, ω² are linearly separable. In other words, we assume that there exists a hyperplane, defined by w^*Tx = 0, such that(3.5)The formulation above also covers the case of a hyperplane not crossing the origin, that is, w^*Tx + w^*0 = 0, since this can be brought into the previous formulation by defining the extended (l + 1)-dimensional vectors x′ ≡ [x^T, 1]^T, w′ ≡ [w^*T, w^*0]^T. Then w^*Tx + w^*0 = w^′Tx′.

We will approach the problem as a typical optimization task ( ...