Statistical Pattern Recognition Chapter 1-2 Notes:

一: Introduction to Statistical Pattern Recognition

1.4 Approaches to statistical pattern recognition

Some important points to make about the design process: a). Finite design set, (overfit — underfit) -> generalization performance; b). Optimality; c). Representative data.

Supervised classification (or discrimination), unsupervised classification (sometimes in the statistics literature simply referred to as classification or clustering)

Decision rule, decision boundaries or decision surfaces .

Two approaches to discrimination: The first assumes a knowledge of the underlying class-conditional probability density functions (the probability density function of the feature vectors for a given class); the second approach introduced in the next section develops decision rules that use the data to estimate the decision boundaries directly, without explicit calculation of the probability density functions.

1.5.1 Bayes’ decision rule for minimum error

A decision rule based on probabilities is to assign \({\bf{x}}\) (here we refer to an object in terms of its measurement vector) to class \({\omega _j}\) if the probability of class \({\omega _j}\) given the observation \({\bf{x}}\), that is \(p({\omega _j}|{\bf{x}})\), is greatest over all classes \({\omega _1}\), …, \({\omega _C}\).

Bayes’ rule for minimum error: \(p({\bf{x}}|{\omega _j})p({\omega _j}) > p({\bf{x}}|{\omega _k})p({\omega _k})\begin{array}{*{20}{c}}

{}&{k = 1,…,C;k \ne j}

\end{array}\).

a posteriori probabilities, a priori probabilities.

1.5.2 Bayes’ decision rule for minimum error – reject option

As we have stated above, an error or misrecognition occurs when the classifier assigns a pattern to one class when it actually belongs to another. In this section we consider the reject option. Usually it is the uncertain classifications (often close to the decision boundaries) that contribute mainly to the error rate. Therefore, rejecting a pattern (withholding a decision) may lead to a reduction in the error rate. This rejected pattern may be discarded, or set aside until further information allows a decision to be made. Although the option to reject may alleviate or remove the problem of a high misrecognition rate, some otherwise correct classifications are also converted into rejects. Here we consider the trade-offs between error rate and reject rate.

1.5.3 Bayes’ decision rule for minimum risk

Bayes’ rule developed assumes that the a priori distributions and the class-conditional distributions are known. In a real-world task, this is unlikely to be so. Therefore approximations must be made based on the data available.

[参考] 1. Statistical Pattern Recognition.[M] Webb. 2011