Classification is extensively used in the context of medical image analysis for the purpose of diagnosis or prognosis. In order to classify image content correctly, one needs to extract efficient features with discriminative properties and build classifiers based on these features. In addition, a good metric is required to measure distance or similarity between feature points so that the classification becomes feasible. Furthermore, in order to build a successful classifier, one needs to deeply understand how classifiers work. This thesis focuses on these three aspects of classification and explores these challenging areas.

The first focus of the thesis is to properly combine different local feature experts and prior information to design an effective classifier. The preliminary classification results, provided by the experts, are fused in order to develop an automatic segmentation method to segment breast tissue and pectoral muscle area from the background in mammogram. The second focus is the choices of metric and its influence to the feasibility of a classifier, especially on k-nearest neighbors (k-NN) algorithm, with medical applications on breast cancer prediction and calcification detection in a cardiovascular disease study. The third focus is to deepen the understanding of classification mechanism by visualizing the knowledge learned by a classifier. More specifically, to build the most typical patterns recognized by the Fisher's linear discriminant rule with applications on characterizing human faces and emphysema disease in lung CT images.