Automatic quantification of carotid artery plaque composition is important in the development of methods that distinguish vulnerable from stable plaques. MRI has shown to be capable of imaging different components noninvasively. We present a new plaque classification method which uses 3D registration of histology data with ex vivo MRI data, using non-rigid registration, both for training and evaluation. This is more objective than previously presented methods, as it eliminates selection bias that is introduced when 2D MRI slices are manually matched to histological slices before evaluation. Histological slices of human atherosclerotic plaques were manually segmented into necrotic core, fibrous tissue and calcification. Classification of these three components was voxelwise evaluated. As features the intensity, gradient magnitude and Laplacian in four MRI sequences after different degrees of Gaussian smoothing, and the distances to the lumen and the outer vessel wall, were used. Performance of linear and quadratic discriminant classifiers for different combinations of features was evaluated. Best accuracy (72.5 ± 7.7%) was reached with the linear classifier when all features were used. Although this was only a minor improvement to the accuracy of a classifier that only included the intensities and distance features (71.6 ± 7.9%), the difference was statistically significant (paired t-test, p