Segmentation of tissue components of atherosclerotic plaques in MRI is promising for improving future treatment strategies of cardiovascular diseases. Several methods have been proposed before with varying results. This study aimed to perform a structured comparison of various classifiers, training set sizes, and MR image sequences to determine the most promising strategy for methodology development. Five different classifiers (linear discriminant classifier (LDC), quadratic discriminant classifier (QDC), random forest (RF), and support vector classifiers with both a linear (SVM_lin) and radial basis function kernel (SVM_rbf)) were evaluated. We used carotid MRI data from 124 symptomatic patients, scanned in 4 centres with 2 different MRI protocols (45 and 79 patients). Firstly, learning curves of accuracy as a function of increasing training data size showed stabilisation of performance after using ∼10–15 patients for training. Best results were found for LDC, QDC and RF. Intraplaque haemorrhage was most accurately classified in both protocols, and lowest accuracy was found for the lipid-rich necrotic core. Secondly, for LDC and RF it was shown that leaving out different MRI sequences usually negatively affects results for one or more classes. However, leaving out T2-weighted scans did not have a big impact. In conclusion, several classifiers obtain generally good results for classification of plaque components in MRI. Identification of intraplaque haemorrhage is the most promising, and lipid-rich necrotic core remains the most difficult.