Ensemble classifiers are strong and robust methods for classification and regression tasks. Considering the balance between runtime and classifier accuracy the learning problem becomes a multi-objective optimization problem. In this work, we propose an evolutionary multiobjective algorithm based on non-dominated sorting that balances runtime and accuracy properties of nearest neighbor classifier ensembles and decision tree ensembles. We identify relevant ensemble parameters with a significant impact on the accuracy and runtime. In the experimental part of this paper, we analyze the behavior on typical classification benchmark problems.