Cluster based ensemble classifier generation by joint optimization of accuracy and diversity

This paper presents an algorithm to generate ensemble classifier by joint optimization of accuracy and diversity. It is expected that the base classifiers in an ensemble are accurate and diverse (i.e. complementary in terms of errors) among each other for the ensemble classifier to be more accurate. We adopt a Multi–Objective Evolutionary Algorithm (MOEA) for joint optimization of accuracy and diversity on our recently developed Non–Uniform Layered Cluster Oriented Ensemble Classifier (NULCOEC). In NULCOEC, the data set is partitioned into a variable number of clusters at different layers. Base classifiers are then trained on the clusters at different layers. The performance of NULCOEC is a function of the vector of the number of layers and clusters. The research presented in this paper investigates the implication of applying MOEA to generate NULCOEC. Accuracy and diversity of the ensemble classifier is expressed as a function of layers and clusters. A MOEA then searches for the combination of layers and clusters to obtain the non–dominated set of (accuracy,diversity). We have obtained the results of single objective optimization (i.e. optimizing either accuracy or diversity) and compared them with the results of MOEA on sixteen UCI data sets. The results show that the MOEA can improve the performance of ensemble classifier.