SWITCH: A novel approach to ensemble learning for heterogeneous data

The standard framework of machine learning problems assumes that the available data is independent and identically distributed (i.i.d.). However, in some applications such as image classification, the training data are often collected from multiple sources and heterogeneous. Ensemble learning is a proven effective approach to heterogeneous data, which uses multiple classification models to capture the diverse aspects of heterogeneous data. If an ensemble can learn the relationship between different portions of data and their corresponding models, the ensemble can selectively apply models to unseen data according to the learned relationship. We propose a novel approach to enable the learning of the relationships between data and models by creating a set of 'switches' that can route a testing instance to appropriate classification models in an ensemble. Our empirical study on both real-world data and benchmark data shows that the proposed approach to ensemble learning can achieve significant performance improvement for heterogeneous data.