Abstract

Extreme Learning Machine is a fast single layer feed forward neural network for real valued classification. It suffers from the problem of instability and over fitting. Voting based Extreme Learning Machine, VELM reduces this performance variation in Extreme Learning Machine by employing majority voting based ensembling technique. VELM improves the performance of ELM at the cost of increased redundancy. This problem can be reduced using ensemble pruning techniques. This work proposes and evaluates Voting based Extreme Learning Machine with Accuracy based ensemble Pruning, VELM_AP. VELM_AP generates component classifier in the same way as VELM.

References

• G. -B. Huang, Q. -Y. Zhu, and C. -K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70. pp. 489–501, 2006.
• L. Breiman, "Bagging predictors: Technical Report No. 421," 1994.
• Y. Freund and R. Schapire, "Experiments with a new boosting algorithm," Mach. Learn. Work. …, 1996.
• J. Cao, Z. Lin, G. Bin Huang, and N. Liu, "Voting based extreme learning machine," Inf. Sci. (Ny). , vol. 185, pp. 66–77, 2012.
• J. Cao, S. Kwong, R. Wang, X. Li, K. Li, and X. Kong, "Class-specific soft voting based multiple extreme learning machines ensemble," Neurocomputing, vol. 149, Part , no. 0, pp. 275–284, Feb. 2015.
• N. Liu and H. Wang, "Ensemble based extreme learning machine," IEEE Signal Process. Lett. , vol. 17, pp. 754–757, 2010.
• J. Zhai, H. Xu, and X. Wang, "Dynamic ensemble extreme learning machine based on sample entropy," Soft Computing, vol. 16. pp. 1493–1502, 2012.
• Y. Lan, Y. C. Soh, and G. Bin Huang, "Ensemble of online sequential extreme learning machine," Neurocomputing, vol. 72. pp. 3391–3395, 2009.
• G. Wang and P. Li, "Dynamic Adaboost ensemble extreme learning machine," in ICACTE 2010 - 2010 3rd International Conference on Advanced Computer Theory and Engineering, Proceedings, 2010, vol. 3.
• L. Yu, X. Xiujuan, and W. Chunyu, "Simple ensemble of extreme learning machine," in Proceedings of the 2009 2nd International Congress on Image and Signal Processing, CISP'09, 2009.
• Z. Lu, X. Wu, X. Zhu, and J. Bongard, "Ensemble Pruning via Individual Contribution Ordering," in The 16th ACM SIGKDD Conference on Knowledge Discovery and Data Mining Washington DC, 2010, no. 1, pp. 871–880.
• T. Windeatt and C. Zor, "Ensemble Pruning Using Spectral Coefficients," IEEE Trans. Neural Networks Learn. Syst. , vol. 24, no. 4, pp. 673–678.
• L. Guo and S. Boukir, "Margin-based ordered aggregation for ensemble pruning," Pattern Recognit. Lett. , vol. 34, no. 6, pp. 603–609, 2013.
• I. Partalas, G. Tsoumakas, and I. Vlahavas, "An ensemble uncertainty aware measure for directed hill climbing ensemble pruning," Mach. Learn. , vol. 81, no. 3, pp. 257–282, 2010.
• G. Martinez-Muñoz, D. Hernández-Lobato, and A. Suarez, "An analysis of ensemble pruning techniques based on ordered aggregation," IEEE Trans. Pattern Anal. Mach. Intell. , vol. 31, no. 2, pp. 245–259, 2009.
• Y. Zhang, S. Burer, and W. N. Street, "Ensemble pruning via semi-definite programming," J. Mach. Learn. Res. , vol. 7, pp. 1315–1338, 2006.
• L. Rokach, "Collective-agreement-based pruning of ensembles," Comput. Stat. Data Anal. , vol. 53, no. 4, pp. 1015–1026, 2009.
• G. Martínez-Muñoz and A. Suárez, "Aggregation Ordering in Bagging," in Proceedings of the {IASTED} International Conference on Artificial Intelligence and Applications, 2004, pp. 258–263.
• Z. H. Zhou, J. Wu, and W. Tang, "Ensembling neural networks: Many could be better than all," Artif. Intell. , vol. 137, no. 1–2, pp. 239–263, 2002.
• Z. -H. Zhou, Ensemble Methods: Foundations and Algorithms. 2012.
• M. Bhardwaj and V. Bhatnagar, "Towards an optimally pruned classifier ensemble," Int. J. Mach. Learn. Cybern. , pp. 1–20, 2014.
• J. Alcalá-Fdez, A. Fernández, J. Luengo, J. Derrac, S. García, L. Sánchez, and F. Herrera, "KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework," J. Mult. Log. Soft Comput. , vol. 17, pp. 255–287, 2011.