Intertwining Binary Decision Trees and Probabilistic Neural Networks for Maximising Accuracy and Efficiency in Classification Tasks
A Pilot /Proof–of–Concept Demonstration on the Iris Benchmark Classification Dataset
Keywords:Binary Decision Trees, Probabilistic Neural Networks, Classification, BDT/PNN combination, Efficiency, cumulative classification accuracy
Intertwining binary decision trees (BDTs) and probabilistic neural networks (PNNs) is put forward as a powerful custom–made methodology for simultaneously maximising accuracy and efficiency in classification tasks. The proposed methodology brings together the complementary strengths of (I) the parametric, global, recursive, efficient as well as maximal dichotomisation of the BDT, and (II) the non–parametric, accurate–to–local–detail, multi–class identification of the PNN. The BDT/PNN combination evolves level–by–level, with each level comprising two steps: (step 1) the optimal BDT (hyper–)linear bi–partitioning of the entire dataset for the first level, and of every non–terminal partition thereof for the next levels, is determined, with each resulting partition being assigned to a new node of the next BDT level; (step 2) as many PNNs are created as there are multi–class partitions resulting from (step 1), implementing the non–parametric creation of the hyperplane that maximises class separability over each corresponding partition of the problem–space. BDT/PNN expansion is applied recursively – and independently – to each non–terminal partition of the dataset, concluding once the cumulative classification accuracy (CCA) of the terminal – BDT and PNN – nodes of the BDT/PNN is maximised. A proof–of–concept demonstration on the iris benchmark classification dataset (IBCD) illustrates the details of the BDT/PNN combination, attesting to its simplicity, effectiveness, and efficiency of operation. Follow–up research shall focus upon the classification of additional benchmark datasets, as well as of “hard” and “Big” real–world problems, for further evaluating and validating the proposed methodology.
Duda, R.O., Hart, P.E., Stork, D.G. (2000). Pattern Classification, 2nd Edition (ISBN: 978–0–471–05669–0 November 2000)
Loh W.Y., Vanichsetakul N. (1988). Tree–structured classification via generalized discriminant analysis, Journal of the American Statistical Association, vol. 83, pp. 715–728
Mitchell, T. (1997). Decision tree learning, in Machine Learning, The McGraw–Hill Companies, Inc., pp. 52–78
Specht, D. (1990). Probabilistic neural networks, Neural Networks, vol. 3, pp. 109–118
Specht, D.F. (1990). Probabilistic neural networks for classification, mapping, or associative memory, IEEE 1998 International Conference on Neural Networks, vol. 1, pp. 111–121
Anderson, E. (1935). The irises of the Gaspé Peninsula, Bulletin of the American Iris Society, vol. 59, pp. 2–5
Hyafil, L, Rivest, R.L. (1976). "Constructing optimal binary decision trees is NP–complete." Information Processing Letters 5 (1976), pp. 15–17
Fisher, R.A. (1936). The Use of Multiple Measurements in Taxonomic Problems, Annals of Eugenics, vol. 7, pp. 179–188
Gorban A.N., Sumner N.R., Zinovyev A.Y 2007 · Topological grammars for data approximation, Applied Mathematics Letters, vol. 20 (2007) pp. 382–386, DOI:10.1016/j.aml.2006.04.022
Curram, S.P., Mingers, J. (1994). Neural networks, decision tree induction and discriminant analysis: an empirical comparison, Journal of the Operational Research Society, vol. 45, pp. 440–450
Bouzida, Y., Cuppens, F. (2006). Neural networks vs. decision trees for intrusion detection, Proceedings of the IEEE/IST Workshop on Monitoring, Attack Detection and Mitigation, Tuebingen, Germany, September 28th–29th, 2006
Tso, G.K.F., Yau, K.W. (2007). Predicting electricity energy consumption: A comparison of regression analysis, decision tree and neural networks, Energy, vol. 32, pp. 1761–1768
Jerez–Aragonés, J.M., Gómez–Ruiz, J.A., Ramos–Jiménez, G., Muñoz–Pérez, J., Alba–Conejo, E. (2003). A combined neural network and decision trees model for prognosis of breast cancer relapse, Artificial Intelligence in Medicine, vol. 27, pp. 45–63
Goel, P.K., Prasher, S.O., Patel, R.M., Landry, A.J., Bonnell, R.B., Viau, A.A. (2003). Classification of hyperspectral data by decision trees and artificial neural networks to identify weed stress and nitrogen status of corn, Computers and Electronics in Agriculture, vol. 39, pp. 67–93
Devijver P.A., Kittler J. (1982). Pattern Recognition: A Statistical Approach, Prentice–Hall, London, U.K.
How to Cite
Copyright (c) 2022 Tatiana Tambouratzis, Dora Souliou
This work is licensed under a Creative Commons Attribution 4.0 International License.