Abstract

The present paper introduces destructive neural network learning techniques and presents the analysis of the convergence rate of the error in a neural network with and without threshold. Also, a constructive algorithm for rule extraction based on a trained neural network using Gene Expression Programming (GEP) is proposed. The rules are not an easy task due to the large number of examples entered to the input layer. Thus, we can use GEP to encode the rules in the form of logic expression. Finally, the proposed model is evaluated on different public-domain datasets and compared with standard learning models from WEKA, and then the results accentuate that the set of rules extraction from the proposed method is more accurate and brief compared with those achieved by the other models.

References

• Baesens, B. , Setiono, R. ; Mues, C. , Vanthienen, J. 2003Using neural network rule extraction and decision tables for credit-risk evaluation. Manage. Sci. , 49, 312-329.
• Jacobsson, H. 2005Rule extraction from recurrent neural networks: A taxonomy and review. Neural Comput. ,17, 1223-1263.
• Kahramanli, H. ; Allahverdi, N. 2009Rule extraction from trained adaptive neural networks using artificial immune systems. Expert Syst. Appl. , 36, 1513-1522.
• Setiono, R. ; Baesens, B. ; Mues, C. 2009A note on knowledge discovery using neural networks and its application to credit screening, Eur. J. Operation. Res. , 192, 326-332.
• Tickle, A. B. , Andrews, R. ; Golea, M. ; Diederich, J. 1998The truth will come to light: Directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Trans. Neural Netw. ,9, 1057-1067.
• Setiono, R. ; Leow, W. K. 2000FERNN: An algorithm for fast extraction of rules from neural networks. Appl. Intell. ,12, 15-25.
• Baesens, B. T. ,Gestel,V. S. ,Viaene, M. Stepanova, J. Suykens and Vanthienen, J. 2003Benchmarking state of the art classification algorithms for credit scoring, vol. 56, no. 6, 627-635.
• Johansson, U. , Konig, R. and Niklasson, L. 2005Automatically balancing accuracy and comprehensibility in predictive modeling.
• Thrun,S. e. a. ,1991 The MONK's problems: A performance comparison of different learning algorithms, Pittsburgh, 91-197.
• Löfström,T. and Odqvist,P. 2004 RULE EXTRACTION IN DATA MINING - FROM A META LEARNING PERSPECTIVE.
• HumarK. ,NovruzA. , 2009"Rule extraction from trained adaptive neural networks using artificial immune systems", Expert Systems with Applications 36, 1513–1522.
• LiMin. Fu, 1994 Rule generation from neural networks, IEEE Transactions on Systems, Man and Cybernetics, Vol. 24 No. 8 , 1114-1124.
• Towell, G. and Shavlik,J. 1993TheExtraction of Refined Rules From Knowledge Based Neural Networks, Machine Learning, Vol. 131, 71-101.
• Setiono, R. ,Wee,K. H. and Zurada,M. J. 2002 Extraction of Rules from artificial neural network for nonlinear regression, IEEE Transaction Neural Networks, Vol. 23 No. 23, 564-577.
• Krishnan,R. ,Sivakumar,G. and Bhattacharya,P. 1999 A search technique for rule extraction from trained neural networks, Pattern Recognit. Lett. , vol. 20, no. 3, Mar. , 273–280.
• Towell,G. G. ,Shavlik,J. W. and Noordewier,M. O. 1990Refinement of approximate domain theories by knowledge-based neural networks, in Proc. 8th Nat. Conf. Artif. Intell. , Boston, MA, 861–866.
• Thrun,S. B. 1994Extracting provably correct rules from neural networks, in Technical Report IAI-TR-93-5, Institut fur Informatik III Universitat Bonn.
• Craven,M. W. 1996 Extracting comprehensible models from trained neural networks, Ph. D. Thesis, University of Wisconsin, Madison.
• OlcayB. , 2002Extracting decision tree from trained neural networks, ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 456-461.
• Zhou, Z. H. , Jiang, Y. , Yang, Y. B. and Chen, S. F. 2003Extracting neural networks from trained neural network Ensembles, AI Communications, Vol. 16 No. 1, 3-15.
• Garcez, A. , d'Avila,S. , Broda, K. , Gabbay, D. M. 2001Symbolic knowledge extraction from trained neural networks: A sound approach, Artificial Intelligence, Vol. 125, 155-207.
• Tickle, A. B. , Orlowski, M. , and Diederich, J. , 1996DEDEC: A Methodology for Extracting Rules from Trained Artificial Neural Networks, Proceedings of the Rule Extraction from Trained Artificial Neural Networks Workshop.
• Bojarczuk,C. C. , Lopes, H. S. , Freitas, A. A. and Michalk. , 2004 A constrained-syntax genetic programming system for discovering classification rules: application to medial database. Artificial Intelligence in Medicine, Volume 30, Issue 1, 27-48.
• Mitchell,M. MIT Press, 1996an Introduction to Genetic Algorithms.
• Dudoit,S. , Yang, Y. H. , Callow, M. J. and Speed,T. P. 2002Statistical Methods for Identifying Differentially Expressed Genes in Replicated cDNA Microarray Experiments, StatisticaSinica, vol. 12, pages 111-139.
• Reiner,A. , Yekutieli, D. and Benjamini, Y. 2003Identifying ifferentially Expressed Genes Using False Discovery Rate Controlling Procedures, Bioinformatics, vol. 19, no. 3 , pages 368-375.
• Efron,A. ,Tibshirani,R. ,Storey,J. D. and Tusher,V. 2001Empirical Bayes Analysis of a Microarray Experiment, J. Am. Statistical Assoc. , vol. 96, pages 1151-1160.
• Creighton,C. and Hanash,S. 2003Mining Gene Expression Databases for Association rules, Bioinformatics, vol. 19, no. 1, 79-86.
• Brown, M. P. S. et al. , 2000Knowledge-Based Analysis of Microarray Gene Expression Data by Using Support Vector Machines,Proc. Nat'l Academy of Sciences USA, vol. 97, no. 1, pages 262-267.
• Jiang, D. , Tang,C. and Zhang, A. 2004Cluster Analysis for Gene Expression Data: A Survey, IEEE Trans. Knowledge and Data Eng. , vol. 16, no. 11, (Nov. 2004) pages 1370-1386.
• Pan,F. et al. ,2003 "Carpenter: Finding Closed Patterns in Long Biological Datasets," Proc. Ninth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '03).
• Cong ,G. et al. , 2004Farmer: Finding Interesting Rule Groups in Microarray Datasets, Proc. ACM SIGMOD Int'l Conf. Management of Data (SIGMOD '04).
• Cong , G. et al. , 2005Mining Top-k Covering Rule Groups for Gene Expression Data, Proc. ACM SIGMOD Int'l Conf. Management of Data (SIGMOD '05).
• Wang,H. C. and Lee,Y. S. 2005 Gene Network Prediction from Microarray Data by Association Rule and Dynamic Bayesian Network, Proc. Int'l Conf. Computational Science and Its Applications (ICCSA), pages 309-317.
• Shang,X. Q. , Zhao, Q. and Li,Z. H. 2009Mining High-Correlation Association Rules for Inferring Gene Regulation Networks, Proc. 11th Int'l Conf. Data Warehousing and Knowledge Discovery (DaWaK '09),pages 244-255.
• Pandey,G. , Atluri, G. ,Steinbach,M. and Kumar, V. 2008Association Analysis Techniques for Discovering Functional Modules from Microarray Data, Nature Precedings.
• Marghny. H. M, Minamoto,T. and Niijima,K. 1990"Rules extraction by constructive learning of neural networks and hidden unit clustering", Lecture Notes in Artificial Intelligence, 1721, Springer, Proc. of the Second International Conference on Discovery Science, pages 343-344.
• Marghny. H. M, and Niijima,K. 2000"Extracting rules from neural network by removing unnecessary connections", Proc. of the Second ICSC Symposium on Neural Computation, pages 322-328.
• Marghny. H. M, 2011 Rules extraction from constructively trained neural networks based on genetic algorithms
• WEKA at http://www. cs. waikato. ac. nz/~ml/wek.
• Marghny. H. M, and Niijima,K. , 2000 " Redundant connections effect on the error rate for the neural networks", Proc. of the Seventh International Conference on Neural Computation Processing, pages 981-985.