Abstract

Deep Learning is a continuously evolving subset of machine learning techniques. New technology has provided solutions to a wide range of complex problems that were once unsolvable due to limitations in human intelligence. Since its conception, several DL architectures have been developed, including recursive neural networks, recurrent neural networks, artificial neural networks, and convolution neural networks. Many of their contributions have been in the area of computer vision, natural language processing, sequence generation, etc. Despite their increasing popularity, many individuals cannot see the bigger picture or comprehend these techniques. In this paper, the various deep learning models are described, as well as how they work. In addition, the article explains a few prominent DL models and their relevance in contemporary technology. As with every rapidly changing technology, DL has some limitations. These limitations are mitigated to some extent in this paper. Further, it emphasizes their continued development, the challenges they face, and the possibilities for future research in their fields.

References

• LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44
• Zhang Z, Cui P, Zhu W. Deep learning on graphs: a survey. IEEE Trans Knowl Data Eng. 2020. https://doi.org/10. 1109/TKDE.2020.2981333.
• Shrestha A, Mahmood A. Review of deep learning algorithms and architectures. IEEE Access. 2019;7:53040–65
• Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E. Deep learning applications and challenges in big data analytics. J Big Data. 2015;2(1):1.
• Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning, vol. 1. Cambridge: MIT press; 2016
• Hasan RI, Yusuf SM, Alzubaidi L. Review of the state of the art of deep learning for plant diseases: a broad analysis and discussion. Plants. 2020;9(10):1302.
• Shorten C, Khoshgoftaar TM, Furht B. Deep learning applications for COVID-19. J Big Data. 2021;8(1):1–54
• Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
• Goh GB, Hodas NO, Vishnu A. Deep learning for computational chemistry. J Comput Chem. 2017;38(16):1291–307.
• Li Y, Zhang T, Sun S, Gao X. Accelerating flash calculation through deep learning methods. J Comput Phys. 2019;394:153–65.
• Yang W, Zhang X, Tian Y, Wang W, Xue JH, Liao Q. Deep learning for single image super-resolution: a brief review. IEEE Trans Multimed. 2019;21(12):3106–21
• Tang J, Li S, Liu P. A review of lane detection methods based on deep learning. Pattern Recogn. 2020;111:107623.
• Zhao ZQ, Zheng P, Xu ST, Wu X. Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst. 2019;30(11):3212–32
• Krizhevsky A, Sutskever I, Hinton GE. Imagenet classifcation with deep convolutional neural networks. Commun ACM. 2017;60(6):84–90.
• He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 770–8.
• Metz C. Turing award won by 3 pioneers in artifcial intelligence. The New York Times. 2019;27
• Nevo S, Anisimov V, Elidan G, El-Yaniv R, Giencke P, Gigi Y, Hassidim A, Moshe Z, Schlesinger M, Shalev G, et al. Ml for food forecasting at scale; 2019. arXiv preprint arXiv:1901.09583.
• Chen H, Engkvist O,Wang Y, Olivecrona M, Blaschke T. The rise of deep learning in drug discovery. Drug Discov Today. 2018;23(6):1241–50.
• Benhammou Y, Achchab B, Herrera F, Tabik S. Breakhis based breast cancer automatic diagnosis using deep learning: taxonomy, survey and insights. Neurocomputing. 2020;375:9–24.
• Wulczyn E, Steiner DF, Xu Z, Sadhwani A,Wang H, Flament- Auvigne I, Mermel CH, Chen PHC, Liu Y, Stumpe MC. Deep learning-based survival prediction for multiple cancer types using histopathology images. PLoS ONE. 2020;15(6):e0233678.
• Brunese L, Mercaldo F, Reginelli A, Santone A. Explainable deep learning for pulmonary disease and coronavirus COVID- 19 detection from X-rays. Comput Methods Programs Biomed. 2020;196(105):608.
• Jamshidi M, Lalbakhsh A, Talla J, Peroutka Z, Hadjilooei F, Lalbakhsh P, Jamshidi M, La Spada L, Mirmozafari M, Dehghani M, et al. Artifcial intelligence and COVID-19: deep learning approaches for diagnosis and treatment. IEEE Access. 2020;8:109581–95.
• Shorfuzzaman M, Hossain MS. Metacovid: a siamese neural network framework with contrastive loss for n-shot diagnosis of COVID-19 patients. Pattern Recogn. 2020;113:107700.
• He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. p. 770–8.
• Saeed MM, Al Aghbari Z, Alsharidah M. Big data clustering techniques based on spark: a literature review. PeerJ Comput Sci. 2020;6:321.
• Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. Human-level control through deep reinforcement learning. Nature. 2015;518(7540):529–33.
• Socher R, Perelygin A,Wu J, Chuang J, Manning CD, Ng AY, Potts C. Recursive deep models for semantic compo- sitionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing; 2013. p. 1631–42.
• Goller C, Kuchler A. Learning task-dependent distributed representations by backpropagation through structure. In: Proceedings of international conference on neural networks (ICNN’96), vol 1. IEEE; 1996. p. 347–52.
• Socher R, Lin CCY, Ng AY, Manning CD. Parsing natural scenes and natural language with recursive neural net- works. In: ICML; 2011.
• Louppe G, Cho K, Becot C, Cranmer K. QCD-aware recursive neural networks for jet physics. J High Energy Phys. 2019;2019(1):57.
• Sadr H, Pedram MM, Teshnehlab M. A robust sentiment analysis method based on sequential combination of convolutional and recursive neural networks. Neural Process Lett. 2019;50(3):2745–61.
• Urban G, Subrahmanya N, Baldi P. Inner and outer recursive neural networks for chemoinformatics applications. J Chem Inf Model. 2018;58(2):207–11.
• Batur Dinler O¨ , Aydin N. An optimal feature parameter set based on gated recurrent unit recurrent neural net- works for speech segment detection. Appl Sci. 2020;10(4):1273.
• Jagannatha AN, Yu H. Structured prediction models for RNN based sequence labeling in clinical text. In: Proceed- ings of the conference on empirical methods in natural language processing. conference on empirical methods in natural language processing, vol. 2016, NIH Public Access; 2016. p. 856.
• Pascanu R, Gulcehre C, Cho K, Bengio Y. How to construct deep recurrent neural networks. In: Proceedings of the second international conference on learning representations (ICLR 2014); 2014.
• Glorot X, Bengio Y. Understanding the difculty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artifcial intelligence and statistics; 2010. p. 249–56.
• Gao C, Yan J, Zhou S, Varshney PK, Liu H. Long short-term memory-based deep recurrent neural networks for target tracking. Inf Sci. 2019;502:279–96.
• Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T, Wang X, Wang G, Cai J, et al. Recent advances in convolutional neural networks. Pattern Recogn. 2018;77:354–77.
• Goodfellow I, Bengio Y, Courville A, Bengio Y. Deep learning, vol. 1. Cambridge: MIT press; 2016
• O. M. Parkhi, A. Vedaldi, C. V. Jawahar and A. Zisserman, ”The truth about cats and dogs,” 2011 International Conference on Computer Vision, 2011, pp. 1427-1434, doi: 10.1109/ICCV.2011.6126398.
• HUBEL DH, WIESEL TN. Receptive fields of single neurones in the cat’s striate cortex. J Physiol. 1959;148(3):574- 591. doi:10.1113/jphysiol.1959.sp006308
• Alzubaidi L, Fadhel MA, Al-Shamma O, Zhang J, Santamar ´ıa J, Duan Y, Oleiwi SR. Towards a better understanding of transfer learning for medical imaging: a case study. Appl Sci. 2020;10(13):4523.
• Tan C, Sun F, Kong T, Zhang W, Yang C, Liu C. A survey on deep transfer learning. In: International conference on artifcial neural networks. Springer; 2018. p. 270–9.
• Weiss K, Khoshgoftaar TM, Wang D. A survey of transfer learning. J Big Data. 2016;3(1):9
• Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. J Big Data. 2019;6(1):60
• Wang S, Sun S, Xu J. Auc-maximized deep convolutional neural felds for sequence labeling 2015. arXiv preprint arXiv:1511.05265.
• Johnson JM, Khoshgoftaar TM. Survey on deep learning with class imbalance. J Big Data. 2019;6(1):27
• Yang P, Zhang Z, Zhou BB, Zomaya AY. Sample subset optimization for classifying imbalanced biological data. In: Pacifc- Asia conference on knowledge discovery and data mining. Springer; 2011. p. 333–44
• Li Y, Huang C, Ding L, Li Z, Pan Y, Gao X. Deep learning in bioinformatics: introduction, application, and perspective in the big data era. Methods. 2019;166:4–21
• Choi E, Bahadori MT, Sun J, Kulas J, Schuetz A, Stewart W. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In: Advances in neural information processing systems. San Mateo: Morgan Kaufmann Publishers; 2016. p. 3504–12
• Ching T, Himmelstein DS, Beaulieu-Jones BK, Kalinin AA, Do BT, Way GP, Ferrero E, Agapow PM, Zietz M, Hof- man MM, et al. Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface. 2018;15(141):20170,387
• Pokuri BSS, Ghosal S, Kokate A, Sarkar S, Ganapathysubramanian B. Interpretable deep learning for guided microstructure-property explorations in photovoltaics. NPJ Comput Mater. 2019;5(1):1–11.
• Ribeiro MT, Singh S, Guestrin C. “Why should I trust you?” explaining the predictions of any classifer. In: Proceed- ings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; 2016. p. 1135–44.
• Sundararajan M, Taly A, Yan Q. Axiomatic attribution for deep networks; 2017. arXiv preprint arXiv:1703.01365
• Platt J, et al. Probabilistic outputs for support vector machines and comparisons to regularized likelihood meth- ods. Adv Large Margin Classif. 1999;10(3):61–74
• Nair T, Precup D, Arnold DL, Arbel T. Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation. Med Image Anal. 2020;59:101557
• Herzog L, Murina E, D¨urr O, Wegener S, Sick B. Integrating uncertainty in deep neural networks for MRI based stroke analysis. Med Image Anal. 2020;65:101790
• Pereyra G, Tucker G, Chorowski J, Kaiser Ł, Hinton G. Regularizing neural networks by penalizing confdent output distributions; 2017. arXiv preprint arXiv:1701.06548
• Naeini MP, Cooper GF, Hauskrecht M. Obtaining well calibrated probabilities using bayesian binning. In: Proceed- ings of the... AAAI conference on artifcial intelligence. AAAI conference on artifcial intelligence, vol. 2015. NIH Public Access; 2015. p. 2901
• Li M, Sethi IK. Confdence-based classifer design. Pattern Recogn. 2006;39(7):1230–40.
• Zadrozny B, Elkan C. Obtaining calibrated probability estimates from decision trees and Naive Bayesian classifers. In: ICML, vol. 1, Citeseer; 2001. p. 609–16
• Steinwart I. Consistency of support vector machines and other regularized kernel classifers. IEEE Trans Inf Theory. 2005;51(1):128–42
• Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural net- works from overftting. J Mach Learn Res. 2014;15(1):1929–58
• Xu Q, Zhang M, Gu Z, Pan G. Overftting remedy by sparsifying regularization on fully-connected layers of CNNs. Neurocomputing. 2019;328:69–74
• Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. Understanding deep learning requires rethinking generalization. Commun ACM. 2018;64(3):107–15
• Xu X, Jiang X, Ma C, Du P, Li X, Lv S, Yu L, Ni Q, Chen Y, Su J, et al. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering. 2020;6(10):1122–9.
• Zhang G, Wang C, Xu B, Grosse R. Three mechanisms of weight decay regularization; 2018. arXiv preprint arXiv: 1810.12281
• Laurent C, Pereyra G, Brakel P, Zhang Y, Bengio Y. Batch normalized recurrent neural networks. In: 2016 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE; 2016. p. 2657–61
• Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural net- works from overftting. J Mach Learn Res. 2014;15(1):1929–58
• Pereyra G, Tucker G, Chorowski J, Kaiser Ł, Hinton G. Regularizing neural networks by penalizing confdent output distributions; 2017. arXiv preprint arXiv:1701.06548
• D’Amour A, Heller K, Moldovan D, Adlam B, Alipanahi B, Beutel A, Chen C, Deaton J, Eisenstein J, Hofman MD, et al. Underspecifcation presents challenges for credibility in modern machine learning; 2020. arXiv preprint arXiv: 2011.03395
• Bharati S, Podder P, Mondal MRH. Hybrid deep learning for detecting lung diseases from X-ray images. Inform Med Unlocked. 2020;20:100391
• Heidari M, Mirniaharikandehei S, Khuzani AZ, Danala G, Qiu Y, Zheng B. Improving the performance of CNN to predict the likelihood of COVID-19 using chest X-ray images with preprocessing algorithms. Int J Med Inform. 2020;144:104284
• Al-Timemy AH, Khushaba RN, Mosa ZM, Escudero J. An efcient mixture of deep and machine learning models for COVID-19 and tuberculosis detection using X-ray images in resource limited settings 2020. arXiv preprint arXiv: 2007.08223.
• Abraham B, Nair MS. Computer-aided detection of COVID- 19 from X-ray images using multi-CNN and Bayesnet classifer. Biocybern Biomed Eng. 2020;40(4):1436–45.
• Shin HC, Orton MR, Collins DJ, Doran SJ, Leach MO. Stacked autoencoders for unsupervised feature learn- ing and multiple organ detection in a pilot study using 4D patient data. IEEE Trans Pattern Anal Mach Intell. 2012;35(8):1930–43.
• Sirazitdinov I, Kholiavchenko M, Mustafaev T, Yixuan Y, Kuleev R, Ibragimov B. Deep neural network ensemble for pneumonia localization from a large-scale chest X-ray database. Comput Electr Eng. 2019;78:388–99
• Hossin M, Sulaiman M. A review on evaluation metrics for data classifcation evaluations. Int J Data Min Knowl Manag Process. 2015;5(2):1
• Dr.R.Parthasarathi1: ANN Based Modeling for Performance and Exhaust Emission of DI Diesel Engine using Emulsified Diesel Fuel
• Sushma Tamta: Estimation of Evaporation in Hilly Area by Using Ann and Canfis System Based Models
• Mruga Gurjar: STOCK MARKET PREDICTION USING ANN
• Laura E. Suarez : Learning function from structure in neuromorphic networks
• Arlene Casey1*, Emma Davidson2 , Michael Poon2 , Hang Dong3,4, Daniel Duma1 , Andreas Grivas5 , Claire Grover5 , V´ıctor Su´arez-Paniagua3,4, Richard Tobin5 , William Whiteley2,6, Honghan Wu4,7 and Beatrice Alex1: A systematic review of natural language processing applied to radiology reports
• Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. Bengio,Y. (2014)Generative Adversarial Nets. NIPS 2014
• Arjovsky M, Chintala S, Bottou L (2017) Wasserstein gan. arXiv preprint
• Chen Y, Lai YK, Liu YJ (2018) Cartoongan: Generative adversarial networks for photo cartoonization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 9465–9474
• Alec Radford Luke Metz, Soumit chintala: UNSUPERVISED REPRESENTATION LEARNING WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS
• Mehdi Mirza,Simon Osindero: Conditional Generative Adversarial Nets
• Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Parallel distributed processing: Explorations in the microstructure of cognition, vol. 1. chap. Learning Internal Representations by Error Propagation, pp. 318–362. MIT Press, Cambridge, MA, USA (1986). URL http://dl.acm. org/citation.cfm?id=104279.104293
• Jinbo Xua: Distance-based protein folding powered by deep learning
• Deep Learning in Medical Imaging: Overview and Future Promise of an Exciting New Technique
• Alan L. Yuille Chenxi Liu, ”Limitations of Deep Learning for Vision, and How We Might Fix Them”, The Gradient, 2019.
• Raghavendra Chalapathy, Sanjay Chawla: DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY,2019.
• Wen, Andrew,Fu, Sunyang: Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLPas- a-service implementation.
• Spyns, Peter. (1997). Natural Language Processing in Medicine: An Overview. Methods of information in medicine. 35. 285-301. 10.1055/s-0038-1634681.
• Pereira S, Pinto A, Alves V, Silva CA. Brain Tumor Segmentation Using Convolutional Neural Networks in MRI Images. IEEE Trans Med Imaging. 2016 May;35(5):1240-1251. doi: 10.1109/TMI.2016.2538465. Epub 2016 Mar 4. PMID: 26960222.
• Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE conference on computer vision and pattern recognition; 2014 Jun 23-28; Washington, DC, USA. Piscataway (NJ): Institute of Electrical and Electronics Engineers; 2014. pp. 580–7.
• (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 5, No. 12, 2014
• Chen, Chung-Chi Huang, Hen-Hsen Chen, Hsin-Hsi. (2020). NLP in FinTech Applications: Past, Present and Future.
• Hirschman, L., Gaizauskas, R. (2001). Natural language question answering:the view from here. Natural Language Engineering, 7, 275–300.
• Michael Tschannen, Olivier Bachem, Mario Lucic: “Recent Advances in Autoencoder-Based Representation Learning”, 2018
• Autoencoders, Lopez Pinaya W,Viera S et.al 2019.
• Chandola et al.
• Pang, Guansong Cao, Longbing Aggarwal, Charu. (2021). Deep Learning for Anomaly Detection: Challenges, Methods, and Opportunities. 1127-1130. 10.1145/3437963.3441659.
• Sublime, Jeremie Kalinicheva, Ekaterina. (2019). Automatic Post-Disaster Damage Mapping Using Deep-Learning Techniques for Change Detection: Case Study of the Tohoku Tsunami. Remote Sensing. 11. 1123. 10.3390/rs11091123.
• arXiv:1312.5663 [cs.LG]
• Dong, Ganggang Liao, Guisheng Liu, Hongwei Kuang, Gangyao. (2018). A Review of the Autoencoder and Its Variants: A Comparative Perspective from Target Recognition in Synthetic-Aperture Radar Images. IEEE Geoscience and Remote Sensing Magazine. 6. 44-68. 10.1109/MGRS.2018.2853555.
• Changfan Zhang, Xiang Cheng, Jianhua Liu, Jing He, Guangwei Liu, ”Deep Sparse Autoencoder for Feature Extraction and Diagnosis of Locomotive Adhesion Status”, Journal of Control Science and Engineering, vol. 2018, Article ID 8676387, 9 pages, 2018. https://doi.org/10.1155/2018/8676387
• Huajie Shao , Zhisheng Xiao :ControlVAE: Tuning, Analytical Properties, and Performance Analysis
• X. Yan, J. Yang, K. Sohn, and H. Lee, “Attribute2image: Conditional image generation from visual attributes,” in European Conference on Computer Vision. Springer, 2016, pp. 776–791
• M.-Y. Liu, T. Breuel, and J. Kautz, “Unsupervised image-toimage translation networks,” in Advances in neural information processing systems, 2017, pp. 700–708.
• A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, Dive into Deep Learning, 2020, https://d2l.ai.
• W. Wang, Z. Gan, H. Xu, R. Zhang, G. Wang, D. Shen, C. Chen, and L. Carin, “Topic-guided variational autoencoders for text generation,” arXiv preprint arXiv:1903.07137, 2019.
• Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, and E. P. Xing, “Toward controlled generation of text,” in Proceedings of the 34th International Conference on Machine Learning-Volume 70. JMLR. org, 2017, pp. 1587–1596
• I. Higgins, L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner, “beta-vae: Learning basic visual concepts with a constrained variational framework.” ICLR, vol. 2, no. 5, p. 6, 2017.
• H. Kim and A. Mnih, “Disentangling by factorising,” in International Conference on Machine Learning, 2018, pp. 2654–2663
• Feng, Jie Feng, Xueliang Chen, Jiantong Cao, Xianghai Zhang, Xiangrong Jiao, Licheng Yu, Tao. (2020). Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification. Remote Sensing. 12. 1149. 10.3390/rs12071149.
• Feng, Jie Feng, Xueliang Chen, Jiantong Cao, Xianghai Zhang, Xiangrong Jiao, Licheng Yu, Tao. (2020). Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification. Remote Sensing. 12. 1149. 10.3390/rs12071149.
• Phung, Rhee,. (2019). A High-Accuracy Model Average Ensemble of Convolutional Neural Networks for Classification of Cloud Image Patches on Small Datasets. Applied Sciences. 9. 4500. 10.3390/app9214500.
• Mishra, Vidushi AGARWAL, SMT PURI, NEHA. (2018). COMPREHENSIVE AND COMPARATIVE ANALYSIS OF NEURAL NETWORK. INTERNATIONAL JOURNAL OF COMPUTER APPLICATION. 2. 10.26808/rs.ca.i8v2.15.
• M. S. B. M. M. P. W. “Research Paper on Basic of Artificial Neural Network”. International Journal on Recent and Innovation Trends in Computing and Communication, vol. 2, no. 1, Jan. 2014, pp. 96-100, doi:10.17762/ijritcc.v2i1.2920.
• Grossi, Enzo Buscema, Massimo. (2008). Introduction to artificial neural networks. European journal of gastroenterology hepatology. 19. 1046-54. 10.1097/MEG.0b013e3282f198a0.
• Christos Stergiou and Dimitrios Siganos,“Neural Networks”
• Gopal, Sucharita. ”Unit 188-Artificial Neural Networks for Spatial Data Analysis.” (2000).
• Wang, Su. (2017). Generative Adversarial Networks (GAN): A Gentle Introduction [UPDATED].
• Lotter, W., Kreiman, G. Cox, D. (2015) Unsupervised Learning of Visual Structure Using Predictive Generative Networks CoRR, arXiv:1151.06380.
• Ledig, C., Theis, L., Huszar, F., Caballero, J., Aitken, A. P., Tejani, A., Totz, J., Wang, Z., Shi, W. (2016) Photo-realistic single Image Super-resolution Using a Generative Adversarial Network CoRR,arXiv:1609.04802.
• Isola, P., Zhu, J.-Y., Zhou, T., Efros, A. A. (2016). Imageto- image
• Translation with ConditionalAdversarial Networks. CoRR, arXiv:1611.07004.
• Donahue, J., Kr¨ahenb¨uhl, P. Darrell, T. (2017) Adversarial Feature Learning CoRR, arXiv:1605.09782.
• Glover, J. (2016)Modeling Documents with Generative Adversarial Networks. CoRR, arXiv:1612.09122.
• Li, J., Monroe, W., Shi, T., Jean, S., Ritter, A. Jurafsky, D. (2017)Adversarial Learning for Neural Dialogue Generation CoRR, arXiv:1701.06547.
• Chen X., Athiwaratkun, B., Sun, Y.,Weinberger, K. Cardie, C. (2016) Adversarial Deep Averaging Networks for Crosslingual Sentiment Classification,. CoRR, arXiv:1606.01614.
• Zhang, Y., Barzilay, R. Jaakkola, T. (2017)Aspectaugmented Adversarial Networks for Domain Adaptation. CoRR, arXiv:1701.00188
• arXiv:1312.6114v10 [stat.ML]
• Khurana, Diksha, et al. ”Natural language processing: State of the art, current trends and challenges.” arXiv preprint arXiv:1708.05148 (2017).
• Alshawi, H. (1992). The core language engine. MIT press. Kamp, H., Reyle, U. (1993). Tense and Aspect. In From Discourse to Logic (pp. 483- 689). Springer Netherlands.
• Tillmann, C., Vogel, S., Ney, H., Zubiaga, A., Sawaf, H. (1997, September). Accelerated DP based search for statistical translation. In Eurospeech.
• Bangalore, S., Rambow, O., Whittaker, S. (2000, June). Evaluation metrics for generation. In Proceedings of the first international conference on Natural language generation- Volume 14 (pp. 1-8). Association for Computational Linguistics
• Nießen, S., Och, F. J., Leusch, G., Ney, H. (2000, May). An Evaluation Tool for Machine Translation: Fast Evaluation for MT Research. In LREC
• Papineni, K., Roukos, S., Ward, T., Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics
• Doddington, G. (2002, March). Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proceedings of the second international conference
• McCray, A. T., Nelson, S. J. (1995). The representation of meaning in the UMLS. Methods of information in medicine, 34(1-2), 193-201.
• McGray, A. T., Sponsler, J. L., Brylawski, B., Browne, A. C. (1987, November). The role of lexical knowledge in biomedical text understanding. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 103). American Medical Informatics Association.
• McCray, A. T. (1991). Natural language processing for intelligent information retrieval. In Engineering in Medicine and Biology Society, 1991. Vol. 13: 1991., Proceedings of the Annual International Conference of the IEEE (pp. 1160-1161). IEEE.
• McCray, A. T. (1991). Extending a natural language parser with UMLS knowledge. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 194). American Medical Informatics Association.
• McCray, A. T., Srinivasan, S., Browne, A. C. (1994). Lexical methods for managing variation in biomedical terminologies. In Proceedings of the Annual Symposium on Computer Application in Medical Care (p. 235). American Medical Informatics Association.
• Liu S, Liu S, Cai W, et al. Early diagnosis of Alzheimer’s disease with deep learning. In: International Symposium on Biomedical Imaging, Beijing, China 2014, 1015–18.
• Brosch T, Tam R. Manifold learning of brain MRIs by deep learning. Med Image Comput Comput Assist Interv 2013;16:633–40.
• Prasoon A, Petersen K, Igel C, et al. Deep feature learning for knee cartilage segmentation using a triplanar convolutional neural network. Med Image Comput Comput Assist Interv 2013;16:246–53
• Deenan, Surya Prabha SatheeshKumar, J.. (2014). Image processing methods and its Role in agricultural sector – A study. International Journal of Business Intelligents. 3. 366- 373.
• Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. ”Reducing the dimensionality of data with neural networks.” science 313.5786 (2006): 504-507.
• Ashish Vaswani, Noam Shazeer: “Attention i sall you need”, arXiv:1706.03762v5
• Maxime: “What is a Transformer?”, https://medium.com/inside-machine-learning/what-is-atransformer- d07dd1fbec04
• Vaswani, Ashish Bengio, Samy Brevdo, Eugene Chollet, Francois Gomez, Aidan Gouws, Stephan Jones, Llion Kaiser, Łukasz Kalchbrenner, Nal Parmar, Niki Sepassi, Ryan Shazeer, Noam Uszkoreit, Jakob. (2018). Tensor2Tensor for Neural Machine Translation.
• Wolf, Thomas, et al. ”Transformers: State-of-the-art natural language processing.” Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations. 2020.
• Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473, 2014
• Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. CoRR, abs/1406.1078, 2014.
• Alex Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850, 2013.
• Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112, 2014.
• Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016.
• Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016
• Stefania Cristina : “The Transformer Model”, https://machinelearningmastery.com/the-transformer-model/
• Nurtas, Marat Baishemirov, Zharasbek Tsay, V. Tastanov, M. Zhanabekov, Zh. (2020). APPLYING NEURAL NETWORK FOR PREDICTING CARDIOVASCULAR DISEASE RISK. PHYSICO-MATHEMATICAL SERIES. 4. 28- 34. 10.32014/2020.2518-1726.62.
• Ensar Seker: ”Recursive Neural Networks( RvNN) and Recurrent Neural Network(RNN)” ”https://ai.plainenglish.io/recursive-neural-networks-rvnnsand- recurrent-neural-networks-rnns-2ff6a067ad01
• Andrew Ng: ”Sparse Autoencoder”, https://web.stanford.edu/class/cs294a/sparseAutoencoder.pdf
• rhum Shafkat:”Intuitively Understanding Variational Autoencoders”,https://towardsdatascience.com/intuitivelyunderstanding- variational-autoencoders-1bfe67eb5daf