Abstract

The goal of this research is to develop a text summarization program using word frequency in Python. The purpose of text summarization is to provide a short and useful summary of the text. The word frequency is used to measure the importance of words and sentences in the text. The basic steps of text summarization are explained: preprocessing text, creating list of words, creating bag of words, creating word frequency, creating list of sentences, creating sentence score, sorting sentence score, calculating average score, and making summary. The developed program was tested on an experimental text from Wikipedia. The program successfully performed the basic steps of text summarization and provided the required results.

References

• Sammut, C., & Webb, G. I. (2011). "Encyclopedia of Machine Learning". Springer.
• Torres-Moreno, J. M. (2014). "Automatic Text Summarization". John Wiley & Sons.
• Das, D., & Martins, A. F. (2007). "A Survey on Automatic Text Summarization". Language Technologies Institute. Language Technologies Institute.
• Orasan, C. (2009). "Comparative Evaluation of Term-Weighting Methods for Automatic Summarization". Journal of Quantitative Linguistics. 16, 67-95.
• Gupta, V., & Lehal, G. S. (2010). "A Survey of Text Summarization Extractive Techniques". Journal of Emerging Technologies in Web Intelligence, 2(3), 258-268.
• Lloret, E. & Palomar, M. (2012). "Text Summarization in Progress: A Literature Review". Artificial Intelligence Review. 37, 1-41.
• Nenkova, A., & McKeown, K. (2012). "A Survey of Text Summarization Techniques". In Mining Text Data, Springer, 43-76.
• Saranyamol, C. S., & Sindhu, L. (2014). "A Survey on Automatic Text Summarization". International Journal of Computer Science and Information Technologies, 5(6), 7889-7893.
• Dalal, V., & Shelar, Y. (2015). "Survey of Various Methods for Text Summarization". International Journal of Engineering Research & Development, 11(3), 57-59.
• Saziyabegum, S., & Sajja, P. S. (2016). "Literature Review on Extractive Text Summarization Approaches". International Journal of Computer Applications, 156(12).
• Kumar, Y.J., Goh, O.S., Basiron, H., Choon, N.H., & Suppiah, P.C. (2016). "A Review on Automatic Text Summarization Approaches". J. Comput. Sci., 12, 178-190.
• Gambhir, M., & Gupta, V. (2017). "Recent Automatic Text Summarization Techniques: A Survey". Artificial Intelligence Review, 47, 1-66.
• Allahyari, M., Pouriyeh, S., Assefi, M., Safaei, S., Trippe, E. D., Gutierrez, J. B., & Kochut, K. (2017). "Text sSummarization Techniques: A Brief Survey". arXiv preprint arXiv:1707.02268.
• Bharti, S. K., & Babu, K. S. (2017). "Automatic Keyword Extraction for Text Summarization: A Survey". arXiv preprint arXiv:1704.03242.
• Zerari, N., Aitouche, S., Mouss, M. D., & Yaha, A. (2017). "Automatic Text Summarization: A Review". In Proceedings of the 9th International Conference on Information, Process, and Knowledge Management.
• Klymenko, O., Braun, D., & Matthes, F. (2020). "Automatic Text Summarization: A State-of-the-Art Review". In Proceedings of the 22nd International Conference on Enterprise Information Systems (ICEI), 1, 648-655.
• Mridha, M. F., Lima, A. A., Nur, K., Das, S. C., Hasan, M., & Kabir, M. M. (2021). "A Survey of Automatic Text Summarization: Progress, Process and Challenges". IEEE Access, 9, 156043-156070.
• Yadav, A.K., Maurya, A.K., Ranvijay, & Yadav, R.S. (2021). "Extractive Text Summarization Using Recent Approaches: A Survey". Ingénierie des Systèmes d Inf., 26, 109-121.
• Cajueiro, D.O., Nery, A.G., Tavares, I., Melo, M.K., Reis, S.A., Weigang, L., & Celestino, V.R. (2023). "A Comprehensive Review of Automatic Text Summarization Techniques: Method, Data, Evaluation and Coding". ArXiv, abs/2301.03403.
• Luhn, H. (1958). "The Automatic Creation of Literature Abstracts". IBM Journal of Research and Development, 2(2), 159-165.
• Salton, G., Wong, A., & Yang, C. S. (1975a). "A Vector Space Model for Automatic Indexing". Communications of the ACM, 18(11), 613-620.
• Salton, G., Yang, C. S., & Yu, C. T. (1975b). "A Theory of Term Importance in Automatic Text Analysis". Journal of the American Society for Information Science, 26(1), 33-44.
• Salton, G. & McGill, M. (1983). "Introduction to Modern Information Retrieval". McGraw Hill Book Co, New York.
• Salton, G., & Buckley, C. (1988). "Term-Weighting approaches in Automatic Text Retrieval". Information Processing and Management, 24(5), 513-523.
• Salton, G. (1989). "Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer". Addison- Wesley Publishing Company, USA.
• Salton, G., Singhal, A., Mitra, M., & Buckley, C. (1997). "Automatic Text Structuring and Summarization". Information Processing & Management, 33(2), 193-207.
• Sparck Jones, K. (1972). "A Statistical Interpretation of Term Specificity and Its Application in Retrieval". Journal of Documentation. 28(1), 11–21.
• Sparck Jones, K. (2004). "IDF Term Weighting and IR Research Lessons". Journal of Documentation, 60(5), 521-523.
• Python: https://www.python.org
• Numpy: https://www.numpy.org
• Pandas: https:// pandas.pydata.org
• Matplotlib: http://www.matplotlib.org
• NLTK: https://www.nltk.org
• SK Learn: https://scikit-learn.org
• Wikipedia: https://en.wikipedia.org