Abstract

The search engines’ evolution from basic keyword-matching systems to AI-enabled search engines has changed how users search for information in the digital landscape. This paper maps out the technological evolution, starting with something as basic as early search engines like Archie and AltaVista, using the initial iterations of PageRank, and leading up to the technologies currently in use, AI-enabled systems that leverage deep learning, natural language processing (NLP), and transformer models like BERT. Areas like understanding semantics, large language models (LLMs), retrieval augmented generation (RAG), and vector databases will be focused on. Applications in e-commerce, healthcare, and research will be discussed, along with challenges including algorithmic bias, misinformation, SEO poisoning, and privacy. This paper will conclude with a preview of the future of retrieval, conversational AI, and multimodal retrieval.

References

• M. Al-Thgfafi, H. Alshamrani, A. Qureshi, and R. Alghamdi. An intelligent semantic search engine for academic journals using hybrid ai techniques. Journal of Intelligent Systems and Applications, 3(1), February 2025.
• Marwah Alaofi, Negar Arabzadeh, Charles LA Clarke, and Mark Sanderson. Generative information retrieval evaluation. In Information Access in the Era of Generative AI, pages 135– 159. Springer, 2024.
• Fedor Borisyuk, Lars Hertel, Ganesh Parameswaran, Gaurav Srivastava, Sudarshan Srinivasa Ramanujam, Borja Ocejo, Peng Du, Andrei Akterskii, Neil Daftary, Shao Tang, et al. From features to transformers: Redefining ranking for scalable impact. arXiv preprint arXiv:2502.03417, 2025.
• Sergey Brin and Lawrence Page. The anatomy of a largescale hypertextual web search engine. Computer networks and ISDN systems, 30(1-7):107–117, 1998.
• Kailash A Hambarde and Hugo Proenca. Information retrieval: recent advances and beyond. IEEE Access, 11:76581– 76604, 2023.
• Dongzhi Jiang, Renrui Zhang, Ziyu Guo, Yanmin Wu, Jiayi Lei, Pengshuo Qiu, Pan Lu, Zehui Chen, Chaoyou Fu, Guanglu Song, et al. Mmsearch: Benchmarking the potential of large models as multi-modal search engines. arXiv preprint arXiv:2409.12959, 2024.
• Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt¨aschel, et al. Retrievalaugmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459– 9474, 2020.
• Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. Pretrained transformers for text ranking: Bert and beyond. Springer Nature, 2022.
• Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. Multi-stage document ranking with bert. arXiv preprint arXiv:1910.14424, 2019.
• SearchUnify. Conversational ai trends in 2025: Explore the future of digital interactions, 2025. SearchUnify Blog.
• Amazon Web Services. Offerup improved local results by 54% and relevance recall by 27% with multimodal search, 2025. AWS Machine Learning Blog.
• LikangWu, Zhi Zheng, Zhaopeng Qiu, HaoWang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, et al. A survey on large language models for recommendation. World Wide Web, 27(5):60, 2024.
• Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, BeiWang, Jimmy Lin, and Vivek Srikumar. A survey of model architectures in information retrieval. arXiv preprint arXiv:2502.14822, 2025.