Abstract

Although machine learning (ML) has transformed many sectors, issues with scalability, robustness, and maintainability are frequently encountered during deployment and upkeep. To ensure that AI systems are durable, scalable, and maintainable, software engineering concepts must be incorporated into the creation of machine learning pipelines. In the context of developing machine learning pipelines, this study examines many software engineering techniques, including version control, modular design, testing methodologies, and continuous integration/continuous deployment (CI/CD).

References

• Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., & Nagappan, N. (2019). "Software Engineering for Machine Learning: A Case Study." International Conference on Software Engineering (ICSE), 291-300.
• Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., & Dennison, D. (2015). "Hidden Technical Debt in Machine Learning Systems." Advances in Neural Information Processing Systems, 28, 2503-2511.
• Zhang, H., & Chen, Y. (2020). "SEML: A Framework for Software Engineering in Machine Learning Pipeline Development." Journal of Systems and Software, 165, 110576.
• Bosch, J., Olsson, H. H., Björk, J., &Ljungblad, J. (2021). "Software Engineering Challenges for Machine Learning Systems." IEEE Software, 38(6), 36-45.
• Breck, E., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2017). "The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction." IEEE International Conference on Big Data, 1123-1132.
• Nourani, M., & Babbar, D. (2020). "Incorporating Software Engineering Principles in Machine Learning Projects." ACM Transactions on Software Engineering and Methodology, 29(4), 27.
• Amershi, S., & Nagappan, N. (2019). "Software Engineering Best Practices in Machine Learning." IEEE Software, 36(1), 92-100.
• Zhang, X., & Zhu, Y. (2021). "Towards a Unified Software Engineering Framework for Machine Learning." Journal of Software: Evolution and Process, 33(6), e2345.
• Hynes, N., & Cho, H. (2020). "Automating the End-to-End Lifecycle of Machine Learning Models with Software Engineering Practices." Proceedings of the 2020 Conference on Machine Learning Systems, 112-125.
• Zou, D., & Wei, Y. (2021). "An Agile Approach to Integrating Machine Learning Models in Software Engineering." International Journal of Agile Systems and Management, 14(3), 201-216.
• Sato, T., & Toyama, S. (2021). "Machine Learning Operations: Integrating DevOps Principles into Machine Learning Projects." Journal of Information Processing, 29, 123-134.
• Breck, E., &Polyzotis, N. (2019). "Data Management Challenges in Production Machine Learning Systems." Proceedings of the VLDB Endowment, 12(12), 2126-2138.
• Rausch, T., &Dustdar, S. (2020). "Engineering Production-Ready Machine Learning Pipelines." IEEE Internet Computing, 24(2), 23-31.
• Chen, Z., & Kwon, S. (2021). "Continuous Integration and Deployment for Machine Learning Pipelines." ACM Transactions on Software Engineering and Methodology, 30(2), 11.
• Lwakatare, L. E., & Kuvaja, P. (2020). "DevOps for AI: MLOps in Practice." IEEE Software, 37(5), 97-103.
• Arpteg, A., &Brinne, B. (2020). "Software Engineering for AI-Enabled Systems: A Case Study." Journal of Software Engineering Research and Development, 8(1), 5.
• Van der Aalst, W. M. P., & Carmona, J. (2021). "Challenges in Integrating Process Mining and Machine Learning." International Journal of Software and Informatics, 15(1), 1-16.
• Krejca, M. S., & Ribeiro, B. (2020). "Machine Learning in Software Engineering: A Study of Practitioners' Perspectives." Empirical Software Engineering, 25, 138-162.
• Yazdani, A., &Zowghi, D. (2021). "Adopting Software Engineering Practices in Machine Learning Projects: A Systematic Literature Review." Information and Software Technology, 132, 106485.
• Menzies, T., & Zimmermann, T. (2020). "Software Engineering for AI-Based Systems: A Survey of Practices and Challenges." IEEE Transactions on Software Engineering, 47(1), 55-70.