Abstract

EdgeShard represents a significant advancement in edge-based large language model (LLM) inference, enabling efficient, accurate, and privacy-preserving deployment by intelligently partitioning and scheduling computation across multiple edge devices. This collaborative approach outperforms traditional quantization and unstable cloud-edge methods. However, distributing inference across heterogeneous and potentially unreliable devices introduces new risks for robustness - such as increased vulnerability to device failures and attacks, and challenges for auditability, including fragmented execution logs and difficulties in tracing and verifying the end-to-end inference process.

References

• Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155.
• Vishwarupe, V., Zahoor, S., Akhter, R., Bhatkar, V. P., Bedekar, M., Pande, M., Joshi, P. M., Patil, A., & Pawar, V. (2023). Designing a human-centered AI-based cognitive learning model for Industry 4.0 applications. In Industry 4.0 Convergence with AI, IoT, Big Data and Cloud Computing: Fundamentals, Challenges and Applications (pp. 84–95). Bentham Science Publishers.
• Dwork, C., & Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3–4), 211–407.
• Sayyed, H., Alwazae, M., & Vishwarupe, V. (2025). BlockSafe: Universal blockchain-based identity management. In B. Alareeni (Ed.), Big Data in Finance: Transforming the Financial Landscape (Vol. 169, pp. 101–118). Springer. https://doi.org/10.1007/978-3-031-80656-8_6
• Vishwarupe, V., Maheshwari, S., Deshmukh, A., Mhaisalkar, S., Joshi, P. M., & Mathias, N. (2022). Bringing humans at the epicentre of artificial intelligence: A confluence of AI, HCI, and human-centered computing. Procedia Computer Science, 204, 914–921. https://doi.org/10.1016/j.procs.2022.08.111
• Rayson Laroca, R., Severo, E., Zanlorensi, L., Oliveira, L., Gonçalves, G., Schwartz, W., & Menotti, D. (2018). A robust real-time automatic license plate recognition based on the YOLO detector. arXiv preprint arXiv:1802.09567.
• Vishwarupe, V., Bedekar, M., Pande, M., & Hiwale, A. (2018). Intelligent Twitter spam detection: A hybrid approach. In X. S. Yang, A. Nagar, & A. Joshi (Eds.), Smart trends in systems, security and sustainability (Vol. 18, pp. 157–167). Springer. https://doi.org/10.1007/978-981-10-6916-1_17
• T. Li, Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50–60.
• Vishwarupe, V., Joshi, P. M., Mathias, N., Maheshwari, S., Mhaisalkar, S., & Pawar, V. (2022). Explainable AI and interpretable machine learning: A case study in perspective. Procedia Computer Science, 204, 869–876. https://doi.org/10.1016/j.procs.2022.08.105
• The Syslog Protocol. (2001/2009). RFC 3164/5424, Internet Engineering Task Force (IETF).
• Wani, K., Khedekar, N., Vishwarupe, V., & Pushyanth, N. (2023). Digital twin and its applications. In Research Trends in Artificial Intelligence: Internet of Things (pp. 120–134). Bentham Science Publishers.
• Xie, C., Koyejo, O., & Gupta, I. (2020). Fall of empires: Breaking Byzantine-tolerant SGD by inner product manipulation. In Proceedings of the International Conference on Machine Learning (ICML).
• Vidgen, B., Harris, A., & Emmery, C. (2021). Challenges and frontiers in abusive content detection. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.
• Vishwarupe, V., Bedekar, M., Pande, M., Bhatkar, V. P., Joshi, P., Zahoor, S., & Kuklani, P. (2022). Comparative analysis of machine learning algorithms for analyzing NASA Kepler mission data. Procedia Computer Science, 204, 945–951. https://doi.org/10.1016/j.procs.2022.08.115
• Blanchard, P., El Mhamdi, E. M., Guerraoui, R., & Stainer, J. (2017). Machine learning with adversaries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems (NeurIPS).
• Vishwarupe, V. (2022, February 10). Synthetic content generation using artificial intelligence. All Things Policy. IVM Podcasts. https://shows.ivmpodcasts.com/show/all-things-policy-Rx64RVpQImivrNQ8/episode/synthetic-content-generation-and-chinas-worries-ja9s-I7rfgZE3IhXRg2Fk
• Kairouz, P., McMahan, H. B., et al. (2021). Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14(1–2), 1–210.
• Sable, N. P., Rathod, V. U., Mahalle, P. N., & Birari, D. R. (2022, March). A multiple stage deep learning model for NID in MANETs. In 2022 International Conference on Emerging Smart Computing and Informatics (ESCI) (pp. 1–6). IEEE.
• Common Event Format (CEF); JSON Logging Standards. ArcSight.
• Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016). Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5), 637–646.
• Zahoor, S., Bedekar, M., Mane, V., & Vishwarupe, V. (2016). Uniqueness in user behavior while using the web. In S. Satapathy, Y. Bhatt, A. Joshi, & D. Mishra (Eds.), Proceedings of the International Congress on Information and Communication Technology (Vol. 438, pp. 229–236). Springer. https://doi.org/10.1007/978-981-10-0767-5_24
• Vishwarupe, V., Bedekar, M., & Zahoor, S. (2015). Zone-specific weather monitoring system using crowdsourcing and telecom infrastructure. In 2015 International Conference on Information Processing (ICIP) (pp. 823–827). IEEE. https://doi.org/10.1109/INFOP.2015.7489495
• Zahoor, S., Bedekar, M., & Vishwarupe, V. (2016). A framework to infer webpage relevancy for a user. In S. Satapathy & S. Das (Eds.), Proceedings of First International Conference on Information and Communication Technology for Intelligent Systems: Volume 1 (Vol. 50, pp. 173–181). Springer. https://doi.org/10.1007/978-3-319-30933-0_16
• Gehman, S., Gururangan, S., Sap, M., et al. (2020). RealToxicityPrompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462.
• Zhang, M., Cao, J., Shen, X., & Cui, Z. (2024). EdgeShard: Efficient LLM inference via collaborative edge computing. arXiv preprint arXiv:2405.14371.
• Deoskar, V., Pande, M., & Vishwarupe, V. (2024). An analytical study for implementing 360-degree M-HRM practices using AI. In Intelligent Systems for Smart Cities: Select Proceedings of the 2nd International Conference, ICISA 2023 (pp. 429–442). Springer Nature.
• Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M., Kolesnikov, A., Duerig, T., & Ferrari, V. (2020). The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale. arXiv preprint arXiv:1811.00982.
• Vishwarupe, V., et al. (2021). A zone-specific weather monitoring system. Australian Patent No. AU2021106275. Australian Government, IP Australia. https://patents.google.com/?inventor=Varad+Vishwarupe
• Vishwarupe, V., Bedekar, M., Joshi, P. M., Pande, M., Pawar, V., & Shingote, P. (2022). Data analytics in the game of cricket: A novel paradigm. Procedia Computer Science, 204, 937–944. https://doi.org/10.1016/j.procs.2022.08.114
• Vishwarupe, V. V., & Joshi, P. M. (2016). Intellert: A novel approach for content-priority based message filtering. In 2016 IEEE Bombay Section Symposium (IBSS) (pp. 1–6). IEEE. https://doi.org/10.1109/IBSS.2016.7940206
• Vishwarupe, V., et al. (2025). Predicting mental health ailments using social media activities and keystroke dynamics with machine learning. In B. Alareeni (Ed.), Big Data in Finance: Transforming the Financial Landscape (Vol. 169, pp. 63–80). Springer. https://doi.org/10.1007/978-3-031-80656-8_4
• Zahoor, S., Akhter, R., Vishwarupe, V., Bedekar, M., Pande, M., Bhatkar, V. P., Joshi, P. M., Pawar, V., Mandora, N., & Kuklani, P. (2023). A comprehensive study of state-of-the-art applications and challenges in IoT and blockchain technologies for Industry 4.0. In Industry 4.0 Convergence with AI, IoT, Big Data and Cloud Computing: Fundamentals, Challenges and Applications (pp. 1–16). Bentham Science Publishers.