Abstract

A single distributed transaction can touch a dozen services, several databases, an authorization layer, a message bus, and one or two third-party dependencies before it either commits or fails. When it fails, the engineer on call rarely sees the cause directly. They see a symptom—a 500 returned to a caller, a stalled queue, a customer ticket—and then spend the bulk of their time reconstructing the transaction path, deciding which team owns the failing hop, correlating logs across service boundaries, and separating an upstream cause from an internal bug from a downstream outage. This paper presents CARMA, a Context-Aware Root-cause Multi-Agent Analysis framework that decomposes this work across specialized agents: a ticket-understanding agent, a log-retrieval agent, a dependency-classification agent, a payload-analysis agent, a policy-checking agent, and a remediation-recommendation agent, coordinated by an orchestrator. The framework’s central idea is evidence-weighted aggregation: each agent emits scored, citable evidence rather than a verdict, and the orchestrator resolves a root-cause class from the weighted set, which keeps the final answer grounded in retrieved signals rather than free generation. A synthetic benchmark is defined that injects seven representative failure classes—malformed request, authorization mismatch, cache inconsistency, timeout propagation, dependency unavailability, schema drift, and business-rule rejection—into generated transaction graphs, together with an evaluation protocol spanning classification accuracy, mean-time-to-diagnosis, false escalation rate, remediation precision, human-review workload, latency, and cost. All reported numbers are illustrative figures produced by the simulation harness and are intended to characterize the framework’s behavior, not to report production performance.

References

• Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In Int. Conf. on Learning Representations (ICLR).
• Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., and Zhou, D. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems (NeurIPS).
• Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., and Kiela, D. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing Systems (NeurIPS).
• Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. 2023. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation. arXiv:2308.08155.
• Sigelman, B. H., Barroso, L. A., Burrows, M., Stephenson, P., Plakal, M., Beaver, D., Jaspan, S., and Shanbhag, C. 2010. Dapper, a Large-Scale Distributed Systems Tracing Infrastructure. Google Technical Report dapper-2010-1.
• World Wide Web Consortium (W3C). 2021. Trace Context. W3C Recommendation. https://www.w3.org/TR/trace-context/
• Cloud Native Computing Foundation. 2024. OpenTelemetry Specification. https://opentelemetry.io/docs/specs/otel/
• Beyer, B., Jones, C., Petoff, J., and Murphy, N. R. 2016. Site Reliability Engineering: How Google Runs Production Systems. O’Reilly Media.
• International Organization for Standardization. 2011. ISO/IEC 25010:2011 Systems and Software Engineering—SQuaRE—System and Software Quality Models.
• Nygard, M. T. 2018. Release It! Design and Deploy Production-Ready Software, 2nd ed. Pragmatic Bookshelf.
• Chen, P., Qi, Y., Zheng, P., and Hou, D. 2014. CauseInfer: Automatic and Distributed Performance Diagnosis with Hierarchical Causality Graph in Large Distributed Systems. IEEE INFOCOM, 1887–1895.
• Kasikci, B., Schubert, B., Pereira, C., Pokam, G., and Candea, G. 2015. Failure Sketching: A Technique for Automated Root Cause Diagnosis of In-Production Failures. In Proc. 25th Symposium on Operating Systems Principles (SOSP).
• Pearl, J. 2009. Causality: Models, Reasoning, and Inference, 2nd ed. Cambridge University Press.
• Zhou, X., Peng, X., Xie, T., Sun, J., Ji, C., Liu, D., Xiang, Q., and He, C. 2019. Latent Error Prediction and Fault Localization for Microservice Applications by Learning from System Trace Logs. In Proc. ESEC/FSE.
• Notaro, P., Cardoso, J., and Gerndt, M. 2022. A Survey of AIOps for Failure Management in the Era of Large Language Models. ACM Computing Surveys 54, 11s.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems (NeurIPS).
• National Institute of Standards and Technology. 2023. Artificial Intelligence Risk Management Framework (AI RMF 1.0). NIST AI 100-1. https://doi.org/10.6028/NIST.AI.100-1
• Divi, V. R. 2026. Multi-Agent Debate for Software Architecture Governance: A Framework for ADR Generation, Risk Review, and Deployment Readiness. International Journal of Engineering Development and Research (IJEDR) 14, 2, 862–. DOI: 10.56975/ijedr.v14i2.308420.
• Gullapalli, S. 2026. Telemetry Quality Indexing for AI-Ready Observability in Multi-Node Cloud Systems: A Multi-Dimensional Framework for Distributed Infrastructure Diagnosis. International Journal of Engineering Development and Research (IJEDR) 14, 2, 878–. DOI: 10.56975/ijedr.v14i2.308419.