Abstract

Software architectural evaluation becomes a familiar practice in software engineering community for developing quality software. Architectural evaluation reduces software development effort and costs, and enhances the quality of the software by verifying the addressability of quality requirements and identifying potential risks and it provides assurance to developers that their chosen architecture will meet both functional and non-functional quality requirements. This paper presents a discussion on different software architectural evaluation methods and techniques and concentrates on summarizing the importance of the different early and late evaluation methods, similarities and difference between them, their applicability, strengths and weaknesses.

References

• G. Abowd, L. Bass, P. Clements, Rick Kazman, L. Northrop, and A. Zaremski. Recommended Best Industrial Practice for Software Architecture Evaluation (CMU/SEI-96-TR-025). Pittsburgh, Pa. : Software Engineering Institute, Carnegie Mellon University, 1996.
• F. Andolfi, F. Aquilani, S. Balsamo, and P. Inverardi. Deriving QNM from MSCs for Performance Evaluation of SA. In the Proceedings on 2nd International Workshop on Software and Performance, pp. 2000
• ANSI/IEEE, "Standard Glossary of Software Engineering Terminology", STD-729-1991, ANSI/IEEE, 1991
• F. Aquilani, S. Balsamo, P. Inverardi. Performance Analysis at the software architecture design level. Technial Report TRSAL- 32, Technical Report SaladinProject.
• M. A. Babar, L. Zhu and R. Jeffery. A Framework for Classifying and Comparing Software Architecture Evaluation Methods. In the Proceedings on Australian Software engineering, pp. 309-318, 2004.
• M. A. Babar and I. Gorton. Comparison of Scenario-Based Software Architecture Evaluation Methods. In the Proceedings on Asia-Pacific Software Engineering Conference, pp. 584-585, 2004.
• S. Balsamo, P. Inverardi and C. Mangano. An approach to performance evaluation of software architectures. In the Proceedings on 2nd International Workshop on Software and Performance, pp. 178-190, 1998
• L. Bass, P. Clements and R. K. Kazman. Software Architecture in Practice. SEI Series in Software Engineering. Addison-Wesley, 1998. ISBN 0-201-19930-0.
• P. Bengtsson and J. Bosch. Scenario Based Software Architecture Reengineering. In the Proceedings of International Conference of Software Reuse, pp. 308-317, 1998.
• P. Bengtsson, J. Bosch. Architecture Level Prediction of Software Maintenance. In the Proceedings on 3rd European Conference on Software Maintenance and Reengineering, pp. 139-147, 1999.
• P. Bengtsson, N. Lassing, J. Bosch, and H. V. Vliet. Architecture-Level Modifiability Analysis. Journal of Systems and Software, vol. 69, 2004.
• J. K. Bergey, M. J. Fisher and L. G. Jones and R. Kazman. Software ArchitectureEvaluation with ATAMSM in the DoD System Acquisition Context. CMU/SEI-99-TN-012. Pittsburg, PA: Software Engieering Institute, Carnegie Mellon University, 1999.
• K. Bergner, A. Rausch, M. Sihling and T. Ternit. DoSAM - Domain-Specific Software Architecture Comparison Model. In the Proceedings of the International Conference on Quality of Software Architectures, pp. 4-20, 2005.
• R. C. Cheung. A user-oriented software reliability model. IEEE Trans. on Software Engineering, vol. 6, pp. 118-125, 1980.
• P. Clements and R. K. Kazman, M. Klein. Evaluating Software Architectures: Methods and Case Studies. Addison-Wesley Professional; 2002. ISBN 0-201-70482X
• V. Cortellessa and R. Mirandola. Deriving a Queueing Network based Performance Model from UML Diagrams. In the Proceedings on 2nd International Workshop on Software and Performance, pp. 58-70, 2000.
• L. Dobrica and E. Niemela. A Survey on Software Architecture Analysis Methods. IEEE Transactions on Software Engineering, vol. 28, no. 7, pp. 638-653, July 2002.
• R. Fiutem , and G. Antoniol. Identifying design-code inconsistencies in object- oriented software: a case study. In the Proceedings of the International Conference on Software Maintenance, pp. 94-102, 1998.
• E. Folmer, J. Gurp and J. Bosch. Software Architecture Analysis of Usability. In the Proceedings on 9th IFIP Working Conference on Engineering Human Computer Interaction and Interactive Systems, pp. 321-339, 2004.
• E. Golden, B. E. John and L. Bass. The value of a usability-supporting architectural pattern in software architecture design: a controlled experiment. In the Proceedings on 27th international conference on Software engineering, pp. 460- 469, 2005.
• T. Kauppi. Performance analysis at the software architectural level. Technical report, ISSN: 14550849, 2003.
• R. Kazman, G. Abowd, and M. Webb. SAAM: A Method for Analyzing the Properties of Software Architectures. In the Proceedings on 16th International Conference on Software Engineering, pp. 81-90, 1994.
• R. Kazman, M. Klein, M. Barbacci, T. Longstaff, H. Lipson, and J. Carriere. The Architecture Tradeoff Analysis Method. In the Proceedings on ICECCS, pp. 68-78, 1998.
• P. A. Keiller, and D. R. Miller. On the Use and the Performance of Software Reliability Growth Models. Software Reliability and Safety, Elsvier, pp. 95-117, 1991.
• M. H. Klein, R. Kazman, L. Bass, J. Carriere, M. Barbacci and H. Lipson. Attribute-Based Architectural Styles. In the Proceedings on First Working IFIP Conference on Software Architecture, pp. 225-243, 1999.
• S. Krishnamurthy and A. P. Mathur. On the estimation of reliability of a software system using reliabilities of its components. In the Proceedings of 8th Int'l Symp. Software Reliability Engineering, pp. 146-155, 1997.
• P. Kubat. Assessing reliability of modular software. Operation Research Letters, 8:35-41, 1989.
• J. C. Laprie. Dependability evaluation of software systems inoperation. IEEE Trans. on Software Engineering, vol. 10(6), pp. 701-714, 1984.
• N. Lassing, D. Rijsenbrij, and H. v. Vliet. On Software Architecture Analysis of Flexibility, Complexity of Changes: Size isn't Everything. In the Proceedings of 2nd Nordic Software Architecture Workshop, 1999.
• M. Lindvall, R. T. Tvedt and P. Costa. An empirically-based process for software architecture evaluation. Empirical Software Engineering 8(1): 83Y108, 2003.
• G. Molter. Integrating SAAM in Domain-Centric and Reuse-based Development Processes. In Proceedings of the 2nd Nordic Workshop on Software Architecture, 1999.
• G. C. Murphy, D. Notkin, and K. Sullivan. Software re°exion models: bridging the gap between source and high-level models. In the Proceedings of the 3rd ACM SIGSOFT symposium on Foundations of software engineering, pp. 18 - 28, 1995.
• L. Prechelt, B. Unger, W. F. Tichy, P. Brssler and L. G. Votta. A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions. IEEE Transactions on Software Engineering, vol. 27, pp. 1134-1144, 2001.
• M. Sefika, A. Sane and R. H. Campbell. Monitoring compliance of a software system with its high level design models. In the Proceedings of the 18th International Conference on Software Engineering (ICSE), pp. 387-397, 1993.
• M. Shooman. Structural models for software reliability prediction. In the Proceedings of 2nd International Conference on Software Engineering, pp. 268-280, 1976.
• C. U. Smith. Performance Engineering of Software Systems. Addison- Wesley, Massachusetts, 570 p. , 1990.
• SoftwareReliability. http://www. ece. cmu. edu/~koopman/des_s99/sw_reliability/#reference
• C. Stoermer,F. Bachmann, C. Verhoef, SACAM: The Software Architecture Comparison Analysis Method, Technical Report, CMU/SEI-2003-TR-006, 2003.
• M. Svahnberg, C. Wohlin, L. Lundberg, and M. Mattsson. A Method for Understanding Quality Attributes in Software Architecture Structures. In Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, 2002.
• B. Tekinerdogan. ASAAM: aspectual software architecture analysis method. In the Proceedings of the Working IEEE/IFIP Conference on Software Architecture (WICSA'04), June 2004, pp. 5-14.
• R. T. Tvedt, M. Lindvall, and P. Costa. A Process for Software Architecture Evaluation using Metrics. In the proceedings of 27th Annual NASA Goddard/IEEE, pp. 191-196, 2002.
• L. G. Williams and C. U. Smith. PASA: A method for the Performance Assessment of Software Architectures. In the Proceedings of the Third International Workshop on Software and Performance (WOSP '02), pp. 179-189, 1990,.
• L. G. Williams and C. U. Smith. Performance Engineering of Software Architectures. In the Proceeding on Workshop Software and Performance, pp. 164 - 177, 1998.
• S. M. Yacoub, and H. Ammar. A methodology for architectural-level reliability risk analysis. IEEE Transactions on Software Engineering 28: 529-547,2002.
• S. Yacoub, B. Cukic, and H. Ammar. Scenario-based reliability analysis of component-based software. In the Proceedings of 10th Int'l Symp. Software Reliability Engineering, pp. 22-31, 1999.
• Paul Clements, Rick Kazman and Mark Klein, Evaluating Software Architectures: Methods and Case Studies, Addison Wesley, 2002.
• "ATAM: Method for architecture evaluation": ATAM - Architecture Trade-off Analysis Method report: http://www. sei. cmu. edu/ata/ata_method. html
• Rick Kazman, Len Bass, Gregory Abowd, and Mike Webb, "SAAM: A Method for Analyzing the Properties Software Architectures," Proceedings of the 16th International Conference on Software Engineering, Sorrento, Italy, May 1994, pp. 81-90. http://www. sei. cmu. edu/ata/publications. html#reports
• "CBAM: Cost Benefit Analysis Method http://www. sei. cmu. edu/ata/products_services/cbam. html
• Thomas J. Dolan, Ph. D. Thesis, "Architecture Assessment of Information-System Families", Department of Technology Management, Eindhoven University of Technology, February 2002.
• P. Shanmugapriya , R. M. Suresh ,A Knowledge Based Approach to Enhance Software Architecture Review Process, International Journal of Information Technology and Knowledge Management July-December 2012, Volume 5, No. 2, pp. 315-318