Abstract

Software systems require the validation of design features through regression testing. Two primary challenges in system validation are ensuring that test suites reflect actual system usage, and managing the test suite size to keep testing costs low while keeping testing results meaningful. To create a test environment that is close to actual system usage, we propose using Markov chains to create system behavioral models from available system usage data. Knowing that certain factors are not captured in system usage, we will use the Markov Modulated Markov Process to model hidden processes. The models are used to create test plans that employ a unique, likelihood-based, test prioritization scheme. The proposed methodology not only provides a stochastic modeling framework for software systems, but also considerably improves the coverage factor of generated test suites. This paper also presents a real-world web application case study to demonstrate the capabilities of the proposed system validation methodology.

References

• L. Huang and B. Boehm, "How Much Software Quality Investment Is Enough: A Value-Based Approach," Software, IEEE, vol. 23, no. 5, pp. 88 –95, Oct. 2006.
• K. -C. Chiu, J. -W. Ho, and Y. -S. Huang, "Bayesian updating of optimal release time for software systems," Software Quality Control, vol. 17, no. 1, pp. 99–120, Mar. 2009.
• Y. F. Li, M. Xie, and T. N. Goh, "A study of project selection and feature weighting for analogy based software cost estimation," Journal of Systems and Software, vol. 82, no. 2, pp. 241–252, Feb. 2009.
• S.M.K Quadri and Sheikh Umar Farooq,”Software Testing-Goals,Principles and Limitations,” International Journal of Computer Applications, Volume 6-No.9,September 2010.
• S. R. Dalal, A. Jain, N. Karunanithi, J. M. Leaton, C. M. Lott, G. C. Patton, and B. M. Horowitz, "Model-based testing in practice," in Software Engineering, 1999. Proceedings of the 1999 International Conference on, 1999, pp. 285 –294.
• M. Utting, A. Pretschner, and B. Legeard, "A Taxonomy of model-based testing," Apr. 2006.
• A. van Lamsweerde, "Formal specification: a roadmap," in Proceedings of the Conference on The Future of Software Engineering, New York, NY, USA, 2000, pp. 147–159.
• J. Dick and A. Faivre, "Automating the Generation and Sequencing of Test Cases from Model-Based Specifications," in Proceedings of the First International Symposium of Formal Methods Europe on Industrial-Strength Formal Methods, London, UK, UK, 1993, pp. 268–284.
• R. M. Hierons, "Testing from a Z Specification," Software Testing, Verification and Reliability, vol. 7, no. 1, pp. 19–33, Mar. 1997.
• M. R. Blackburn and R. D. Busser, "T-VEC: A tool for developing critical systems," in In Proceedings of the 1996 Annual Conference on Computer Assurance (COMPASS 96, 1996, pp. 237–249.
• S. Zweben, W. Heym, and J. Kimmich, "Systematic Testing of Data Abstractions Based on Software Specifications," Software Testing, Verification, and Reliability, vol. 1, no. 4, pp. 39–55, 1992.
• J. Offutt, S. Liu, A. Abdurazik, and P. Ammann, "Generating test data from state-based specifications," The Journal of Software Testing, Verification and Reliability, vol. 13, pp. 25–53, 2003.
• E. Farchi, A. Hartman, and S. S. Pinter, "Using a model-based test generator to test for standard conformance," IBM Syst. J. , vol. 41, no. 1, pp. 89–110, Jan. 2002.
• H. S. Hong, Y. G. Kim, S. D. Cha, D. H. Bae, and H. Ural, "A test sequence selection method for statecharts," Software Testing, Verification and Reliability, vol. 10, no. 4, pp. 203–227, Dec. 2000.
• A. Pretschner, O. Slotosch, E. Aiglstorfer, and S. Kriebel, "Model-based testing for real," International Journal on Software Tools for Technology Transfer (STTT), vol. 5, pp. 140–157, Mar. 2004.
• M. Chen, X. Qiu, W. Xu, L. Wang, J. Zhao, and X. Li, "UML Activity Diagram-Based Automatic Test Case Generation For Java Programs," The Computer Journal, vol. 52, no. 5, pp. 545 –556, 2009.
• J. Whittaker and M. Thomason, "A Markov Chain Model for Statistical Software Testing," IEEE Trans. Softw. Eng. , vol. 20, no. 10, pp. 812–824, Oct. 1994.
• A. Avritzer and E. J. Weyuker, "The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software," IEEE Trans. Softw. Eng. , vol. 21, no. 9, pp. 705–716, Sep. 1995.
• B. Regnell, "Towards integration of use case modelling and usage-based testing," Journal of Systems and Software, vol. 50, pp. 117–130, Feb. 2000.
• S. Sampath, S. Sprenkle, E. Gibson, L. Pollock, and A. S. Greenwald, "Applying Concept Analysis to User-Session-Based Testing of Web Applications," Software Engineering, IEEE Transactions on, vol. 33, no. 10, pp. 643 –658, Oct. 2007.
• S. Elbaum, A. G. Malishevsky, and G. Rothermel, "Test Case Prioritization: A Family of Empirical Studies," IEEE Trans. Softw. Eng. , vol. 28, no. 2, pp. 159–182, Feb. 2002.
• S. Elbaum, A. G. Malishevsky, and G. Rothermel, "Prioritizing test cases for regression testing," in Proceedings of the International Symposium on Software Testing and Analysis - ISSTA '00, Portland, Oregon, United States, 2000, pp. 102–112.
• W. E. Wong, J. R. Horgan, S. London, and H. Agrawal, "A study of effective regression testing in practice," in PROCEEDINGS The Eighth International Symposium On Software Reliability Engineering, 1997, pp. 264 –274.
• Z. Li, M. Harman, and R. M. Hierons, "Search Algorithms for Regression Test Case Prioritization," IIEEE Trans. Software Eng. , vol. 33, no. 4, pp. 225–237, Apr. 2007.
• L. R. Rabiner, "A tutorial on hidden markov models and selected applications in speech recognition," PROCEEDINGS OF THE IEEE, vol. 77, p. 257–286, 1989.
• Y. Ephraim and W. J. J. Roberts, "An EM Algorithm for Markov Modulated Markov Processes," IEEE Trans. Signal Process. , vol. 57, no. 2, pp. 463–470, Feb. 2009.
• A. Kashyap, W. Roberts, S. Sarkani, and T. Mazzuchi, "A Model Driven approach for System Validation," in IEEE International Systems Conference, Vancouver, BC, Canada, 2012.
• R. Kohavi, C. E. Brodley, B. Frasca, L. Mason, and Z. Zheng, "KDD-Cup 2000 organizers' report: peeling the onion," SIGKDD Explor. Newsl. , vol. 2, no. 2, pp. 86–93, Dec. 2000.
• A. L. Montgomery, S. Li, K. Srinivasan, and J. C. Liechty, "Modeling Online Browsing and Path Analysis Using Clickstream Data," Marketing Science, vol. 23, no. 4, pp. 579–595, 2004.
• S. Elbaum, G. Rothermel, S. Kanduri, and A. G. Malishevsky, "Selecting a Cost-Effective Test Case Prioritization Technique," Software Quality Control, vol. 12, no. 3, pp. 185–210, Sep. 2004.