Testing Methodologies

Despite of several improvements regarding the software engineering methodologies in recent years, errorless programs, even under application of novel methods, are an all-time goal in practice. In achieving a high product quality, test and debugging takes an important role: According to Bododan Korel (Automated Software Test Generation, IEEE Transaction on Software Engineering, 16(8):870-879, August 1990) costs for testing and debugging sum up to 50 percent of a software project's overall costs. Due to this, the development of an efficient, automated test methods is a crucial issue for Austria's information- and communication technology sector.

This sub-project's objective is the development and automation of existing test design and test case generation methodologies.

IDATG (Integrated Design and Automated Test Case Generation) is a test case generation tool, developed by the Siemens PSE, and primarily supports a designer to support a designer in creating design specifications and the tester in preparing and conducting software tests. Therefore, activities such as designing test cases and conducting tests can be fully automated. In particular, this sub-project deals with extending the IDATG tool with novel test case generation techniques.

With respect to foundation of test pattern generation, this project addresses the following objectives:

• analysis of test pattern generation in terms of multi-dimensional equivalence classes
• coverage of boundary regions with a minimal number of test cases
• analysis of test pattern generation with probabilistic approaches such as statistical testing and monkey testing
• testing and diagnosis
• quality assessment for test cases with mutation testing

Furthermore, the developed prototype undergoes an empirical evaluation. In this respect, we expect manyfold benefits:

• In practice, multi-dimensional equivalence classes appear rather often. However,

due to complex computations for computing them, the test effort increases when employing them. The techniques developed in this project will reduce this effort considerably.

• Techniques like statistical testing and monkey testing often can detect faults

that are not detectable with well-known techniques. Clarifying to what extent such techniques can be used complementary to other techniques appears to be realistic goal during this project.

• The CMMI (Capability Maturity Model Integrated) requires a higher stage of maturity w.r.t. test coverage. Mutation testing considerably aids in determining the number of test cases for detecting all customer-relevant faults.
• A tighter integration of test case generation tools, such as IDATG with test management tools like SiTEMPPO.