Wipro Technologies

Number of people on the test team: 10

Number of developers supported by the test team: 30

Best Practice Description:

Testing is one of the key phases in the software development life cycle (SDLC). Experience shows that about 25-30% of the total SDLC effort is contributed by the testing phase. As part of adhering to best practices, Wipro encourages the implementation of various tools and techniques in testing projects with the following objectives:

• The testing process is carried out in a systematic way
• The System Under Test (SUT) should have minimum possible residual defects when it is ready for release
• Testing effort is as minimal as possible and should not be at the cost of quality.

As part of Wipro Technologies' Testing Services, we have a dedicated practice that caters to servicing clients in the financial domain by developing solutions that enable significant reduction in effort across the testing lifecycle.

While developing one such solution we applied Wipro OA (Orthogonal Array) tool in the test design and development phases to generate an optimum number of test cases for execution.

Brief background of the solution :

Wipro Technologies' Testing Services' spotted an opportunity to develop a Generic Testing Solution for testing applications that process complex financial messages against defined standard protocols. These standard protocols prescribe a set of messages classified according to a range of financial operations.

These messages consist of a set of elements and sub elements. Each message describes a specific transaction / event. Any particular transaction or event can be described by several probable combinations of these elements and their sub elements. When this is applied across several types of transactions or events, the possible combinations may run into millions.

Testing needs to be completed within stringent timelines and therefore it is impractical to execute all possible combinations. Presently, the combinations are selected intuitively based on the criticality and usage of the application or the past experience of the expert. Since, there was no mathematical or statistical technique involved so the quality of the product could always be questioned. Hence, there was a need to reduce the number of test cases by elimination of redundancies and at the same time ensuring only a minimal impact on coverage.

Wipro used a standardized and statistical technique to optimize this testing solution, which would further help in reduction of efforts from the customer's end in terms of customization and test execution.

Test design :

Due to the complexity of application the test cases were running into millions. Also, the applications under test were so critical for the customer that the coverage could not be compromised considerably. This required a solution which would reduce test case development and execution time ensuring optimum coverage.

We evaluated various Design of Experiments methodologies to achieve this and finally narrowed down to 2 possible methodologies for Test optimization, Orthogonal Array (OA) Technique and Equivalence class partitioning.

Application of Equivalence class partitioning did not result in a significant reduction in the number of test cases when compared to the OA method. Further, some of the elements/ sub elements had some dependencies (both negative and positive) and elimination of such infeasible combinations was possible only through the use of advanced OA techniques. Hence, Wipro's OA tool was found the best fit.

Wipro OA tool :

It is a statistical tool which is based on the premise that interaction integration is the major sources of errors. Also, the errors occur not because of any complex interactions but simple pair-wise interactions.

Wipro OA tool requires the input in the form of factors and levels. This reduction of the entire application in the form of factors and levels is the most crucial step involved and requires a deep understanding of the application. After analyzing the application in terms of factors and levels it generates the optimized set of combinations. These combinations ensure that all the levels are checked against all other levels of all the factors, in other words 100% pair wise coverage is obtained.

The output is taken as a base to generate the test cases, also if required some additional test cases can be added to offer additional coverage to critical applications.

Test execution / development :

As opposed to the manual testing approach where a project manager decides on the test cases as well as the number by his experience and the criticality of an application, the Wipro OA tool gives a scientific and easy solution to the creation of test cases without any bias.

Wipro OA tool is a systematic & statistical method of testing pair-wise combinations of selected factors/variables across their levels. This ensures that all areas of the entire application under test are evaluated at least once.

Applying Wipro OA tool on select pair wise elements/ sub elements not only resulted in an optimal set of combinations with lesser number of test cases, but also increased the number of realistically created test cases.

An illustration of the reduction can be observed from the below observation while applying Wipro OA tool to generate the number of test cases while comparing it with the maximum possible test cases:

• A particular Message Type consisted of 66 elements.
• Seventeen (17) of these elements were critical and each of these had several sub elements ranging between 2 and 11.

Calculation of all possible combinations of these elements and sub-elements (where each combination can be assumed as a test case) derived a huge figure of 319,334,400.

While applying Wipro OA tool in this case, 154 combinations (test cases) were obtained.

Presuming that a significant number of the 319 odd million combinations did not make the business sense to be considered as valid test cases, the Wipro OA tool still resulted in a dramatic reduction in the number of test cases from several thousands to as few as 154.

The drastic reduction in the number of test cases might prompt questioning the authenticity of the tool. However, our careful observation confirmed that all the pair wise combinations were tested thereby ensuring that enough coverage was still being offered. This is a clear pointer to the amount of effort we could save by eliminating the redundancies in the test cases.

Benefits :

The following benefits were realized after applying Wipro OA tool in the development phase of Generic Testing solution

• Approximately 60-70% of effort in writing and executing the test cases was reduced, and at the same time we were confident of the coverage being offered.
• Customers benefit directly in two ways:
• The test case development effort comes down by 30% which is enabled by the Generic Testing solution
• As the Generic Testing solution already comes with an optimized test suite, it reduces the test execution time

Conclusion :

The application of Wipro OA tool helped the Test Execution Team in reducing cycle time of the test phase, finding maximum defects with minimal test cases and offering maximum coverage with minimum test cases. Test Case creation based on Wipro OA tool has increased the confidence levels in the system by executing a concise set of tests and uncovering most of the bugs.

Considering that Wipro OA tool has been adopted in Wipro's Generic Testing solution, it does immensely benefit all the users of the product. Thus, the benefits proliferate and are shared across the software community.