Word and Brown

Number of people on the test team: 15

Number of developers supported by the test team: 70

Best Practice Description : Software Testing Best Practices Award Application from Word and Brown

As a growing QA team, we have drawn from several sources to enhance our skills in the areas of requirements analysis, test design, test execution and test management.

We have hired top notch QA Analysts who possess skills and attributes of professional testers. We have attended seminars, conferences, training classes and local QA chapter meetings. We have met as a team and shared the knowledge we have learned from these venues to allow the whole team to gain from what was taught. And most importantly, we have support from the top – our CIO fully supports the QA efforts for growth and improvement through Best Practice education and implementation.

In the area of requirements analysis, we have become more involved in the SDLC by attending, participating and actively reviewing the Business Cases and the Business Requirements documents. This has helped us to understand the project and product earlier and to be active participants in the project planning. As Barbara Patterson taught in IIST Software Inspections and Reviews for QA Professionals, we learned Best Practices of how to identify the basic steps of inspections, and then we implemented the inspection process in the QA department. We then shared our information with the project team to help to identify areas of ambiguity in project documentation prior to the start of coding. Her teaching also helped us to avoid common pitfalls such as aligning our strategy with the IT strategy, dealing with personalities, the lack of standards and training, and understanding the maturity of our SDLC.

Building on the knowledge we gained through the requirements analysis we then apply the product knowledge to our test design process. We have improved our test design process based on information gained in the IIST T2: Managing the Test Project by Robin Goldsmith. We have started utilizing Work Breakdown Structures to help us define and refine our major areas to test. This helps us to identify the big picture and to drill down to the details needed for test design. We utilize out Test Plan to proactively identify risks, test environment needs, training needs, resource needs and constraints and to prioritize what we plan to test and in what order to test. From this we move into our test case documentation. We combine simple and complex test cases into a Test Case Repository (TCR) which will help us to enhance the code delivery schedule we have learned about during various planning sessions. The TCR is also useful in identifying our progress, test execution progress and defect metrics.

As we move into test execution we already have a well defined plan of expected steps and results based on our requirements analysis and test design processes. We have identified those functional areas that can be tested in sequence or concurrently to best utilize our resource allocations. As we learned from Dr. Magdy (in his presentation to the SCQAA-Orange County Chapter) testing is more of a science than an art. Throughout the test execution phase we track test cycles, defect counts by severity, pass/fail rate and test execution rate. Sometimes we do not finish execution based on a block or a non-functioning area of code. We work with the developers to identify the blocks, the steps to reproduce it and the data requirements. We reuse our existing test cases when we are executing regression testing, thereby saving time and validating the already delivered code is still stable.

Our test management comes into play when we monitor the test milestones already identified in our Test Plan and track our progress, all the while reporting it back to the project team to help them in their planning tasks. As our execution metrics are developed from our TCR, we develop graphs to help in the analysis and see trends that help us make project decisions based on facts, not feelings. When adjustments are needed, we look back at our prioritization of the test cases and use this to help us maximize our test effort. We utilize a requirements traceability matrix or a burndown chart to be sure we have delivered, tested and certified all functionality that was expected to be delivered.

We utilize simple process improvement sessions at the end of each project. We utilize the Plan-Do-Check-Act model introduced in the IIST T6: Software Quality Assurance Methods and Techniques by Dr. Rebecca Staton-Reinstein. This Best Practice model is also enforced in our studies of the Quality Assurance Institute QA Body of Knowledge, chapter 2. We utilize root-cause analysis to help identify areas that negatively contribute to the Cost of Quality as we examine the costs associated with Production, Prevention, Appraisal and Failure. We have had several exercises where we define the current process steps then improve on the process by seeing gaps, areas to improve, or areas of duplication.

As we look to our future, we are building an automated regression set of suites for the 14 business applications we support. Our test automation strategy is aligned with the Best Practices we learned from Marnie Hutchison in F6: Test Automation Strategies and Architectures. She taught us that the automation time is spent on setup, execution, analysis and maintenance. Therefore we developed a strategy to develop Smoke tests first, and then concentrate on the most important functions while choosing the tests that are easiest to automate and which will be run the most often. We want to build on our success, so we stop and evaluate progress after each test suite is completed and take time to celebrate our successes. Marnie stated data prep pays the automation bills, so we have implemented a component based framework which will allow us to identify, store and reuse our test data in our automation tool. She also taught us to wait until the application is stable before attempting to add test case automation. Therefore, we look at our test management reports to identify when an application is stable enough for us to move in.

The team is excited about pursuing test certifications. We learned about the CSTP, CSTE and CTM certifications while attending the SCCTC in Long Beach this summer where a representative from IIST discussed the benefits of test certifications.

As the Word and Brown QA team has progressed and improved, we have benefited greatly from the Best Practices insight we have gleamed from several sources to enhance our skills in the areas of requirements analysis, test design, test execution and test management.