Managing stakeholders - Test Manager perspective

Test manager will be addressing on day-to-day basis questions like:

• Why actual QA effort was more than planned?
• Why are we exceeding from number QA cycles planned in the project plan?
• Why do we need to do so much testing for few changes done to the code?
• Why are we still doing unit level tests at system level tests?
• What can we do to test cases attributes to qualify which helps us to decide on priority, during test execution cycle?
• How many defects logged are not traced to test cases? What are we doing about this?
• What risks were anticipated during test plan? How they were mitigated?

I feel these can be addressed better with focus on following:

• Good test case architecture where all levels of test cases fit and with right set of attributes defined helps test case selection on need basis
• Quality gates or levels of testing better defined with defect types to be uncovered as focus than use same old Unit, Integration and System testing as levels of tests
• Introduce escape tracking at each quality gate
• Define criteria to move testing from one level to another level and improve the definition continuously
• Introduce better tracking report to look at progress aspects, cleanliness aspects, test coverage aspects
• User lessons learnt and fix the gaps for future cycles of testing
• Use defect centered activity breakdown during planning phase

Take a look at http://slidesha.re/HBT6of6

I like to hear any comments or different views from the community

Effective project review

You may be wondering, in spite of we spending so much time on project reviews why some of our projects are failing? The challenge here is, during review which are the aspects you focus and what measurements you have to judge where you stand on each of these aspects. It is recommended apart from progress aspects, watch out for product test coverage and cleanliness aspects.

The challenge here is not only defining right metrics to collect, the place and frequency to collect must also be defined. The phases of measurement journey in the organization are: measurements defined, collection happening in many projects, analysis and feedback done, accuracy improved and we have organization trend or control limits for key metrics. have a look at PPT below

http://slidesha.re/HBT4of6

Test automation challenges

Organization may be having the best test automation practice but test assets are inadequate. Such automation investment is of no use. Management will be wondering on so much money invested- on tools, training, hiring skilled engineers and scripting effort but defect escape to production is still bothering them.

Many parents who wanted their children to be successful in their life, sponsor them for whatever certification courses their son/daughter wanted to do after their graduation. Common mistake which I have observed is, parents sending them for some test automation tool training and get the certification. If you see the data points on how many were successful with that path, it is not that encouraging. Why? What went wrong? Automation of wrong tests may not yield anything as business value to organization. They need to understand what are the right test to automate is before starting on test automation. If that skill is not there difficult fro them to be successful in this path.

Another issue is organization management tracking on what percentage of tests automated than how it was done? They get some shocks like the automated scripts not working on new version of product but effort required to maintain scripts not possible to allocate with release pressure and scripts were dumped as useless assets. No frame work used and no standards in scriting maintenance is complicated.

The tool used for automation will not be supported from vendor from that year!. (Phased out tool), but automated scripts is really reducing test effort, helping in doing better regression tests, helping multi-platform tests etc. Oops! what to do? How can we accelerate porting these automated test solution to another tool?

HBT helps to address some of these challenges. Take a look at the following PPT

http://slidesha.re/HBT5of6

Effective and cost-efficient validation strategy

The standard practice is to divide product testing into different levels like Unit, Integration, System and UAT. There is always a claim in all the organization that they have planned some of these tests and executed. What is surprising is everything in software terminology is evolving with time but these definition for levels of testing remains same even now! But what you can observe is the confusion on definition of an "unit" within the team. Can we look at differently the definition of level of testing which is acceptable to all with no confusion? How about coming up with defect types and define the quality levels where these defects types must be uncovered. HBT recommends 9 levels. This goal-centric approach to testing will also help us to organize test assets better and helps to demonstrate the adequacy of test assets. Overtime it will also help us to prioritize tests for execution in each cycle of testing, select right type of test for automation and determine the right competency required for execution of certain types of tests. All these focus finally leads to effective and cost-efficient validation strategy acceptable to any stakeholders.

Take a look at how personal test methodology HBT helps you here http://slidesha.re/HBT3of6

Intensely goal focused testing

I am sure some of the challenging questions Test Manager will face when things don't go well is - why did these defects escape our team testing? Do we have sufficient test cases to test our product? Why are we spending so much money on re-work and re-testing?

This is where I strongly recommend to re-look at how did we set the goal for testing team in a project. What did we learn from history? Can we predict upfront potential defect types which impacts customer rating on quality of our product? Is there something we can generate as defect catalog for our domain and check the completeness of test assets we have using this catalog as reference? By finding answers to these question you will land up in defining a very clear goal for testing.

It is important to realize that sharper the goal definition for all levels of testing higher is the return on investment (ROI) made in test engineering.

Take a look at how personal test methodology help you here http://slidesha.re/HBT-Solution-2of6

Understanding customer expectation

One of the toughest job is to clearly understand customer expectation. I feel it is a staged approach but getting the entry into the customer's organization is a major break through. This will happen only when customer realize that they are talking to the organization which has systematic way of extracting all their needs step-by-step. Credibility increases when customer see value added by organization in other engagements. During the journey it is important that delivery team delivers whatever they promise and also one should focus on sharing with customer what problems were observed at each key stage of engagement and how it got eliminated or reduced. That is one of the key factor for engagement health to strengthen over time.

In the validation space, starting from understanding customer products & existing test assets to deliver the clean software to production, we realize there are various existing problems one can focus and solve in the organization. Look for some information shared here as 1 of 6 series which I am planning to share.

hbt-solutions10f6

I appreciate comments and look forward to learn from community in the process.

Quick assessment of test process maturity

If you believe that increasing effectiveness of test process in the organization impacts positively on top and bottom line of your business, you must find ways and means to know how is your test process behaving at present? and is there a systematic way to improve the test process?

Here I am sharing some symptoms to quickly assess the test process maturity and if you see consensus among decision makers at your organization to focus on process improvement, I recommend you to look at TMMi framework. The steps involved according to me are:

1. Understand TMMi framework
2. Assess where the current process maturity is
3. Identify gaps to reach a specific level of test process maturity
4. Find appropriate way to fix those gaps effectively

An improved process along with a good test methodology like HBT (coming from STAG) you will see accelerated path to achieve your goal.

http://www.slideshare.net/stagsoft/an-introduction-to-hypothesis-based-testing

http://www.slideshare.net/stagsoft/hbt-a-revolutionary-approach-to-testing-software

http://slidesha.re/MCN-Quick-TP-Assess