Thursday, August 11, 2011

Managing stakeholders - Test Manager perspective

Test manager will be addressing on day-to-day basis questions like:
  • Why actual QA effort was more than planned?
  • Why are we exceeding from number QA cycles planned in the project plan?
  • Why do we need to do so much testing for few changes done to the code?
  • Why are we still doing unit level tests at system level tests?
  • What can we do to test cases attributes to qualify which helps us to decide on priority, during test execution cycle?
  • How many defects logged are not traced to test cases? What are we doing about this?
  • What risks were anticipated during test plan? How they were mitigated?

I feel these can be addressed better with focus on following:
  • Good test case architecture where all levels of test cases fit and with right set of attributes defined helps test case selection on need basis
  • Quality gates or levels of testing better defined with defect types to be uncovered as focus than use same old Unit, Integration and System testing as levels of tests
  • Introduce escape tracking at each quality gate
  • Define criteria to move testing from one level to another level and improve the definition continuously
  • Introduce better tracking report to look at progress aspects, cleanliness aspects, test coverage aspects
  • User lessons learnt and fix the gaps for future cycles of testing
  • Use defect centered activity breakdown during planning phase


I like to hear any comments or different views from the community



Friday, August 5, 2011

Effective project review

You may be wondering, in spite of we spending so much time on project reviews why some of our projects are failing? The challenge here is, during review which are the aspects you focus and what measurements you have to judge where you stand on each of these aspects. It is recommended apart from progress aspects, watch out for product test coverage and cleanliness aspects.

The challenge here is not only defining right metrics to collect, the place and frequency to collect must also be defined. The phases of measurement journey in the organization are: measurements defined, collection happening in many projects, analysis and feedback done, accuracy improved and we have organization trend or control limits for key metrics. have a look at PPT below

Test automation challenges

Organization may be having the best test automation practice but test assets are inadequate. Such automation investment is of no use. Management will be wondering on so much money invested- on tools, training, hiring skilled engineers and scripting effort but defect escape to production is still bothering them.

Many parents who wanted their children to be successful in their life, sponsor them for whatever certification courses their son/daughter wanted to do after their graduation. Common mistake which I have observed is, parents sending them for some test automation tool training and get the certification. If you see the data points on how many were successful with that path, it is not that encouraging. Why? What went wrong? Automation of wrong tests may not yield anything as business value to organization. They need to understand what are the right test to automate is before starting on test automation. If that skill is not there difficult fro them to be successful in this path.

Another issue is organization management tracking on what percentage of tests automated than how it was done? They get some shocks like the automated scripts not working on new version of product but effort required to maintain scripts not possible to allocate with release pressure and scripts were dumped as useless assets. No frame work used and no standards in scriting maintenance is complicated.

The tool used for automation will not be supported from vendor from that year!. (Phased out tool), but automated scripts is really reducing test effort, helping in doing better regression tests, helping multi-platform tests etc. Oops! what to do? How can we accelerate porting these automated test solution to another tool?

HBT helps to address some of these challenges. Take a look at the following PPT