ROI (return on investment) for investing on Test Process Improvement

I am sure every organization will have project analysis after a major release and followed by a formation of a  task team to implement some set of actions resulted out of project analysis meeting. Some of them which will always appear in the list are
1. Improve unit testing and define some criteria for integration and system testing team to accept build for their testing
2. Automate many of our tests which are used at various stages repeatedly (nightly build test, regression test)
3. Regression test automation % should progressively grow release after release
4. There should be clear production release criteria and risk assessment before release
5. Improve estimation
and so on

Many will finally agree that test process improvement seems to cover majority of the list  in the action items coming after project review.

Let me also share from my experience who will get the advantage investing in test process improvement. You need to answer "YES" or "NO" to following questions. It may look common sense but we ignore these patterns.
1. Are you able to make your last release with exact number of planned test cycles?
2. Can you assure that majority of the tests done by development team is not repeated again in next level of testing?
3. Number of patch releases made after final production release is under control therefore support team and customers are not cribbing?
4. You have a well defined maintenance team which work on customer issues and enhancements and none of your product next version road map plan are impacted due to maintenance release pressure?
5. When you analyse the defect found after release majority of them were not traced to "missing test case" or "incomplete regression test" due to lack of time?

If your answer is "No" to any two of the questions above I feel you must look at formal test process improvement in your organization. Some TIPs on how to go about it.
http://slidesha.re/MCN-Quick-TP-Assess

Like to hear from our software engineering community on their views.

Challenges for colleges to enhance job opportunities for their students

The colleges will have placement cell to ensure their students get into a best job after they complete the degree. They look forward to invite prestigious organizations to come for campus interviews. Organizations apart from campus interviews have tried various other models also to hire fresh talents and I could see some challenges they are facing with speed of acquiring business not going as per plan. There are different perspectives to look at the challenges all players are facing in industry - student’s perspective, college's perspective and organization's perspective. In this blog I am talking about college's perspective challenges.

The college concerned people will be thinking, which organization we should give chance first for campus interview when they approach? The one which was prompt in making reasonable offer and honoring the same last year or one which gave very good offer but failed in adhering to the commitments as in the offer letter last year? Or should we tie-up with some organization who will partner with us to ensure our student’s employment index goes up year on year? Getting prestigious organizations as preferred by students for campus interview also is another BIG challenge. What should college showcase to attract the prestigious organization to visit them for campus interview?

The colleges may look for solutions where students are taken closer to “deployment ready” kind of talent for software projects so that organizations will look at them differently when they interview during campus recruitment. As there is a constant communication from some top industry leaders from India that the academy and industry is not working together and there are gaps in skills when the engineers come out of graduation. The investment required on training engineers from the college taking time and effort for organization, delaying the business acquisition and hence the growth. Therefore college is looking for platform like Sandbox environment where students will learn in parallel along with academics, various skills required by the software industry and apply them on well designed exercises to sharpen their skills and come out confidently to face interviews.

Love to hear different views or comments from the college management community. 

Challenges for students to start their career in software engineering

The students will look forward to the college placement office to organize campus interviews from prestigious organization. Apart from this they may look at options to build skills and competencies which may increase their probability of getting the right job while they are in final year or after completing their course. There are different perspectives to look at the challenges all players are facing in industry - student’s perspective, college's perspective and organization's perspective. In this blog I am talking about student’s perspective.

Not knowing exactly what one should take up as their career will attend the first campus interview and do their best to get selected, with an acceptable offer in hand but no date of reporting mentioned in that! College placement center as a policy will not allow this student to participate in another campus interview organized later. Student will get confused if someone who is less performed academically compared to him/her gets an offer with date of joining also mentioned. Added to this challenge, they also get a letter sometimes from the organization where they have offer letter in hand, that the recruitment plan and training moved to next quarter due to business reasons. Students are really in confused status at that stage. The parents of such students are more worried about what to do when their qualified son/daughter is idle with offer letter in hand but not sure when he/she will start the career.

Parents may look for opportunity to sponsor for some other skill building training so that their probability of getting job increases. Potential skills to enter into software engineering are: Basics knowledge of software development life cycle (SDLC), Process models (like ISO 9001-2008, CMMi, TMMi), Test engineering skills. This will help them to understand the role of software testing in building quality software. From this stage either they can take career in test engineering path or software development path (after learning their choice of programming languages and/or various other software development tools).

Love to hear different views or comments from the student or impacted parents community. 

Challenges for organization to scale their team

I observed a pattern where many organizations will prefer to hire fresh engineers from prestigious college and invest reasonable time and effort in training them on various skills before start realizing contribution to the business from those new hires. Organizations have tried various other models also to hire fresh talents and I could see some challenges they are facing with speed of acquiring business not going as per plan. There are different perspectives to look at the challenges all players are facing in industry - student’s perspective, college perspective and organization perspective. In this blog I am talking about organization perspective challenges.

HR/Talent acquiring division gets requirements of new talents from different business units keeping in mind business plan under discussion. The budget of new talent acquisition is carefully used, HR team visits all top colleges, evaluate students as per organization process and make offer to the best candidates keeping the joining date approximate (subject to change) as requested by business units hiring them. Business situation force HR to move the date further and they do their best to get connected to the potential new hire but they see a risk of losing some of the right talent when they get clarity to bring them on board. Even if everything goes well they see some best students drop out giving admissions they got for higher studies as reasons, or not happy with role which is not their passion (forced test engineer role or forced development engineer role) or region/place of work not as expected by them etc. Oops! All investment to acquire talent and train them is going waste.

The organization must be wondering, how nice if they have a solution where they can get a potential talent pool from different sources just in time (JIT) to start contributing from day 1 of joining to the project. 

Assuming few steps normally happens when you hire fresh talent from campus – talent acquisition team effort, travel and stay, training investment on new hires for minimum three months on various skills from internal academy or external vendor, may cost totally per student approximately rupees 2.5 to 3 lakhs. Assuming a minimum salary plus over heads for three months, minimum training charges investment per day, total hiring cost spent by talent acquisition team divided by number of students hired we may find Rs. 3 lakhs as investment per student before they start contributing in project assigned.. This may go up if you are providing accommodation and food also during training period. From my experience there will be few exits due to job allocation which is not in their line of interest. That is another loss on investment made so far. 

JIT hiring opportunity of engineers well trained and groomed by a professional organization who understands the needs of industry, easily will save Rs. 3 lakhs minimum per student by going through this path. Assuming organization hiring in bulk like 100 in a batch every year will save rupees 300 lakhs!. The software professionals hired in this model are productive from first week after you train them on organization process and domain on which they are going to work.

Love to hear different views or comments from the software community. 

Managing stakeholders - Test Manager perspective

Test manager will be addressing on day-to-day basis questions like:

• Why actual QA effort was more than planned?
• Why are we exceeding from number QA cycles planned in the project plan?
• Why do we need to do so much testing for few changes done to the code?
• Why are we still doing unit level tests at system level tests?
• What can we do to test cases attributes to qualify which helps us to decide on priority, during test execution cycle?
• How many defects logged are not traced to test cases? What are we doing about this?
• What risks were anticipated during test plan? How they were mitigated?

I feel these can be addressed better with focus on following:

• Good test case architecture where all levels of test cases fit and with right set of attributes defined helps test case selection on need basis
• Quality gates or levels of testing better defined with defect types to be uncovered as focus than use same old Unit, Integration and System testing as levels of tests
• Introduce escape tracking at each quality gate
• Define criteria to move testing from one level to another level and improve the definition continuously
• Introduce better tracking report to look at progress aspects, cleanliness aspects, test coverage aspects
• User lessons learnt and fix the gaps for future cycles of testing
• Use defect centered activity breakdown during planning phase

Take a look at http://slidesha.re/HBT6of6

I like to hear any comments or different views from the community

Effective project review

You may be wondering, in spite of we spending so much time on project reviews why some of our projects are failing? The challenge here is, during review which are the aspects you focus and what measurements you have to judge where you stand on each of these aspects. It is recommended apart from progress aspects, watch out for product test coverage and cleanliness aspects.

The challenge here is not only defining right metrics to collect, the place and frequency to collect must also be defined. The phases of measurement journey in the organization are: measurements defined, collection happening in many projects, analysis and feedback done, accuracy improved and we have organization trend or control limits for key metrics. have a look at PPT below

http://slidesha.re/HBT4of6

Test automation challenges

Organization may be having the best test automation practice but test assets are inadequate. Such automation investment is of no use. Management will be wondering on so much money invested- on tools, training, hiring skilled engineers and scripting effort but defect escape to production is still bothering them.

Many parents who wanted their children to be successful in their life, sponsor them for whatever certification courses their son/daughter wanted to do after their graduation. Common mistake which I have observed is, parents sending them for some test automation tool training and get the certification. If you see the data points on how many were successful with that path, it is not that encouraging. Why? What went wrong? Automation of wrong tests may not yield anything as business value to organization. They need to understand what are the right test to automate is before starting on test automation. If that skill is not there difficult fro them to be successful in this path.

Another issue is organization management tracking on what percentage of tests automated than how it was done? They get some shocks like the automated scripts not working on new version of product but effort required to maintain scripts not possible to allocate with release pressure and scripts were dumped as useless assets. No frame work used and no standards in scriting maintenance is complicated.

The tool used for automation will not be supported from vendor from that year!. (Phased out tool), but automated scripts is really reducing test effort, helping in doing better regression tests, helping multi-platform tests etc. Oops! what to do? How can we accelerate porting these automated test solution to another tool?

HBT helps to address some of these challenges. Take a look at the following PPT

http://slidesha.re/HBT5of6