January 2008

I have been in various discussions in the last week or so on the uniqueness of Product Testing. We burnt quite a lot of hours trying to figure out what it is, and articulating what we arrived at, and taking feedback from people to ensure that it really is unique. 

Product testing is effective only when the customer is brought to the party and this involves bringing the customer’s requirements as a part of the testing. How can this be attempted? Though, we have been doing this for years together, we have never found it necessary to define them in detail. We arrived at a concept called Integrated QA platform.

Integrated QA platform improves the Test Coverage of the product and this is done by the QA team interacting with multiple stakeholders of the product that include Product Manager, Business Analysts, Support Team, Beta Users and Developers. Effectively, this involves possibly all the stakeholders who have the responsibility of bringing the customers to the table in defining what needs to be present in the product.

This Integrated QA platform allows for better test coverage and reduces the reworking costs on the product, due to reduction in post implementation bugs. The fallout of this is advantage to the customers, for it directly reduces the cost of ownership of the product.

Apart from diligence and the coverage in terms of Operating Systems, multiple platforms, multiple languages and requirements specific to the product, the underlying thread in product testing is all about bringing customer to the table.


Let us start this with a few examples:

  1. Starbucks was founded on the insight that coffee is not a product sold in a shop, but rather it is an experience at a third place between home and work where people can gather and relax.
  2. Wal-Mart defied conventional logic by refusing to locate their operations in high traffic neighborhoods and airports.
  3. Apples ipod – the largest number of songs divided by the smallest physical size. People want to carry all their music in an unobtrusive manner

Effectively, what appears obvious need not be the truth. But customer insights are hard to come by with the traditional research methodologies. For instance, I was travelling with a friend of mine, Satish, on his Maruti WagonR, and it was raining pretty heavily and the glasses were rolled up. He is someone who believes in driving, by using the rear view mirror. The rear-view mirror on the right was not visible as there was no wiper on the sides and it really was an insight that Maruti can do with. How do they collect this feedback from a customer; it is through postal feedback and through one of those market research companies. This probably results in a lot of data but not much else.

I don’t believe that lot of insights come out of quantitative market research. Organizations though continue to spend millions on it, generate tons of data, do statistical analysis without producing much or for that matter any customer insight.

In order for better customer insights to come through, organizations need to employ both qualitative and quantitative research together. Qualitative research will generate insights and quantitative research can validate those insights. One cannot go without the other as they both are essential for understanding customers.

You can definitely ensure that you are sitting under a tree that is full of ripe apples; it is most likely that an apple will fall on your head. There is nothing that prevents you from having the same level of insight as Sir Isaac Newton did, when the apple falls on your head.

Software product developers are trying to do more with less and this specifically is one of the reasons for product failures. Sample this, software vendors will increase new product releases by 70 percent and upgrade/version releases by almost 40 percent. But, only 30 percent are expecting to increase their rate of R&D spending. Look at some of the scenarios:

  • Product rollout schedules are tightly linked with marketing deadlines
  • Mistakes are repeated from one project to another
  • No customer feedback has been sought by the developers
  • Resources aligned to one important project is diverted to even more important projects within 6-8 weeks
  • Developers divide their time among numerous projects

This is typical of most products that dont succeed in the market. Unless the software vendors develop a good product development system, they would never build better products.

A good product development system involves all aspects of a product company that include Product Management, Product Engineering, Product Marketing, strategy, communications and human resources as it is about technology and technical issues. Some of the key factors to be considered for a product development system are vision, process, customer requirements, resource planning, appropriate metrics and continuous improvement.

One of the best strategies to improve the product development system is to partner with specialists that have experience of working in multiple products. Global product development has become a necessity and reality for software product companies. It makes a lot of sense for them to partner with specialists and adopt the best practices of a product development system that will make their products a success in the market place.