Measuring Innovation

Yesterday I had two conversations about metrics for innovation; what should be measured, how do you measure it, and one question about whether it can or should be measured at all.  As my definition of innovation is doing something new that adds value to the business, I believe innovation should be measured in order to capture that value. So, in the macro sense innovation can, and should, be measured. 

However, I do understand the basis of the question of whether innovation can be measured at all. If you're dealing with a truly disruptive innovation, how can you measure when there are no existing benchmarks?  Ultimately, you may find that innovation cannot be measured by using existing metrics.  It then becomes necessary to develop new metrics as part of the innovation process.  Here are some of my thought processes for measuring whether a disruptive innovation with no existing benchmarks will be a success. I'm being careful to call them thought processes rather than methods because the resulting process is different based on each individual context.

Benchmarks  Typically, improvements to existing products or processes are measured against the existing offering.  A truly new process or product or process cannot be compared directly, and a different context needs to be developed.  I often use scenarios in this situation.  For example, I may show a scenario in which a consumer is engaged in a specific task. I then ask evaluators two questions.  "Describe what is going on in this scenario - what is the person trying to do?" and then "What does XYZ product need to do to help accomplish this task?" After they describe the attributes of a successful offering, I present the options (including the current and competitive offerings) and ask which one is most likely to achieve what they described.  What happens is that the benchmark shifts from a current embodiment to a goal to be achieved.

Consumer involvement  Never underestimate the consumer's bias toward what is familiar.  In the evaluation method described above, consumers are not asked their preference of one product over another.  When you are evaluating a truly disruptive innovation the offering may look so foreign or exotic, that consumers may not readily prefer it. It's necessary to create a context where they are evaluating based on how well a new offering will accomplish the intended criteria.  That said, the offering shouldn't be offensive, but keep in mind that successful innovations often fail initial preference tests.

The numbers  When evaluating a truly disruptive innovation, it's important to quantify the value provided before defining the cost structure.  This is especially helpful when developing an offering that has no existing benchmarks.  In the example above, a second step would be to refine different iterations of the embodiment that performed the best.  An evaluation could then be performed to figure out the perceived value of the new offering. (These evaluations can get very involved, so I won't try to define it in detail here. If people are interested I can show an example in another post.) Once the value is determined, a new cost structure can be defined as part of the development process.

Overall success At the end of the day, we need to know whether our efforts have been successful.  It is a fairly straightforward exercise to measure the holistic success after the new offering has been in the market.  I've often been asked about the success of a specific component, such as the pricing, marketing campaigns, or internal processes.  At this point, I have not found a good (reliable) way to measure the success of any specific component of the process.  I'm hoping that if we can start working with the ideas above with greater frequency and rigor, then the value of specific components will start to emerge.  I did do one project where we were able to quantify some of the consumer perceived elements of the design, so I know it can be done.  It was a fairly involved process, but if the stakes are high in terms of investment or brand equity it is well worth the effort.

I'm interested to know how others view innovation metrics, and whether there are other ways to look at the areas I've mentioned.