Measuring the black box: how to design and implement innovation metrics.

More than two decades ago, management guru Tom Peters penned an editorial titled "What Gets Measured Gets Done." Indeed, one of the findings from the research that Peters summarized in the 1982 business classic In Search of Excellence is that excellent firms use measurements and metrics to make sure people spend time on the things that really matter.

[ILLUSTRATION OMITTED]

The theory is simple. A senior manager hoping to influence behavior has no stronger lever than his or her choice of measures. Measures serve as tangible guideposts that help middle and junior managers make the critical on-the-ground resource-allocation decisions that--more than any senior management fiat--ultimately determine a company's innovation strategy.

The challenge for companies seeking to improve their ability to create growth through innovation is that the metrics that many companies use to measure innovation run a high risk of actually leading companies in the wrong direction. Managers hoping to unleash their innovative potential need to be mindful of critical measurement traps, think about creating a widespread set of metrics, and ensure their executive dashboard constantly monitors the innovation metrics that matter most.

[ILLUSTRATION OMITTED]

Measurement Traps

Putting metrics on innovation is difficult because innovation is a complicated, diffuse activity. Even a metric that seems to make all the sense in the world can actually lead to behavior antithetical to the long-term pursuit of profitable growth. Generally, there are three measurement traps companies should guard against:

1. Too short a list of metrics. Many companies fixate on a single innovation metric. Some try to calculate the return on their innovation activities. While this metric can be quite useful, on its own it can lead companies to inadvertently prioritize measurable markets over difficult-to-measure but higher potential markets.

No one magic metric measures the right target and aligns incentives appropriately. This is due to the fact that companies that are good at innovation master the ability to introduce different types of innovation. They also recognize that getting good innovation outputs requires tracking the right inputs and the right processes. Having single-minded metrics can lead to prioritizing the wrong things.

2. Encouraging sustaining behavior. Many metrics implicitly--or explicitly--encourage companies to focus on close-to-the-core sustaining innovations that at best promise incremental returns. These incremental innovations aren't bad, but they are insufficient for companies seeking to create substantial growth.

For example, companies that focus on the percentage of revenues from recently launched products have to carefully watch to ensure that they don't subtly encourage very close-to-the-core, low-risk innovations. They can implicitly crowd out the disruptive innovations that transform existing categories and create new ones through simplicity, convenience, accessibility or affordability.

3. Focusing on inputs over outputs. The goal of any company's innovation efforts is creating growth. Companies that only track input-related metrics run the risk of having resources (particularly scientific ones) work on interesting but ultimately low-impact projects.

As a simple example of the limits of focusing on input-related measures, consider a 2006 study that highlighted private sector companies with the largest R & D budgets. Leading the pack in the U.S. was Ford, which, advertising notwithstanding, isn't on anyone's short list of innovative companies.

Remember, there is a marked difference between invention and innovation. In They Made America, Harold Evans describes how Thomas Edison would implore his scientists: "We've got to come up with something. We can't be like those German professors who spend their whole lives studying the fuzz on a bee." Put simply, the output matters.

Winning Metrics

Organizations like the Boston Consulting Group suggest using a balanced mix of metrics to assess a company's innovation-related activities. We agree. The metrics described below come in three categories: input-focused metrics, process-focused metrics and output-focused metrics.

Metrics are rarely one-size-fits-all. The best set of measures varies considerably depending on the company, its values, its industry and its aspirations. For example, as part of a project with the American Press Institute, we developed very specific metrics for newspaper companies seeking to transform themselves. The metrics included industry-specific factors, such as the percentage of revenues deriving from new online advertising models, and factors that tied directly to the inhibitors to innovation we found at many newspaper companies.

Finally, the 15 metrics below are intended to encourage both sustaining innovations that extend the core business and disruptive innovations that build new growth businesses.

Input-Related Measures

* Financial resources dedicated to innovation. Although in isolation this variable can be dangerous, innovation requires real resource commitment. However, apportioning a huge budget to innovation can lead to the "big bet" trap. In fact, limiting funding can be the right thing to do. Scarce resources force teams to zero in quickly on critical assumptions, find cheap ways to test those assumptions and develop lean, flexible structures. So start with "just enough" and add more.

* Resources focused on innovation. This metric ensures that there is dedicated time for innovation activities. In many companies, the really scarce resource isn't money, it's time.

* Ring-fenced resources for non-core innovations. The previous two metrics ensure that the company generally allocates resources to innovation. It's important that some of those resources are specifically focused on non-core innovations, and that those resources are fiercely protected, even when times get bad. Kennametal, a $2.3 billion manufacturer of metal-cutting tools and mining equipment, established a centralized breakthrough technology group to focus solely on long-term innovations. The group evaluates new technologies, new markets and ways the company can bring significant game-changing innovations to existing markets.

* Senior management time invested in new growth innovation. Innovations that are markedly different from core initiatives require careful nurturing from senior leaders. For example, when Procter & Gamble is working on category-creating products, senior executives join in consumer observation sessions, go into the labs to review early prototypes and actively participate in day-long brainstorming sessions.

* Number of patents filed. Again, this measure (or an equivalent for nontechnology companies) can be quite meaningless on its own. But combined with the other metrics, it can be an important interim measure that ensures a constant effort to develop new technology.

Process- and Oversight-Related Metrics

* Process speed. An ideal innovation process quickly moves ideas from conception to critical decision points. That decision point might not always be market launch, it might be a decision to kill or enter a trial market.

* Breadth of idea-generation process. Senior management does not have an exclusive license to develop good ideas. In fact, the best ideas can originate from people who are close to markets, such as sales representatives. A good idea-generation process seeks ideas far and wide--from customers, channel partners, even competitors. Measuring the percent of ideas that come from outside the company is a good proxy for the breadth of the idea-generation process.

* Innovation portfolio balance. A good innovation portfolio is a balanced innovation portfolio. Balance can exist along multiple dimensions, such as the stage of development, the target domain and the amount of risk. Clorox ensures that investments are balanced in diverse areas ranging from introducing line extensions to creating new categories by classifying its projects into three categories (sustaining, breakout and disruptive) and investing accordingly.

* Growth gap. To develop a balanced portfolio, companies should have a good understanding of the gap between their strategic objectives and their current innovation activities. The results must be reasonably risk-adjusted; if success requires that every innovation project meets its current projections, a company should think about developing more (or different) projects.

* Distinct processes, tools and metrics for different types of opportunities. Ideas can look different through different lenses. Tools that help screen and shape core initiatives can unintentionally weed out great--but different--ideas. A company's core stage-gate process can ruthlessly reshape even the most novel idea to resemble what a company has done before. This metric ensures that a company has different screens, tools and metrics for different types of innovation. For example, IBM classifies opportunities by time-to-market and risk-level and applies the appropriate innovation processes to best cultivate the project.

Output-Related Metrics

* Number of new products or services launched. Clearly, a well-oiled innovation machine should produce tangible output. Tracking the number of outputs makes sure the engine is running appropriately.

* Percent of revenues in core categories from new products. As mentioned above, this metric in isolation can unintentionally encourage needless line extensions. But coupled with other metrics, it can ensure that a company has appropriately seized the close-to-the-core opportunities that are critical to growth.