OREANDA-NEWS. April 19, 2016. Imagine a chef who desires to create a new, crowd-pleasing recipe. She makes a first attempt at this, then shares it with colleagues and friends to get their feedback. As a result, she may add or subtract ingredients, alter the cooking time or make a number of other adjustments. She then creates a new version of her recipe and again solicits input from taste-testers, which she uses to inform her next version of the recipe. She repeats this until she and her testers feel that she has perfected the recipe.

This is an example of “iterative product development.” Most CPG companies follow a similar, if more formal, cycle of testing, refining and re-testing when developing their own product ideas. Typically, small teams build concepts, get qualitative or quantitative feedback from consumers, refine concepts, collect another round of consumer feedback, and so on, until they arrive at a “winning” concept.

This technique works well enough, but it suffers from one major drawback: It often produces ideas that are good enough but not the best. Why? Traditional iterative testing takes time and money, so teams must considerably narrow the set of ideas at the front end of the product-development process. There’s no guarantee that the best ideas will be adequately explored, or even considered. In fact, the numbers practically guarantee that they won’t. So the team may well arrive at a successful idea – but overlook a better one in the process.

This constrained exploration translates to potential missed opportunities in market, and leads to a counterintuitive notion: When it comes to ideas, quantity drives quality.

An anecdote in David Bayles’ and Ted Orlands’ book Art & Fear, often cited by tech startups, explains the dangers of exploring a very limited number of options. A ceramics teacher divided his new class into two groups, telling one group that they would be graded solely on the quantity of work produced, and the other that they would be graded solely on quality. On the last day of class, the first group’s work would be weighed, and graded accordingly. The second group would be graded on quality, however much or little work they produced. When grading time came, however, it turned out that the highest quality works all came from the group being graded for quantity. While the quality group agonized over what would constitute first-class work, producing very little in the process, the quantity group went at it, emphasizing productivity – but also, being artists, always trying to improve.

For a similar reason, the greatest artists and writers are generally highly prolific. Picasso, Rembrandt and Cezanne all produced more than 1,000 works (Picasso produced something like 50,000). Bach, Mozart and Beethoven were among the most prolific composers. Artists of great genius but modest output – such as James Joyce and Leonardo Da Vinci – are much rarer than prolific ones.

It’s the same in the more prosaic realm of product development, and no surprise: Most new products are not made by creative geniuses, so talent alone is not enough. The more concepts you test, the more likely you are to identify a concept that will perform well in market. One study estimated that it takes 3,000 raw ideas to identify one substantially new commercially successful product (1).

The product development creative process is also helped along if the facts, ideas, faculties and skills come from a larger group: A recent Nielsen study found that teams of six or more generated concepts that were 58% more preferred than the “starting point” concept in pre-market testing, whereas concepts generated by smaller teams were only 32% more preferred than the “starting point” concept (2).

How to Iterate at Scale

It makes sense, then, that iteration at scale should result in better products. The good news is that technology is now making it possible to develop and test not just a few concepts, but tens or hundreds of thousands of them.

To iterate at scale, you need to dramatically increase the number of people creating concepts without expanding your timeline. Because the success of larger teams is likely due to the greater diversity “in the room,” it’s good if you can increase diversity, whether in terms of the functions involved in the process, or the backgrounds of the people involved, or both.

Technology can help through what are known as “collaborative technology platforms,” which allow remote teams to create concepts more efficiently, making it easier to expand the number of people involved, as well as the diversity of the teams. The normal challenges of distance – emails back and forth with different versions of documents, perhaps requiring a project manager to consolidate input – is obviated by the platform, which allows for real-time feedback that a project manager can accept or reject. Order is also imposed by the project manager, who controls who qualifies as a collaborator, and when to “lock” the creative process.

But technology platforms can only relax the typical constraints in a linear fashion – doubling the number of teams does not quadruple the number of ideas. One striking way in which technology can relax these constraints almost entirely is through the use of evolutionary algorithms.

As used in product development, evolutionary algorithms draw on ideas from biological evolution, such as mutation, combination, and selection to speed the development process. Instead of investing time in building discrete concepts the traditional way, teams seek to identify promising “components” and combine them in different ways to form literally thousands of concepts. As consumers react to the different concepts, the teams learns which variants and combinations of variants score most highly. These are then selected for the next “generation” of concepts. New ideas (mutations) can be introduced at any point in the process.

Where does technology come in?  In the most robust model of evolutionary algorithms, the process is conducted using computers to combine components into concepts, and to respond to feedback, maintaining the favored concepts or components or both at each stage of the process.

One real-world example of evolutionary algorithms is the popular music personalization app Pandora. Though most users are unaware of it, each song in Pandora’s database is composed of a distinct combination of variants — musical characteristics such as the gender of the lead vocalist, the level of distortion on the guitar, the type of background vocals, and so on. (“Each song in the Music Genome Project is analyzed using up to 450 distinct musical characteristics by a trained music analyst.”) As users “thumbs up” or “thumbs down” individual songs, the algorithm learns which elements and combinations of elements the user prefers, and adjusts the songs it offers up accordingly.

In the case of product concepts, the elements are not musical characteristics but different ingredients, flavors, appearance, product claims, and so on. As consumers express a preference for one concept over another, the system learns.

Success by the numbers

It might seem as if it would be much harder to use evolutionary algorithms in the development of physical products than in the development of a music service such as Pandora, or the recommendation engines we see at work every day on Google, Amazon and LinkedIn. It’s obvious that Amazon is learning your tastes. Far fewer people recognize that Google searches are as useful as they are because it is ordering results by its understanding of their relevance to you. And how did you think LinkedIn selected, in no time flat, the half dozen “people you may know” out of a database of hundreds of millions?

But the proof is already available: Our data shows CPG concepts optimized in this way yield, on average, 38% more volume potential. For an industry with high rates of new product failure, these new methods are the next iteration of traditional innovation processes.

Notes

  1. Greg A. Stevens & James Burley (1997) 3,000 Raw Ideas = 1 Commercial Success!, Research-Technology Management, 40:3, 16-27
  2. Nielsen, “How Collaboration Drives Innovation Success,” 2015