⚡ Get All Content for 20% Off ⚡

Kill That Project Dead! (Or at Least Rethink It.) Early experimentation separates viable products from ones that looked better on paper. Here are four signs that you should bail.

By Rudy Mutter

entrepreneur daily

Opinions expressed by Entrepreneur contributors are their own.

demaerre | Getty Images

Product development is all about experimentation. Consider Twitter, which announced in late September that it would begin testing 280-character tweets.

Related: 10 Experiments to Test Your Startup Hypothesis

Twitter believes strongly in A/B testing, which is why the company is constantly prototyping, scrapping and revising new ideas. One experiment from 2014, involving a user's ability to add commentary when retweeting someone, almost didn't make it to the implementation stage. Today, it's one of the platform's most-used features.

Whether this latest experiment succeeds is not the point, however. The point is to prototype and test viable ideas, eliminating the ones that flounder, and throwing resources behind the ones that show promise. Twitter even has an "experiment shepherd" on staff who ensures that the team understands the experiment life cycle and follows set procedures.

Although most entrepreneurs recognize that experimentation is important, few recognize just how important. By embracing experimentation, founders can save money, boost sales and build stronger products. Here are the specifics of why experimentation is important.

Experimentation leads to innovation.

Experimenting earlier in the product's life cycle takes only a few weeks, compared to months when the product is mostly finished. It's a better, cheaper alternative than bringing to life every idea your team has.

Running experiments on every idea might sound expensive, but it's not nearly as expensive as building an entire product only to realize in the final stages that the product will fail. According to Harvard Business School professor Clayton Christensen, more than 30,000 consumer products go to market every year, and 80 percent fail. Early prototypes' failures cause a lot less stress than a full-blown product failure (not to mention the lost revenue).

Related: The 7 Steps of Effective Product Development

In advertising and monetization, A/B testing small UX changes helps businesses fine-tune their signup processes. Slow rollouts of changes to hardware and software also provide opportunities for experimentation. The key is to test as early as possible using a close approximation of the final product.

Of course, the purpose of an experiment is to determine whether an idea will work -- meaning many won't. Early experimentation separates viable ideas from ones that looked better on paper. But, to make experimentation valuable, businesses need to recognize the warning signs of a bad experiment and know when to bail or change course.

When to kill an experiment -- factors to help you decide

1. Deadlines and budgets are blown. Whether for digital or physical products, experiments are supposed to be temporary. Ideally, a product stakeholder will follow both budget and timeline and then determine whether the idea is worth pursuing further -- but the unknown element of experimentation can knock plans off track.

When working with the unknown starts inflating the budget or pushing the deadline far beyond initial projections, halt the project. Ask whether the potential outcome justifies the added investment. If not, give it up for good.

My company faced this situation when we were developing a mobile app for our client National Fitness Campaign. We wanted to experiment with Instagram's API to build an integration that would make our client's app more robust. But when we noticed both our deadline and budget cap looming without the accomplishments we'd wanted by then, it became apparent that we needed to change course. We ditched our original idea to find a smarter solution.

2. Data and analytics don't support the hypothesis. Set reporting parameters before beginning an experiment, and check the numbers regularly. If the results of the experiment are discouraging, don't ignore them. Recognize the need to adapt or cancel the project.

Twitter will rely on data to determine whether to keep the new 280-character limit. According to the blog post announcing the experiment, English users of the platform hit the current 140-character limit 9 percent of the time. Whether that group makes use of the extra space, along with many other factors, will determine the experiment's fate.

3. Scope creep derails the initial experiment. Scope creep refers to experiments that grow beyond their initial objectives (it can also refer to times when clients ask for so much extra work, the initial quote no longer covers the cost of labor).

According to the Standish Group's CHAOS Manifesto 2012, 43 percent of projects go over budget, beyond the deadline or beneath the promised results. When scope creep begins, projects can end up costing much more than their initial projections. As ROI shrinks and maintenance costs rise, consider how well the experiment is progressing. Some projects deserve extra investment; others should be cut.

4. Users are pushing back. User testing and interviews are critical components of successful experiments. Negative feedback merits a closer look.

Do users genuinely dislike the experiment, or are they simply afraid of change? Jakob Nielsen recommends using five-person test groups to get an accurate assessment. Fewer than five people could skew results, while more than five could become prohibitively expensive.

Related: When to Selectively Listen to Feedback and Ignore What Users Say

User satisfaction is key to any successful experiment. Customers 2020, a study from Walker, estimates that the customer experience will become more important than the price or product features by 2020. Listen to users, judge the merit of their responses and halt the project if they have too many shared complaints.

Related: The 3 Biggest Roadblocks in Product Development

Experimentation is the heart of innovation, but not every experiment will pan out. The solution isn't to stop experimenting, though; it's to recognize early which ideas are worth pursuing and which should be shut down.

Rudy Mutter

EVP of Technology and a Founding Partner, Yeti LLC.

Rudy Mutter is the EVP of technology and a founding partner at Yeti LLC, a product-focused development and design studio in San Francisco.

Want to be an Entrepreneur Leadership Network contributor? Apply now to join.

Side Hustle

The Remote Side Hustle a 43-Year-Old Musician Works on for 1 Hour a Day Earns Nearly $3,000 a Month: 'All From the Comfort of Home'

Sam Ziegler wanted to supplement his income as a professional drummer — then his tech skills and desire to help people came together.

Business News

Costco CFO Reveals Uncertain Fate of $1.50 Hot Dog and Soda Combo

CFO Richard Galanti reveals that the price will stay the same — but only "for a while."

Business News

The Most Unexpectedly Popular Side Hustle of the Decade Has Low Startup Costs and High Markups

A new report shows that vending machines are a popular investment — and the industry is set to grow up to $3 billion by 2031.

Marketing

Ever Wonder Why Certain Websites Rank Higher Than Yours? This SEO Expert Reveals The Secret to Dominating Search Results

It's often the smart use of SEO, now supercharged with AI, particularly in keyword optimization.

Business News

AI Is Impacting Jobs. Here Are the Gigs Affected the Most, According to an Analysis of 5 Million Upwork Postings

The researcher said in the report that freelance jobs were analyzed first because that market will likely see AI's immediate impact.

Leadership

Former Interrogator Shares 5 Behaviors Liars Exhibit and How to Handle Them

Five deceptive behaviors to look for and how to respond to those behaviors when you encounter them.