Kill That Project Dead! (Or at Least Rethink It.) Early experimentation separates viable products from ones that looked better on paper. Here are four signs that you should bail.

By Rudy Mutter

Opinions expressed by Entrepreneur contributors are their own.

demaerre | Getty Images

Product development is all about experimentation. Consider Twitter, which announced in late September that it would begin testing 280-character tweets.

Related: 10 Experiments to Test Your Startup Hypothesis

Twitter believes strongly in A/B testing, which is why the company is constantly prototyping, scrapping and revising new ideas. One experiment from 2014, involving a user's ability to add commentary when retweeting someone, almost didn't make it to the implementation stage. Today, it's one of the platform's most-used features.

Whether this latest experiment succeeds is not the point, however. The point is to prototype and test viable ideas, eliminating the ones that flounder, and throwing resources behind the ones that show promise. Twitter even has an "experiment shepherd" on staff who ensures that the team understands the experiment life cycle and follows set procedures.

Although most entrepreneurs recognize that experimentation is important, few recognize just how important. By embracing experimentation, founders can save money, boost sales and build stronger products. Here are the specifics of why experimentation is important.

Experimentation leads to innovation.

Experimenting earlier in the product's life cycle takes only a few weeks, compared to months when the product is mostly finished. It's a better, cheaper alternative than bringing to life every idea your team has.

Running experiments on every idea might sound expensive, but it's not nearly as expensive as building an entire product only to realize in the final stages that the product will fail. According to Harvard Business School professor Clayton Christensen, more than 30,000 consumer products go to market every year, and 80 percent fail. Early prototypes' failures cause a lot less stress than a full-blown product failure (not to mention the lost revenue).

Related: The 7 Steps of Effective Product Development

In advertising and monetization, A/B testing small UX changes helps businesses fine-tune their signup processes. Slow rollouts of changes to hardware and software also provide opportunities for experimentation. The key is to test as early as possible using a close approximation of the final product.

Of course, the purpose of an experiment is to determine whether an idea will work -- meaning many won't. Early experimentation separates viable ideas from ones that looked better on paper. But, to make experimentation valuable, businesses need to recognize the warning signs of a bad experiment and know when to bail or change course.

When to kill an experiment -- factors to help you decide

1. Deadlines and budgets are blown. Whether for digital or physical products, experiments are supposed to be temporary. Ideally, a product stakeholder will follow both budget and timeline and then determine whether the idea is worth pursuing further -- but the unknown element of experimentation can knock plans off track.

When working with the unknown starts inflating the budget or pushing the deadline far beyond initial projections, halt the project. Ask whether the potential outcome justifies the added investment. If not, give it up for good.

My company faced this situation when we were developing a mobile app for our client National Fitness Campaign. We wanted to experiment with Instagram's API to build an integration that would make our client's app more robust. But when we noticed both our deadline and budget cap looming without the accomplishments we'd wanted by then, it became apparent that we needed to change course. We ditched our original idea to find a smarter solution.

2. Data and analytics don't support the hypothesis. Set reporting parameters before beginning an experiment, and check the numbers regularly. If the results of the experiment are discouraging, don't ignore them. Recognize the need to adapt or cancel the project.

Twitter will rely on data to determine whether to keep the new 280-character limit. According to the blog post announcing the experiment, English users of the platform hit the current 140-character limit 9 percent of the time. Whether that group makes use of the extra space, along with many other factors, will determine the experiment's fate.

3. Scope creep derails the initial experiment. Scope creep refers to experiments that grow beyond their initial objectives (it can also refer to times when clients ask for so much extra work, the initial quote no longer covers the cost of labor).

According to the Standish Group's CHAOS Manifesto 2012, 43 percent of projects go over budget, beyond the deadline or beneath the promised results. When scope creep begins, projects can end up costing much more than their initial projections. As ROI shrinks and maintenance costs rise, consider how well the experiment is progressing. Some projects deserve extra investment; others should be cut.

4. Users are pushing back. User testing and interviews are critical components of successful experiments. Negative feedback merits a closer look.

Do users genuinely dislike the experiment, or are they simply afraid of change? Jakob Nielsen recommends using five-person test groups to get an accurate assessment. Fewer than five people could skew results, while more than five could become prohibitively expensive.

Related: When to Selectively Listen to Feedback and Ignore What Users Say

User satisfaction is key to any successful experiment. Customers 2020, a study from Walker, estimates that the customer experience will become more important than the price or product features by 2020. Listen to users, judge the merit of their responses and halt the project if they have too many shared complaints.

Related: The 3 Biggest Roadblocks in Product Development

Experimentation is the heart of innovation, but not every experiment will pan out. The solution isn't to stop experimenting, though; it's to recognize early which ideas are worth pursuing and which should be shut down.

Wavy Line
Rudy Mutter

EVP of Technology and a Founding Partner, Yeti LLC.

Rudy Mutter is the EVP of technology and a founding partner at Yeti LLC, a product-focused development and design studio in San Francisco.

Editor's Pick

A Father Decided to Change When He Was in Prison on His Son's Birthday. Now His Nonprofit Helps Formerly Incarcerated Applicants Land 6-Figure Jobs.
A Teen Turned His Roblox Side Hustle Into a Multimillion-Dollar Company — Now He's Working With Karlie Kloss and Elton John
3 Mundane Tasks You Should Automate to Save Your Brain for the Big Stuff
The Next Time Someone Intimidates You, Here's What You Should Do
5 Ways to Manage Your Mental Health and Regulate Your Nervous System for Sustainable Success

Related Topics

Business News

'I've Got the Bug for Business': See All of Mark Wahlberg's Entrepreneurial Endeavors, From Marky Mark and the Funky Bunch to Wahlburgers

Mark Wahlberg owns businesses in several categories, including entertainment production, apparel, fitness, and nutrition.

Business News

'I Am Just Floored': Woman Discovers She Won $1 Million Lottery Prize While Checking Her Email at Work

Initially, she thought the email was a scam, but went to lottery headquarters and walked away with a six-figure check after taxes.

Business News

South Park Creators Spent 'Infinity Dollars' Renovating Iconic Colorado Restaurant, Set to Reopen Soon

Casa Bonita, a long-time favorite of South Park creators Trey Parker and Matt Stone, went bankrupt during the pandemic. The duo purchased and painstakingly renovated the Mexican spot "like a piece of art," Stone said.

Business News

Amazon Might Soon Be Adding Ads to Prime Video

The company wants to generate more ad revenue for its streaming service.

Growing a Business

Want Startup Advice? Start by Getting Off TikTok

TikTok is better used for cooking demos than startup advice. You'll get a lot more value face-to-face.


How to Identify and Handle Toxic Business Leadership

Toxic business leaders are the danger of modern corporate life. Unfortunately, they may hide behind good reputations. Let's discuss four signs of toxic leadership and strategies to deal with it.