Why the Big Data Swamp Needs to Be Purified to Usher in a Revolution in Lending

When used by commercial lenders, a data swamp can prove to be damaging to entrepreneurs and small-business owners.

learn more about Michael Carter

By Michael Carter


Opinions expressed by Entrepreneur contributors are their own.

In today's modern world, data has become the new gold. Yet, too much data can cause problems of its own, making it increasingly difficult for businesses big and small to understand how to extract insightful and important information from the large stockpiles of data they've obtained. Even more worrying is how this mass of data, when used incorrectly, affects entrepreneurs who are looking to obtain a commercial loan for their business. To better serve business owners and the 28.8 million small businesses that currently operate in the U.S., according to the U.S. Small Business Association, this excess of data needs to be filtered to improve the loan process.

Related: 3 Strategies for Getting Into Lending Shape

In the past, storing large volumes of data was simply cost-prohibitive and impractical. The introduction of distributed cloud-based storage has meant that businesses from every sector are storing all kinds of data (consumer and commercial) -- including when consumers make a transaction, enter a store or even browse the internet -- just because they can. In 2004, I wrote a whitepaper called "The Death of the Data Warehouse" and it would appear that over 10 years later, that has come. This new, enhanced distributed environment has enabled monitoring to occur for every transaction and online information has created murky, burdensome and difficult to maintain bodies of data that for some, are impossible to turn into value.

This vast amount of data has created what is called "data lake": a method of storing data within a storage system or repository in its raw or native format. In other words, think of it as a large pool where companies drop massive amounts of data to be accessed at a later date for analysis. But, data lakes can create mismanaged data sets, turning what was meant to be an easily accessible storage system into a "data swamp" where there is little governance, data is hard to find, hard to use and is consumed (visualized) out of context.

Related: 6 Tips to Navigate New Online Lending Options

When used by commercial lenders, a data swamp can prove to be damaging to entrepreneurs and small-business owners. As Bloomberg notes, traditionally, the data that is used for credit decisions include such items as the applicant's income, employment history and payments records for credit cards and other conventional debts, as well as public information such as bankruptcy filings and foreclosures. Those are the type of factors that go into a FICO or credit score -- the score that dictates whether individuals are able to get a mortgage, auto loan or credit card -- or a rating from one of the major credit bureaus.

Yet today, lenders have started to delve even further into consumer data to assess credit worthiness, including a consumer's email addresses, social media and educational background, raising questions about how this will affect decisions on lending and credit, including privacy concerns and potential discrimination in decision-making. This means that even one misguided social media post can potentially lead to a business being denied a loan, reflecting the arbitrariness of the credit scoring system.

To declutter this plethora of information and create a fair and accurate process for entrepreneurs to receive a commercial loan, technology should be used more effectively to sift through only the most relevant information when assessing a customer. To do this, organizations should explore their existing data lakes and catalog what's inside, creating business rules to validate, match and cleanse their data. Purifying customer data has the power to provide transparency to both borrowers and lenders, giving borrowers more information on the terms other businesses are receiving, while lenders are able to mitigate risk by having a complete view of the businesses' valuation, revenue, capital and collateral.

Related: Busting 5 Myths About Small-Business Lending

At BizEquity, we believe that more innovation through the use of new and better big data technology and analytics will work to help business owners make informed decisions and for commercial lenders to make improved lending judgments. The existing FICO score has proven to be a lagging indicator of real risk, credit and capacity, and incorporating additional data insights into the commercial lending process, like a business valuation of the individual's lifetime value, will only better protect the creditor and provide the business owner further opportunities to develop the business they've worked so hard to create. As data dependency continues to grow, we must continue to drive innovation at a faster pace so that many of our young and upcoming entrepreneurs aren't left behind.

The Holy Grail for lenders is to measure "capacity," that is, the ability of the business or consumer to take on additional credit or debt to grow and create value into the system. Before things like data lakes, and advanced big data insights like business value, this was only a dream. Now, what is needed are a few revolutionaries on the business and consumer credit side to break this dam and drain the swamp.

Related Video: Time for a Bank Loan? What Lenders Are Looking For in Small-Business Owners.

Michael Carter

Founder and CEO of BizEquity

Michael M. Carter is the founder and CEO of BizEquity and the inventor of an online valuation service that's helping the small business economy. BizEquity was recently recognized as one of the top 360 companies in the United States by Entrepreneur Media for two years in a row.

Related Topics

Editor's Pick

Have More Responsibilities at Work, But No Pay Bump? Use This Script to Get the Raise You Deserve.
Black and Asian Founders Face Opposition at All Levels — Here's Why That Has to Change

Invest in Yourself: 10 Things Every Working Woman Should Do This Year

When striving for success, it is easy to forget about your mental and physical health. But without health, you cannot fully succeed. Follow these ten lifestyle strategies for success in your personal and professional life.

Thought Leaders

5 Small Daily Habits Self-Made Millionaires Use to Grow Their Wealth

We've all seen what self-made millionaires look like on TV, but it's a lot more subtle than that. Brian Tracy researched what small daily habits these successful entrepreneurs adopted on their journey from rags to riches.

Growing a Business

How Facing Your Demons Improves Your Business and Your Relationships

Facing the big monsters in your life will give you unshakable confidence in your ability to perform in the future.

Business News

'Don't Go to Mars': Bill Gates Shades Elon Musk's Plans to Spend Money on Mars Missions

The two billionaires have had a longstanding feud after Musk accused Gates of shorting Tesla stocks last spring.