For Subscribers

I Was a Highly-Paid Executive In Customer Experience. Then I Started Working Minimum Wage Jobs, And Realized Everything I'd Gotten Wrong. It wasn't until I started working frontline jobs that I realized the massive disconnect between how CEOs see customer service, and how customers experience it. That showed me how to fix the problem.

By Scott Gilbey Edited by Jason Feifer

This story appears in the July 2024 issue of Entrepreneur. Subscribe »

If you've ever worked a minimum wage job, you've surely watched a lot of training videos. Narrated training videos are a key component of Learning Management Systems, or LMS, and they're how you learn your job.

I am now ashamed to admit this, but I made a lot of those videos. I spent years in the executive suite of an international company, working as a global senior vice president of customer experience, where I directed and produced many training videos. I thought they were smart and helpful. I assumed they increased efficiency.

And then I left my well-paying job and became a frontline worker making $16 an hour at a big-box retailer in Florida. I spent my first two weeks watching LMS videos, which trained me on the art of customer service and how my store functioned, and then I was thrust out into the store itself. That's when I realized: Most of this stuff didn't help me at all. The videos got me "lawyered up" with cautions and "branded up" with soothing marketing messages, but they left me incapable of doing anything useful on the job. All I kept hearing from customers was, Where's my order? I need a refund! What do you mean you don't know?

Related: Why This 3-letter Word Will Solve (Nearly) All Your Customer Service Problems

After 10 weeks, I decided to cut the phrase "In Training" from my name badge. It was starting to feel embarrassing. For how long could I possibly be in training? I showed this to my coworker Jim — a guy I had met while hunched over a shared laptop in the corner of a manager's office, watching yet another LMS video together — but Jim laughed and said I was crazy.

"I'm staying 'in training' forever," he told me. "That way, I don't have to explain to customers why I don't know what I'm doing."

I have a lot of stories like this. Over a period of 30 months, I went on a "frontline immersion" by working at three retail brands across six different stores, plus an airline and a public school district. My goal was to see what I never saw as an executive — where customer experience breaks down, and why customers and employees become frustrated. It was emotionally difficult and often embarrassing. Yet I also came away hopeful. These problems aren't unfixable. We really can make our customer experiences better, for businesses big and small.

But first, we need to see where it all starts to go wrong — on the frontline.

Image Credit: Zohar Lazar

I didn't set out to be a customer experience (CX) executive. Instead, like many people, I started in the service industry.

At age 17, I worked as a black-tie waiter/bartender and then on a cruise ship. This taught me a lot about how good experiences were profitable. The more food and drinks I sold, and the more pleased the customer, the more cash I took home. I never once met a dining customer who said, "Glad I didn't eat or drink or dance."

After a stint in engineering, I migrated into manufacturing sales, fascinated by how other managers worked, shadowing the best of them. One was a purchasing manager named Ron, from whom I ironically learned how to sell. I remember how he'd work:

Ron: "What is your best price?"

Bidder: "$590,000."

Ron: "Is that your best price?"

Bidder: "OK, $570,000."

Ron: "Is that your best price?"

Bidder: "Ron, you're killing me. Will $530,000 get me the order?"

Ron: "Is that your best price?"

Bidder: "My boss is going to fire me. We have no margin left. $520,000."

There were initially seven bidders. One by one, they dropped out, deflated, defeated, until only one was left standing. Ron never once said, "Your price is high." He never suggested, "Can you lower your price?" He only asked, "Is that your best price?"

Years later, I would understand this to be a key component of closing sales and building customer experiences, and employee experiences too. It's all about asking good questions. Would you recommend it? What could be better? The key is to listen to what is being asked and what is being said — or not said.

Listening is the greatest skill you can develop and use. In theory, my peers in CX know this. As I reflect on my career, I realize I could have practiced better listening in many arenas — engineering, sales, marketing, data, and general management. I became certified by the Customer Experience Professionals Association (CXPA), which involves taking a test in customer insights and understanding, customer experience strategy, metrics, measurements, and return on investment (ROI). All of which is to say: It's a test about listening. How well can you find and process feedback?

I wound up working at a global industrial service and equipment company, rising in the ranks to senior vice president of customer experience and service offering. In early 2020, the company went through cutbacks and a change in direction. I left. It was an emotional, perhaps rash, decision. I thought there'd be plenty of work out there for someone like me — or maybe I'd start a consultancy. Then the pandemic froze me in place. I couldn't find work. I couldn't find clients. I kept busy with DIY projects at home, like remodeling the bathroom.

Then, one day, I was talking to a friend in the CX world, and he said something that stopped me cold: "You love customer experience. You also keep talking about your DIY home renovation projects. Why don't you combine those interests somehow? Go get a job at a hardware store."

Out of sheer curiosity, I applied for a job doing customer service. Not as an executive, mind you. I mean an on-the-floor job, paying $16 an hour, wearing an apron at a big-box retailer in Naples, Florida. I applied on Thursday. The following Tuesday, I was on the schedule as an hourly employee.

That's when my education in customer experience really began.

Related: Empower Your Customers and Exceed Expectations With These 3 Customer Service Strategies


Eighty percent of CEOs say they deliver a great customer experience. Only 8% of customers agree.

That's what Bain & Company found in 2005, in a study that's famous among customer experience professionals. The study called this the Delivery Gap (a.k.a. Experience Gap): Executives often overestimated the quality of service their companies were delivering, while customers themselves were often dissatisfied with the actual service they received.

As a CX executive, I knew about this. Similar findings have been published over the years — and while the numbers may change, the skew never does. Overwhelmingly and consistently, company leaders tend to think more of themselves than do customers. And when I was a company leader, I really, truly did think we were the exception.

As I stepped into my first day at the national retailer, I started to think: The executives here probably think that they are the exception too. Now I get to see the reality.

I quickly encountered unexpected problems, starting with the onboarding process. I wondered if these problems were unique to this one retailer, so I took a job at another and then another — and experienced the same things! Then I wondered if these problems were specific to retail, so I took a customer-service job at an airline, and then as a guest teacher with a local public school. Once again, the same problems emerged. I was onto something.

I won't name these organizations, because I don't want to be viewed as criticizing them. Aside from the local school, these are national companies you've shopped and flown with. They are admired leaders in their respective fields. I love them all and have learned a lot from them. Each has a strong mission, fantastic people, superb products, excellent service, and top financial or operational results.

But as I'd find out, they are not immune to the difficult reality of customer service, especially as the bar of CX continues to rise. While already great, each is looking for ways to improve.

In the rest of this article, I'm going to share stories from my frontline immersion. I won't attribute individual experiences to specific organizations, and I want to stress: I do not believe these problems are unique to these institutions. These are problems that everyone struggles with, which is why it's so important to recognize and take advantage of a new perspective.

What I learned was this: The Experience Gap is not a singular thing. It is really the combination of three more specific gaps — a Competence Gap, a Supervision Gap, and a Data Gap. To solve the big problem, we must look closely at the smaller ones.

Related: 4 Ways to Provide Excellent Customer Service

The Competence Gap

I have been onboarded five times now — by four great companies, and one great school board. The onboarding experience is big on compliance, marketing, and socialization. Each brand instills pride in the new employee. I meet nice people. I hear stories. I end my first day giddy with satisfaction and excitement. The emotion is real.

And then the work begins.

As a frontline worker, I want to be competent, and I want to be seen as competent. So do my peers. According to a Washington Post/Ipsos poll from 2023, most workers (61%) say they try to excel in their jobs; 33% say they do their job well but don't go beyond what they're paid to do; and only 4% say they're doing just enough to get by.

So the question is: How do we become competent? This is where the Experience Gap begins.

In my time at the frontline, I saw onboarding happen in four phases:

Phase 1: Videos

And more videos! These are the LMS training videos I described earlier. I watch hour upon hour of these things. Some are long. Some are snippets. Few are easy to remember. Part of the problem is simply volume; I'm receiving so much information, all at once, that it's impossible to process it all. Another part of the problem is context; learning is low because I don't yet understand the basic context and vocabulary these videos are using.

There is typically no secondary way to learn. I have no other avenues for reviewing or following up on information. "Just watch it," one supervisor said when I asked for clarification.

Ultimately, these videos are efficient for sharing mass information — which is why, as an executive, I thought they were a good idea. However, practically speaking, unless these videos are paired with other learning actions during training, video learning is low and fades quickly.

Phase 2: Checklists

Every company has checklists. They are essential. As an executive, I loved checklists: They ensure that procedures are followed, and that small things aren't overlooked.

But in practice on the frontline, checklists are not a failsafe. They feel like one more thing to do — a very literal box to check, just to make corporate happy. I rarely saw them used as designed or intended. I asked one supervisor to review my checklist and the response was essentially, "Don't waste my time." At another job, I attempted to book a 90-day review with a manager but could not because the activity had already (unbeknownst to me) been marked as completed.

Phase 3: Ad hoc tribal knowledge

What happens if I can't consult a checklist? The answer is: I figure it out while a customer waits.

This is how I learn to do the critical tasks of my job. Whether ordering a piece of 2-by-4 lumber, searching for lost luggage, or providing substitute lessons in school, there is zero practice before I interact with customers — zero. I simply jump in and try my best. Over time, the tasks gradually become familiar. And that's the thing: They're just familiar. Competence is something else.

As a fellow frontliner, Sondi, says to me: "No one trains me on nothing. I just use my brain."

Phase 4: The honeymoon period

During my first 10 or so days, I am considered fragile and new. Next is a period of 30 to 90 days when I am simply new. One day I am suddenly expected to know and do everything, whether my capability has been validated or not.

I managed this honeymoon phase. Unfortunately, I saw others — both young and seasoned — struggle and talk of quitting even while their onboarding was underway.

Now, let's review: I have come to do a job. While I want to feel competent in that job, the path to competence is agonizingly slow and filled with distractions. Sometimes I am showered with fake empathy. Supervisors assume I need to be motivated, but what I really want is to identify the 10 to 20 specific, recurring, and critical tasks I need to perform each day. I want to be good at those tasks. Otherwise, my interactions with customers can be ad hoc and awkward, which is frustrating for the customer and embarrassing for me. Watching videos doesn't cut it.

And that's not even the worst part. If I get through all of this, and I actually do achieve competence, then I discover something horrible:

I am not required to be competent!

This is my biggest takeaway from my 30 months on the frontline. The key word is "required." I am not required to be competent. The company may expect me to become competent, and I might even become competent. My supervisors, however, are entirely focused on my being compliant and obedient. Whether I am competent at my job does not seem to matter.

"No way!" you say. "Yes way," I answer. In each of my five frontline engagements, without exception, what I do during the day with customers is rarely acknowledged or questioned. What matters is when I clock in, when I clock out, and where I stand or otherwise position myself during my shift.

Here's a classic example: One day, I am speaking with a customer building a $5,400 order. Out of the corner of my eye, I see my one-up supervisor say something to my direct supervisor, who approaches me, leans into my personal space, and says, "Scott, you have to leave!" It was 5:07 p.m. My shift was to 5:00 p.m. and my name would now appear the next day on the "overtime offenders" list. That's what I'm talking about: My compliance with the time-clock rules was more important than my competence to secure a $5,400 sale.

Many companies will say their onboarding is strategic and well-thought-out. Please think again! Most companies are rightly concerned about employee experience. At the same time, widely published data show an alarming trend of low employee engagement and an increasing Experience Gap between boardroom strategy and frontline reality.

The gap begins on day one with onboarding. I was there and I saw it. It can be different. It must be better.

Related: 10 Reasons Why Good Customer Service Is Your Most Important Metric

The Supervision Gap

All day, every day, I run into unique customer challenges. Someone will ask me something, and my response will be, "I'm sorry, I don't know."

To whom do you think I turn?

If I need rules, I ask headquarters (via online search). If I need permission, I ask a supervisor (by radio if I have one). If I need help, I ask a peer (standing next to or near me).

My peers, I quickly realize, are the real training resource at every company. My supervisors are often unsympathetic to my needs; they're here to supervise me, not to develop me. Frontline workers therefore create an unofficial peer-to-peer training system — and this is how I learn the day-to-day details of my job, the keystrokes, how to avoid trouble, and how to be successful.

Is this a problem? In theory, no: Peers should help each other! But this ad hoc training slows everyone down. It also allows misinformation to spread; if one of my peers only half-knows how to do something, I will only half-learn it. And it creates an us-versus-them mentality, where we believe our success comes from sticking together, not from rising through proper channels.

As I see this evolve, I keep wondering: What's wrong with the supervisors?

Then I realize the answer: They're who I would become.

My supervisors tend to be good people. Good companies attract good people. However, being a good person does not automatically make one a good supervisor. Supervisors are frontliners too, with different job roles. They've learned their tasks and assumed behaviors on the fly, just as I'm doing. Their peers helped them when they were at my level. Now they expect my peers to help me. It's not their job. This is an oral tradition, superseding whatever organizational structure the company has created.

It leaves me with a question: What is the purpose of frontline supervision in the organizational layer? Imagine if, instead of expending time and energy on compliance and obedience, your frontline supervision was single-mindedly focused on helping the frontline be competent, and to help each other serve customers. "Servant leadership" comes to mind. The Experience Gap would shrink.

Related: 4 Reasons Why Your Customer Service Is About to Get a Whole Lot Better in 2024

The Data Gap

In my prior life as senior vice president, I would immerse myself in data — real-time machine data, supply chain data (where is that 2-by-4?), financials, customer master data, access and login rights, GDPR regulations, and much more.

Data can set you free. As an executive, I wanted to know things like: If we increase this here, does it improve the business over there?

Questions like those cascade down the ladder. If an executive wants some metric to increase, then many people must now work to make that happen. Honestly, I never thought about how disruptive that is...until I worked the frontline.

Here's an example: A customer asks me about a branded advertisement offering 50% off, and as usual, I don't know the answer. No peer recognizes the promotion, so I ask a supervisor. To my confusion, the supervisor asks, "Has the customer enrolled in our loyalty program?"

Huh? That doesn't address the customer's question.

"Does the customer have a store-branded credit card?" the supervisor continues.

Again, I tell her I don't understand.

"Last week we had only 23 new credit cards," she says. "We need more this week."

Now I have a different task. My customer, who wanted an answer about the sale, is further from my mind. "How many more credit cards do we need?" I ask.

"Just get more!" she curtly demands.

Why did this happen? Probably because the supervisor had recently attended the weekly store manager meeting, and the low number of credit applications was high on the agenda, and therefore top of her mind. Now it was top of my mind.

My frontline colleagues and I are awash in data. Everywhere we look, there is a new number to hit, a new metric to achieve, a new goal to count. It is rarely contextualized. I don't know why we're interested in that number, or how long we're supposed to care about it, or what happens when we reach it. I don't know if it's my responsibility, or our collective responsibility, or really nobody's responsibility. And this information rarely helps me navigate the bulk of my daily tasks. I am awash in seemingly useless information, and short on the information I actually need, such as my log-on credentials, on-hand inventory, and customer identification.

This is the Data Gap at the frontline. It is the gap between the truckloads of information (alphanumeric and otherwise) gushing from headquarters and the cupful I need right now. Every corporate data-related decision or proclamation will ripple outward in real-world ways, like a butterfly flapping its wings in California and creating a tsunami across the Pacific.

Related: Are the Days of Good Customer Service Over? These 7 Hacks Can Bring it to Life at Your Company

Image Credit: Zohar Lazar

So, how do we fix this?

As a CX executive, I thought I understood what was happening on the frontline. Now I realize that I mostly heard about two things — the extremely good stuff that makes executives happy, and the extremely bad stuff that hits social media.

In reality, the exceptional and the terrible together might account for 5% of all customer experiences. The rest occurs in what I call the Boring Middle — the everyday customer interactions that keep a business afloat, with all its accompanying boredom, sometimes confusion, and routine interactions and transactions.

Now that I've recently lived at the frontline myself, I know: Here lies the secret to breaking the mystery of the Experience Gap. Executives must turn their attention to fixing the Boring Middle.

How? Let's start by closing those three gaps.

The Competence Gap: Frontline employees often embark on their roles armed with theoretical knowledge from LMS videos, and devoid of practical experience. This results in employees struggling to perform basic tasks, leading to customer dissatisfaction and employee frustration. The prevailing approach to training — with a focus on participation rather than skill acquisition — exacerbates this issue. To bridge this gap, companies must prioritize hands-on training and real-world practice, ensuring that frontline staff are equipped with the necessary skills to excel in their roles. AI-driven simulators, for example, could speed the path to competence.

The Supervision Gap: While frontline employees crave guidance and support from their employer, we often encounter a lack of effective local leadership. Supervisors, burdened with compliance responsibilities and performance metrics, prioritize tasks over mentorship. As a result, frontline workers turn to their peers for assistance, bypassing the supervision layer. To address this gap, companies must redefine the role of supervision, emphasizing mentorship and skill development rather than blindly focusing on compliance and obedience. By empowering supervisors to act as "servant leaders," companies can foster a culture of continuous learning and improvement at the frontline.

The Data Gap: In today's data-driven landscape, frontline employees are inundated with information, much of which is irrelevant to their immediate tasks. This "data deluge" distracts employees from their core responsibilities and hampers decision-making. Moreover, frontline staff often lack access to critical data that could enhance customer interactions. If you don't have a data strategy, start there! Tackle the Data Gap head-on and provide frontline employees with specific and relevant data tailored to their roles and the tasks at hand.


Doing this won't be easy. It requires a shift in mindset, prioritizing employee skills and empowerment. By investing in hands-on training, fostering supportive supervision, and streamlining data delivery, companies can transform their frontline operations to deliver consistently exceptional customer experiences in an increasingly competitive landscape.

I sure would have appreciated that during my time on the frontline.

Related: This 4-Step Secret is Key to Exceptional Customer Service — And it Requires A Lot More Than Just Smiles


My final task, after working those five jobs across 30 months, was to be a substitute teacher for a three-day stretch. Seven class periods per day, each a 49-minute duration, and each with 24 to 33 students, in grades 9 to 11.

I arrived home wondering whether I had mattered to those students. Would I have loved to make an inspirational impact on their lives? Absolutely! In reality, however, I was a temporary blip to them. My job was to ensure a continuity of service, not to insert or create anything new myself. Success meant getting the students (my customers) through the next 49 minutes, on task with assignments completed.

Then I realized something else: "On task with assignments completed" is a great definition of experience success generally — and organizations should embrace such focus rather than constantly chasing the ethereal moonshot.

Together, collectively as an organization, we want to create long-lasting and wonderful experiences for people. But to get there, we need to truly embrace the short-term, seemingly small things. Good teachers tend to model this very well; they deliver small wins daily. We CX people also need to become very good at the immediacy of now, which means appreciating the severe importance of specific moments and specific transactions. We need to look at our frontline workers and say: I will help you be great right now, so you can help customers be happy today and tomorrow.

And we need to mean it.

Related: This Is the Biggest Customer Service Mistake You're Making — And How To Fix It Fast

Want to be an Entrepreneur Leadership Network contributor? Apply now to join.

Buying / Investing in Business

Former Zillow Execs Target $1.3T Market

Co-ownership is creating big opportunities for entrepreneurs.

Side Hustle

She Quit Her Job at Trader Joe's After Starting a Side Hustle With $800 — Then She and Her Brother Grew the Business to $20 Million

Jaime Holm and Matt Hannula teamed up to build a business in an industry that "didn't exist" yet.

Business Ideas

70 Small Business Ideas to Start in 2025

We put together a list of the best, most profitable small business ideas for entrepreneurs to pursue in 2025.

Business News

Microsoft Is Laying Off Over 6,000 Employees, About 3% of Its Workforce. Here's Why.

The company said the cuts will affect all divisions and locations, with a focus on managers.

Business News

Klarna Is Hiring Customer Service Agents After AI Couldn't Cut It on Calls, According to the Company's CEO

Klarna released an AI chatbot and implemented an AI-induced hiring freeze last year.

Business News

Nissan Is Doubling Its Initial Layoff Announcement, Cutting 20,000 Jobs: 'A Wake-Up Call'

The automaker is dealing with slowing sales and a recent failed merger with Honda.