Bots Used to Be a Threat. Now They’re Your Customers.

For the last thirty years, digital marketers have been focused on keeping bots out. The very term “bots” had a negative connotation, and the layers and layers of protection we’ve built are now a liability to brands’ web stacks.

This isn’t because security stopped mattering… It hasn’t. But all of the CAPTCHAs, rate limits, login walls, fingerprinting and internet fortresses we’ve tried to build must be re-examined in the modern context of AI. Brands may be doing themselves a disservice with the default position that automated traffic is hostile until proven otherwise.

My co-founder has been preaching about this for years, and I’ve nodded along for most of them. Only recently have I felt the upside-downness of it, and how much of what we built to protect brands now works against us.

Understanding the scale of the problem

Bots are now 53% of web traffic — that’s the stat everyone repeats on LinkedIn. Although if you double-click, the context is even more important since agents are doing way more than they used to. Often, they don’t just scan sites; they interact and execute workflows, which makes good and bad traffic harder to tell apart.

That is where the problem lies and what we need to solve for — Your defenses can’t tell the difference between someone scraping your catalog nefariously and someone’s assistant trying to buy from it. Same door, same knock. It is hard to distinguish between bots working on behalf of a company, agents working for a consumer, and bad actors (who are still bad actors).​

Brands are starting to adopt new strategies

As agents act more and more on behalf of consumers, websites begin to hold a very different spot in the customer journey. A customer tells their assistant: “Find me trail runners, men’s size 10.5, waterproof, under $150, at my door by Saturday.” That agent lands on your site and starts guessing. It reads your page code, infers what your buttons do, and clicks.

Now think about what happens when it doesn’t work or it can’t get proper access. The agent doesn’t email support or try again tomorrow. It moves to a site that was easier to operate and tells your customer it found the shoes there. No cart, no session, no signal anything happened.

That’s the shift marketers should feel in their chest. Site friction used to cost you conversion rate. Now it costs you the consideration set, silently, before a number lands in your analytics.

Instead of the agent reverse-engineering your site, brands need to adopt new protocols and site structures that are helpful, not defensive of bots.​ The risk is that skipping this means agents may route around you to marketplaces and resellers where the data is easy to get. Your product becomes a row in somebody else’s comparison table. No brand story, no bundle, no reason to pick you over the line below except price and a star.​​

Bots must be considered a first-class audience when optimizing your website and digital experience. Here are a few things that brands need to consider as they evolve their web experiences for both human consumers & bots.

1. Find out what you’re blocking and make it easier for “good bots” to consume your site

Your bot policy likely got set by a vendor default at some point and often nobody has opened it since. There’s a real chance you’re blocking some of your own demand and calling it protection. ​

Example: Plenty of brands still block important buts like ChatGPT, Claude, and Perplexity use to fetch pages when someone asks a question. If those bots can’t reach your site, you can’t be cited when a shopper asks AI what to buy.

2. Make sure that your site structure is laid out well for agents, and not just well styled

Structure beats styling when it comes to agents. Structuring data such as pricing, availability/inventory, FAQs, sizing, shipping windows, return policy, etc. is incredibly important. 

​Example: Having Ratings & Reviews data live a well designed carousel or an image, means they don’t exist to agents.

3. Map your customer journey and make sure the actions agents need to take are well articulated

Start by identifying the key things a customer would want an agent to do on your site and optimize specifically for those.

Example: A footwear brand would want to make sure agents can easily navigate to buy their products, read reviews, and compare to their main competitors. Given these are all steps in journey an agent is likely to take on behalf of a shopper, brands must re-engineer parts of their website to make these steps seamless not just for human shoppers but agents too.

4. Revamp your analytics stack

Often an agent visits your site and reports back to the consumer on a different surface than your website. Ultimately, what matters is whether you showed up, got cited, and got picked within the AI Search interaction between AI and your customer. But it also means your web analytics are not a trustworthy source of truth when agents are the ones visiting it, and then taking the data and experience onto new surfaces not mapped by your web analytics.

​Example: Your organic search traffic may have dropped 35% over the past few quarters, but that does not necessarily mean you have gotten 35% less organic impressions. It’s likely that the impressions have just moved from your website (between you and your shopper), and are now taking place across large language models (between an agent and your shopper). For many brands this means a new set of metrics and analytics are required to understand how bots and AI are changing organic discovery. ​

Conclusion

None of this means the last thirty years were a mistake. The gates were correct for the internet we had. They’re just pointed at the wrong thing now.

It’s also important to acknowledge that evolving for an agentic internet isn’t just an SEO tweak or checklist to work through. It’s architectural, and marketing owns more of it than marketing expects. Nobody from security is going to walk into your Monday standup and tell you the bot rules are costing you customers.