Why Most Companies Are Stuck in AI Experimentation Mode, and How to Get Unstuck

In this week’s episode of Voices of Search, we spoke with Dr. Fern Halper, founder of the AI Foundations Group and VP and Senior Research Director for Advanced Analytics at TDWI. Fern’s career began at Bell Labs and AT&T, where she helped pioneer early machine learning applications like customer churn prediction long before “AI” was a boardroom buzzword. 

Currently, she researches why organizations struggle to move AI from pilot projects into measurable business value, and her perspective from outside the SEO industry offers marketers a useful outside-in read on what’s really happening at the enterprise level.

The conversation covered why so many organizations remain stuck in experimentation instead of scaling meaningful results, what foundational systems every successful AI initiative depends on, and why original thought may become the most valuable skill in an AI-saturated market.

Key Takeaways From This Episode:

• Access to AI tools isn’t the bottleneck anymore. Operational readiness is.
• Buying AI technology is not the same as being ready to scale it across the business—AI is a capability, not a collection of tools.
• Organizations that see measurable ROI ground their AI applications in enterprise data, governance, and operational controls, not public models used in isolation.
• Data pipelines and infrastructure need dedicated maintenance, or models will quietly degrade as upstream data changes.
• “Human in the loop” is often reduced to a compliance checkbox; leaders need to define what meaningful human oversight actually looks like.

The Real Bottleneck Isn’t Access to AI

Fern’s research point is straightforward: nearly every organization is experimenting with generative AI, but very few have the infrastructure to scale it. “A lot of companies can demonstrate AI in pilot mode,” she said, “but very few are putting it into business processes and governing it effectively.”

The gap isn’t enthusiasm or executive pressure—that part is already there. The gap is data maturity. Organizations reporting real, measurable ROI are the ones grounding their applications in enterprise data, semantic context, governance, and operational controls, rather than relying on public models alone.

Fern’s outlook on this was firm: “Using public AI models can help even if you’re using them privately—it can help to improve individual productivity, but it can’t reliably support business decisions or automate workflows without trusted company data.”

Start With the Business Problem, Not the Tool

Fern described the approach that tends to work best: begin with an actual business problem, not a shiny new capability. If customer churn is the issue, the question becomes whether AI can help solve it, and whether the company has the data to support that.

From there, teams inventory their data, assess its quality and structure, and then start building. Fern recalled her own experience predicting customer churn—structured demographic and billing data was useful, but the real signal was buried in unstructured call center notes, which is exactly the kind of problem generative AI is now well suited to unlock.

That inventory process naturally surfaces the harder questions: Is the data trustworthy? How is it stored? Who manages it? Answering those honestly before building is what allows a pilot to actually reach production instead of stalling out.

Why Fast Building Creates a Maintenance Trap

As teams move faster with AI-assisted development, Fern warns of a predictable trap: an explosion of pilots and prototypes with no plan for who maintains them. Data changes constantly, and models depend on pipelines that need continuous integration, monitoring, and governance.

“If something changes upstream, then the data that’s being fed to these models is not going to be the same data that you’re expecting. That’s going to be a problem.”

She pointed to MLOps as the discipline many organizations underinvest in—the team responsible for testing applications before launch, monitoring them once live, and catching model degradation before it becomes a business problem. Without that layer, organizations end up with a growing pile of unmaintained pilots rather than a smaller number of applications actually delivering value.

Turning Executive FOMO Into Foundational Buy-In

Fern’s framing of governance is deliberately optimistic: it’s an enabler, not a barrier. Organizations that succeed with AI use governance to build confidence in their data, models, and outcomes—establishing policies around data quality, lineage, access, monitoring, and human oversight from the start rather than adding them after deployment.

She described trust as a one- or two-chance proposition. If people stop trusting AI outputs, adoption stalls. If they trust those outputs too much without the right controls, risk increases. 

The organizations getting this right treat trust as the product of leadership alignment, reliable data, the right skills, proper operationalization, and accountable governance working together—not any single fix.

The Human-in-the-Loop Question No One Is Really Answering

One of the more nuanced parts of the conversation focused on how loosely “human in the loop” gets defined inside most organizations. Fern distinguished between a meaningful version—humans working alongside AI to make new discoveries and creative decisions—and a hollow version, where a person is present only to check a compliance box.

“Are they in the loop because they’re monitoring the outputs? Are they the validators, the monitors?” She pointed out.

Fern also raised a longer-term concern tied to that gap: if junior employees are only validating AI output rather than doing the underlying analysis themselves, where does deep expertise come from later? 

It’s a workforce development question more than a technology one, and Fern noted that TDWI is actively researching how organizations should be defining this role going forward.

Why Original Thinking Is the Real Differentiator

Generative AI has made it easy to produce large volumes of content quickly, but Fern is concerned about the long-term effect on thinking itself. Original ideas, she argued, don’t come from the center of a model’s distribution—they come from friction, outliers, contradictions, and lived experience.

“If everyone’s relying on the same models to generate ideas, then competitive advantage is going to be harder to sustain because everyone is just sounding and thinking alike.”

She connected this directly to a growing body of research on cognitive offloading, where reliance on AI reduces critical thinking over time. Her own practice is to deliberately think through a problem before consulting AI, rather than the reverse—a habit she sees as increasingly valuable precisely because it’s becoming rare.

What SEOs and Marketing Leaders Should Do Next

Fern’s closing advice was less about a single tactic and more about a mindset shift: organizations need to get out of experimentation mode. Testing and failing fast is still valuable, but without the foundational work—data quality, governance, operational capability—experiments never convert into business value.

Here’s what she meant by this:

• Get the data foundation in order first. AI is only as trustworthy as the data behind it.
• Build AI literacy programs so teams across the organization, not just technical staff, understand data privacy, security, and appropriate use.
• Treat governance as infrastructure to build in from day one, not a compliance layer added at the end.
• Define what human oversight actually means for each AI use case, rather than defaulting to a vague “we’ll have a human in the loop.”
• Protect space for original, critical thinking rather than letting AI-generated output go unquestioned.

For SEOs specifically, Fern sees a real opportunity for teams that have spent years working adjacent to AI-driven systems like Google’s. They’re well positioned to help guide their organizations through this transition, as long as the conversation moves beyond “what can I do with this tool right now” and into the organizational readiness work that makes any of it scalable.

Closing the Gap Between Experimenting and Scaling

The throughline of this conversation is that AI adoption isn’t really an AI problem—it’s a data, governance, and organizational design problem wearing an AI costume. The companies that pull ahead aren’t the ones running the most pilots at once. They’re the ones willing to do the less glamorous work of building trustworthy data foundations, defining real human oversight, and protecting the kind of original thinking that no model can generate on its own.

As Fern put it, the biggest misconception executives still hold is that buying AI technology is the same thing as being ready to scale it. Closing that gap is the work ahead for the next twelve months, whether you sit in SEO, marketing, or the C-suite.

Voices of Search is a daily SEO and content marketing podcast hosted by Jordan Koene and Tyson Stockton. The show delivers actionable strategies and data-driven insights to help marketers navigate the ever-evolving world of search engine optimization and content marketing. New episodes air weekly, covering everything from technical SEO to AI discovery, featuring industry leaders and practitioners sharing real-world frameworks and proven tactics.

Subscribe to Voices of Search on Apple Podcasts, Spotify, or your favorite podcast platform. Follow Previsible on LinkedIn for updates and subscribe to the VOS YouTube channel for video episodes and clips.