Solving the data readiness conundrum

This is a guest resource from Patrick Vinton, chief technology officer at Analytics8.

Most companies are racing to implement AI, yet lack the foundational investment that makes it work: quality data infrastructure. Reliable, standardized, and accessible information isn't merely an operational concern—it's a strategic requirement. Without this foundation, even advanced AI models fail to produce meaningful or dependable outcomes.

Many businesses deploy AI for task automation and productivity gains, but leading organizations push further—using AI to strengthen decision-making across business functions. This advanced approach depends on proprietary data, transforming internal information into competitive advantage, and enabling faster, more informed choices throughout the enterprise.

However, achieving this vision presents significant obstacles. Gartner’s findings indicate that 63% of organizations lack adequate data management practices for AI initiatives, or remain uncertain about their capabilities. The consequences are severe: Gartner predicts that 60% of AI projects will be discontinued by the end of next year, primarily due to insufficient data readiness.

Unstructured data: The hidden barrier to AI readiness

While tools like dbt have transformed how we model and manage structured data for analytics and AI, unlocking AI's full potential requires organizations to also govern massive volumes of unstructured data—emails, documents, transcripts, logs, images, audio files, and more. This content often sits siloed in file-sharing platforms like Microsoft SharePoint. Chief data and information officers have established governance for human access to unstructured data, but now face the urgent task of adapting these frameworks to support AI consumption patterns.

Mid-market firms face distinctive challenges in preparing data for AI and business intelligence. Positioned between large enterprises with substantial IT budgets and smaller digital-native companies operating on modern SaaS platforms, mid-sized organizations often lack the talent, tools, and resources needed to wrangle vast amounts of source data in inconsistent formats across incompatible legacy systems.

What we learned from the mid-market

To understand what prevents mid-market organizations from achieving data readiness, we conducted a targeted research study in September 2025. We surveyed business and technology leaders from 100+ North American companies spanning financial services, insurance, health sciences, and consumer products sectors.

Respondents were grouped into three categories based on revenue performance since 2020:

Leaders: Companies with 15%+ revenue growth

Followers: Organizations with 1% to less than 15% revenue growth

Laggards: Firms experiencing revenue contraction

Our analysis identified six major challenge categories spanning data foundations, data transformation, and analytics tools that block AI and business intelligence initiatives from delivering business value. While our focus centers on mid-market firms, these findings apply equally to larger organizations facing similar data readiness obstacles.

What the research uncovered

The readiness gap is substantial. Only 14% of surveyed mid-market companies report achieving complete data readiness. More concerning, 15% indicate that 10% or less of their data is AI-ready. Performance differences are stark: 87% of high-performing companies report at least 75% of their data is prepared for AI, compared to just 11% of moderate performers. None of the declining organizations reached this 75% threshold.

Structured vs. unstructured data management shows a critical divide. While 57% of respondents consider their organizations effective or extremely effective at managing structured data, only 41% claim comparable proficiency with unstructured data. This gap is particularly significant given that unstructured data readiness is essential for both AI and business intelligence success.

Strategic investment remains insufficient. Data strategy receives just 14% of IT spending on AI and analytics initiatives. This underinvestment in foundational planning directly affects whether platforms, tools, and broader AI programs deliver meaningful outcomes.

Legacy tools and workflows aren’t adequate. 40% of respondents rate their data ingestion and analytics tools as ineffective. Most troubling, nearly one in five organizations (19%) describe their tools as extremely ineffective. The data readiness gap often stems from legacy ETL tools and brittle workflows that can’t support today’s scale or complexity. Companies need modern technologies like dbt, which emphasize modular data transformation and version control, to help teams standardize and trust their data pipelines.

85% of companies say inconsistent data architecture, inadequate data hygiene, and siloed systems are the biggest barriers — Left unresolved, these problems compound technical debt, leaving data inaccessible, unreliable, and unusable for AI applications. Relatedly, 73% of surveyed organizations point to talent shortages as a major obstacle, reflecting the scarcity of skilled professionals who can build and maintain modern data infrastructure.

Technical and organizational challenges stall AI deployment. Technology limitations prevent 17% of AI initiatives from reaching production. Organizational factors prove equally problematic: 15% cite budget misalignment as a barrier, while 10% identify insufficient executive sponsorship.

About Analytics8

Analytics8 helps organizations make smart, data-driven decisions by translating their data into meaningful and actionable information. Our data consultants help with the entire data and analytics lifecycle — from strategy to implementation — so companies can make sense of their data and use it to solve complex business problems.