Many Companies Use AI. Few Know How to Build an AI-Native Enterprise Data Platform.

have already adopted AI to improve everyday productivity over the past few years. AI has rapidly become part of enterprise workflows:

• Building internal chatbots to answer HR, IT, or policy questions.
• Using ChatGPT or Microsoft Copilot to search and summarize documents.
• Assisting developers with code generation and debugging.
• Drafting reports, emails, meeting notes, and business presentations.

These applications have undoubtedly improved individual productivity. But if we believe that’s where AI’s potential ends, we’re missing the important parts. The reality is that many companies stop here and fail to tap into AI’s most transformative capabilities.

AI can do much more. In my view, one of its most powerful applications lies in transforming the enterprise data ecosystem.

Beyond Chatbot: What AI Agents Actually Do

Data teams in many organizations spend a significant amount of time every day answering questions from business users. For example, if you are a data analyst working for an E-commerce platform, you may receive tons of questions from business like: “Which product categories contributed most to revenue growth in Southeast Asia last quarter?“

As a data analyst, here’s what you typically do:

Business Question
      ↓
Write SQL
      ↓
Export Data
      ↓
Create Charts
      ↓
Explain Findings

Now, you hand this over to an AI agent, and the workflow becomes:

Business Asks
      ↓
Agent Retrieves Semantic Information
      ↓
Generates SQL
      ↓
Returns Explanation

On the surface, the business user is still just having a conversation with AI by throwing a question to get an answer. Feels a lot like chatting with a bot, doesn’t it? But working with an AI agent is fundamentally different from chatting with a chatbot.

What is an AI Agent?

An AI agent is an autonomous system that perceives its environment, makes decisions, and takes concrete actions to achieve a goal.

The key difference between an AI agent and a chatbot is that an AI agent can take actions instead of simply generating responses. While chatbots primarily answer questions through conversations, AI agents execute multi-step tasks, interact with software and tools, make decisions, and work toward completing a specific goal autonomously.

To be more specific, the key differences between them are:

Although the business users may feel like they’re just having a conversation with the AI agent, behind the scenes the agent is busy executing a series of actions—retrieving relevant context, generating and running SQL queries, interpreting the results, and then delivering a polished answer.

In the world of data, these AI agents are usually called data agents. They focus on retrieving, querying, analyzing, and explaining enterprise data through natural language interactions. Most data platforms, like Microsoft Fabric, Snowflake, and Databricks, have data agents integrated into themselves. For example, Fabric has the Fabric data agent, Snowflake has Cortex Analyst, and Databricks has AI/BI Genie. If you don’t want to be tied to a specific platform, you can choose Julius AI or Tellius, which can connect to most mainstream data platforms, either natively or indirectly.

Data agents are designed to act as AI data analysts. They reduce the repetitive work of pulling data, writing routine queries and generating standard reports so that analysts spend less time performing repetitive data retrieval and reporting tasks, and more time on work that requires human judgment and critical thinking. Business users also benefit. They get analytical support 24/7 without waiting, and the agent can proactively surface insights instead of requiring someone to manually explore the data.

Looks beautiful? But in practice, simply relying on data agents often leads an organization to the following problems:

• Ambiguous business terminology
• Multi-step reasoning
• Business rules
• Inconsistent answers
• Retrieval quality
• Handling edge cases that sit outside predefined semantic layers
• Keeping up when data schemas change
• Maintaining accuracy across different business contexts

These aren’t small annoyances. For example, when the business user typed “What is the percent of revenue growth in Southeast Asia last quarter?“, it would be very frustrating if the agent answers with no data provided or provides incorrect number.

When a data agent gets something wrong, it doesn’t just frustrate the users. To make matters worse, it can feed bad information into a business decision.

The bottom line? Relying on data agents alone isn’t enough. The real path forward should be connecting data platforms with enterprise AI architectures.

Where AI Fits in the Data Platform

A typical enterprise data platform workflow looks like this: data engineers design the architecture, implement the creation of ETL pipelines and data warehouse and manage the data governance. Business users raise business-related questions, data analysts create BI reports or dashboard. Business users then use dashboard for analysis and generate insights.

This workflow has run for decades and effectively supported and empowered many businesses. Then AI came. People start thinking:

• Why do business users keep asking the same questions?
• Why do data engineers spend hours validating ETL jobs?
• Why do analysts manually investigate KPI changes?

Quickly, AI is embedded into the data platform. Data agents are used. Agentic coding is introduced. Then come new questions:

• Why do we trust AI-generated answers without measuring their quality?
• Why does AI become less reliable as business rules grow more complex?

These aren’t isolated problems. They’re symptoms of a traditional data platform that was designed for storing and reporting data instead of collaborating with AI.

Maybe it’s time for us to rethink the architecture itself rather than treat AI as an add-on application to existing data platform.

For the AI architecture thing, there’s no standard answer yet. There might never be a standard answer. The AI architecture can be customised by industry, enterprise scale, business strategy and data/AI technology maturity level.

In my view, organizations should include at least the 3 key AI components in their data workflow – Data Agent, AI QA Agent and AI Governance & Observability.

Enterprise AI doesn’t eliminate the need for robust data engineering implemented by humans. Instead, AI can enhance it. No matter how smart AI agents are, before they can answer business questions or validate data quality, the underlying data platform must already be reliable and scalable. In a previous article, What Can We Do When Memory Becomes the New Bottleneck in Data Engineering?, I discussed one of the challenges that every data engineer would face when processing large-scale datasets and provided several practical solutions for different scenarios.

Let’s go back to the problems that most data agents face:

• Ambiguous business terminology
• Multi-step reasoning
• Business rules
• Inconsistent answers
• Retrieval quality
• Handling edge cases that sit outside predefined semantic layers
• Keeping up when data schemas change
• Maintaining accuracy across different business contexts

To resolve these issues, we can use AI Agent SDKs to either build autonomous systems from scratch or extend the capabilities that existing data agents don’t provide out of the box. The most popular tools in the market include LangGraph, Microsoft Agent Framework, or Google ADK. I’ll discuss how to build a data agent in my next article.

How AI Is Transforming Data Quality Assurance

Imagine you are working for a healthcare company. Every day, you need to process millions of patient records—lab results, insurance claims, clinical notes, prescription logs. When the data arrive, you must ensure your pipelines ingest, transform and load them correctly because it’s not just about clean dashboards; more importantly, it’s about patient safety, regulatory compliance, and financial accuracy. So you prepare the list to check:

• Row counts (did we drop records during ingestion?)
• NULL checks (are required fields empty?)
• Duplicate detection (same record entered twice?)
• Schema validation (right data types, right column names?)
• Range checks (is a blood pressure reading of 999 realistic?)
• Format validation (do date fields follow YYYY-MM-DD? Are email fields actually emails?)
• Referential integrity (does a patient ID in the claims table exist in the patient table?)
• Freshness checks (did today’s data actually arrive on time?)

Based on this list, you define rules, schedule jobs to run these checks, and get alerts when something breaks. Mostly you use SQL-based validation queries, YAML or JSON rule configurations, and dashboard monitors showing pass/fail rates. Your workflow works until it doesn’t. Why? Because they only catch what you already know to look for. If you didn’t anticipate a failure mode, there’s no rule for it. Therefore, you have to manually change the rules. But for an environment with huge datasets or with data changing frequently, the rule library maintenance becomes a nightmare.

AI-Powered QA doesn’t replace traditional checks. Instead, it adds a layer that learns.

Traditionally, you follow the process to complete your data QA.

Define rules
      ↓
Run checks
      ↓
Get pass/fail alerts
      ↓
Investigate manually

But when you hand your QA work over to AI models, they learn what normal data looks like from historical patterns rather than solely rely on predefined rules. They catch anomalies like subtle distribution shifts, unusual correlations between fields, emerging data drift that signals a pipeline issue upstream. These anomalies haven’t yet been added to the checklist by you in advance. For the healthcare example, AI-powered QA might catch that lab results from a specific clinic which suddenly has test values 10x higher than their historical average. Traditional QA would give you a pass because the dataset has the same format, valid ranges, no NULLs and no duplicates. But AI flags it because it doesn’t look right compared to what that clinic has always produced. Embedded with AI, the whole QA workflow becomes:

Learn patterns
      ↓
Detect anomalies
      ↓
Surface with context
      ↓
Explain possible cause

Similar to data agents, there are also a few AI-powered QA tools available to support enterprise QA. Popular tools include Great Expectations (rule-based primarily, with extensibility for anomaly detection through custom expectations and integrations), Soda (combining rule-based checks with ML-powered anomaly detection via Soda Cloud), Databricks Lakehouse Monitoring (native profiling and drift detection for data and ML model features) and AWS Glue Data Quality (automated quality rule recommendations and anomaly detection within the Glue ecosystem).

For example, if you’d like to combine your original rule-based QA with AI for the healthcare company data anomaly detection, you can use the following method.

from soda.scan import Scan
from soda.contracts.contract import Contract
from soda.contracts.check import AnomalyCheck, SchemaCheck, UserDefinedCheck

# Traditional checks: rules you define
traditional_contract = Contract(
    checks=[
        SchemaCheck(
            name="Schema validation",
            fail_if_missing_columns=["patient_id", "diagnosis_code", "lab_result"]
        ),
        UserDefinedCheck(
            name="No duplicate patient records per day",
            query="""
                SELECT patient_id, admission_date, COUNT(*)
                FROM patient_records
                GROUP BY patient_id, admission_date
                HAVING COUNT(*) > 1
            """,
            fail_if_rows_returned=True
        )
    ]
)

# AI-powered checks: anomaly detection based on learned patterns
ai_contract = Contract(
    checks=[
        AnomalyCheck(
            name="Anomaly: lab result distribution shift",
            metric="mean(lab_result)",
            anomaly_detection="ml",
            sensitivity=0.8,
            fail_if_anomaly_severity="critical"
        ),
        AnomalyCheck(
            name="Anomaly: missing diagnosis codes",
            metric="missing_count(diagnosis_code)",
            anomaly_detection="ml",
            fail_if_anomaly_severity="warning"
        ),
        AnomalyCheck(
            name="Anomaly: record volume by source",
            metric="row_count",
            anomaly_detection="ml",
            group_by=["data_source"],  # monitors each hospital's volume independently
            fail_if_anomaly_severity="critical"
        )
    ]
)

# Run the scan
scan = Scan()
scan.set_data_source_name("healthcare_db")
scan.add_contracts([traditional_contract, ai_contract])
scan.set_verbose(True)
scan.execute()

In addition to anomaly detection without predefined thresholds and root cause investigation, AI-powered QA methods have the capabilities of contextual understanding and pattern recognition across multiple dimensions. AI models can continuously relearn what “normal” means rather than wait for someone to update thresholds manually. With these features, AI greatly improves the efficiency and accuracy of data QA workflows.

AI Can Get it Wrong. How Do We Trust It?

Many people think AI governance means security: role-based access, data masking and confidential information safely stored. But after AI is fully integrated to your enterprise system, governance is about something broader: can you explain and stand behind every answer your AI gives?

Imagine you are a portfolio manager in an investment firm. One day, you asked a data agent: “Which funds exceeded their ESG targets last quarter?” The agent pulled data, ran the numbers and returned an answer. A month later, you asked the same question but got a different answer. In the past month, nobody changed the query or updated the data. And nobody knew what shifted inside the agent and why.

Now AI governance matters. Unlike traditional IT governance or data governance, AI governance and observability usually focus on the following areas:

Prompt Versioning

Prompt versioning means treating prompts like any other software artifact. Similar to the process in software engineering, AI engineers store prompt versioning in Git, tag releases, and log which version was active when a query ran. So when the portfolio manager asks why last month’s answer is different, the first place to look is whether the prompt changed. If it did, you have your explanation. If it didn’t, you need to dig deeper. It matters for data agents because a small wording change can shift results without anyone realizing it.

Hallucination Detection

Data agents hallucinate and it’s dangerous because a hallucinated number looks like a real number. That’s why hallucination detection is one of the hottest areas that many AI experts research on.

When you take the hallucination detection for data agents, you can verify outputs against source data. Methods include SQL execution validation, results grounding and confidence scoring.

Tracing

Tracing is the “what happened” layer, which records every step the AI application took. If you want to trace a data agent, you can use tools to record the user’s question, how it was interpreted, which SQL was generated, which tables were queried, what results came back, and how the final answer was composed. LLM tracing tools include LangSmith, Weights & Biases, and Phoenix, which are commonly used alongside data platforms.

Monitoring

Monitoring is tracing plus time. Just as you monitor data pipelines for freshness and anomalies, you monitor AI agents for behavioral drift. You monitor your AI tools by signals. For example, you can monitor the signals like query success rate, answer latency, answer refusal rate and user feedback trends for a data agent. As these signals are critical for you to determine if your agent is actually good at its job, AI monitoring system is equally important to AI-empowered QA system. The two monitoring systems should feed into the same observability stack.

Security

In addition to the traditional security questions discussed by data governance, there are specific concerns brought by AI data agents – query injection, data exfiltration through prompting and over-permissioning.

• Query injection: When a user types a question, the agent generates query which has a chance to slip in destructive commands. The solution to this problem is to use parameterized queries, enforce read-only execution, and block any statement that tries to modify data rather than run generated query directly.
• Data exfiltration through prompting: A user could craft a prompt that tricks the agent into pulling sensitive data and sending it somewhere it shouldn’t go. The solution is to conduct tool-call allowlisting and output scanning which allow the agent only to do what you’ve explicitly permitted and check anything leaving the system.
• Over-permissioning: AI agents can run with a broad service account that sees everything. So there’s risk that they serve data to the user who shouldn’t have access to. The solution is to pass the end user’s security context through to the data layer so every generated query respects the user’s actual permissions.

Human Feedback

Only user feedback helps you find the room for improvement that you never anticipated. There are many ways to collect feedback.

Human feedback matters because real users will ask questions you’ve never anticipated. In order to collect feedback, the simplest method is to allow users to thumbs-up / thumbs-down on every answer, with an optional comment field. But when AI governance and observability is set properly in the enterprise AI architecture, you can get more from the system. If a user marks an answer as incorrect, the system can capture the full trace so that AI engineers can investigate. The feedback improves the evaluation dataset, identifies confusing business terms, highlights queries where the agent consistently struggles and tell you where to invest in prompt engineering over time.

Governance and observability sound bureaucratic. But in practice, they differentiate a demo from something you can trust and make decisions on. As the three key components of an AI-driven enterprise data architecture, data agents, AI-empowered QA, and AI Governance work together to build a trustworthy collaborator with humans.

Thank you for your reading!

Buy me a coffee if you like this article!