Building AI apps backwards: Model product issues

Most AI developers build their products backwards. They start with the foundation model, wrap it in an interface and wonder why users don't get the results they need.

I can see this pattern everywhere. Developers treat models like APIs. However, the foundation model is not a utility. Engineers are the starting point necessary to shape, train and match the specific problems that users face every day.

Why generic foundation models fail in enterprise applications

Consider developers at financial services companies looking to build investment research applications using GPT-4. This model can analyze financial documents and answer questions about market trends. Still, it doesn't understand the difference between everyday revenue calls and what indicates a major strategic change in the business. It is not possible to distinguish between important regulatory applications between internal compliance teams and non-compliance teams.

Openai trained GPT-4 for a wide range of features across thousands of domains. However, the financial services company needs a model that understands financial analysis, regulatory requirements, and the nuances of the specific workflows of the team's portfolio managers.

Fundamental Problems: Two Different Design Goals

Two completely different groups with different objectives design basic models and end-user AI applications – researchers and product developers. Leading model lab researchers optimize a wide range of features across academic benchmarks. Often, they use the same training dataset as their competitors. However, developers build applications aimed at specific customers and market needs. It is an automated competitive analysis for customer service for food delivery apps, generative graphic design for productivity platforms, and market research.

These approaches are off by design. Researchers at Openai, Deepseek, Meta, or Anthropic have not built a model, so when Deloitte consultants say “analyze this report,” they find that they compare their client's data to specific competitors. Or the difference between a model who understands the “urgent” differences between a customer service representative and a doctor who makes a medical diagnosis. The developers are.

The model is raw material. Developers' unique data shaping products that match the user's workflow and business requirements.

What does this mean for your AI product strategy?

To achieve model product alignment, developers need to prioritize.

Data Feedback Loop: Build a model that actually learns from users

A working application creates a feedback loop where all user interactions make the model better for a particular task. When the user modifys the output of the application, the modification returns to model training. If the user ignores certain suggestions, the model learns to stop creating them.

Data feedback loops create compound interest benefits. More users generate better training data, improve model performance, and attract more users. These systems improve instead of hitting performance walls while scaling.

This approach requires that model development be treated as part of product development, rather than vendor management. This means managing model performance across different user segments and solving new issues with model version version and deployment.

Model composition: How large companies combine multiple AI systems

The most differentiated AI applications do not rely on a single model, but instead configure multiple models and modalities to solve problems. Logistics companies can combine computer vision of package scans with natural language processing for customer communication with predictive modeling for route optimization. Everything functions as a unified system.

This requires engineers to understand the model architecture and allow strategic decisions to be made about when to use the retrieved generation and tweak the special model.

The winning application is not an application with privileged access to the closed source model. They have dedicated resources to tailor open source models to valuable final products through careful alignment, data collection and iterative improvement.

Developers who solve this alignment problem early will build moats that are not possible to replicate. Those who don't, switch to the latest closed source model API and compete for features that competitors can copy.