Most business data is tabular and stored in data warehouses, CRMs, and financial ledgers, but building reliable models from it requires training new models from scratch for each dataset, maintaining hyperparameter tuning loops, maintaining feature engineering, and retraining pipelines to combat data drift. Google Research suggests ways to work around this issue. It’s a new foundational model called TabFM that treats tabular prediction as a learning problem in context.
You can generate predictions for new unconfirmed tables in a single forward pass. For enterprise developers and AI engineers, this reduces weeks of pipeline engineering to a single API call, reducing time to production.
Traditional ML challenges
To extract reliable predictions from gradient-boosted trees, data scientists must build and maintain complex data pipelines. You need to clean up cluttered input, impute missing values, encode categorical variables into numeric format, and design custom feature intersections.
Once the data is ready, we need to iterate through a hyperparameter optimization loop, searching for learning rate, tree depth, subsampling ratio, and regularization grid to find the optimal configuration.
Once deployed, these traditional models “incur ongoing operational debt from monitoring data drift and retraining pipelines to maintain accuracy,” Google Research researcher Weihao Kong told VentureBeat.
Meanwhile, the rest of the AI industry is moving forward. Text and computer vision generation AI models are seamlessly transitioning to zero-shot inference, allowing models to perform entirely new tasks simply by being guided by context.
Large-scale language models (LLMs) are already great at learning in context, so why can’t we feed tables into an off-the-shelf LLM?
Because LLM is trained on natural language rather than structured data, it has a hard time processing tables directly. First, a medium-sized table containing only a few thousand rows and a few hundred columns will quickly exhaust the context limit. Second, LLM suffers from tokenization inefficiencies, which partition numbers unnaturally and destroy mathematical precision. Finally, they suffer from structural blindness. When a 2D table is serialized as a 1D text string, as the table grows, LLM loses track of which values belong to which rows and columns.
“So it’s now much more efficient to use LLM to handle feature engineering and write code that calls XGBoost, rather than asking LLM to read the table itself,” Kong said.
What is TabFM?
To perform inference with TabFM, you do not update the model weights. Instead, it takes past examples (training rows with known labels) and target rows (new data to predict) and passes them to the model as a single unified prompt. The model learns to interpret column and row relationships directly from this context at runtime.
For example, consider a business analyst trying to predict customer churn. Instead of building a bespoke data pipeline to train an XGBoost model, you can simply pass samples of past user session data to TabFM along with new active sessions. In one forward pass, the model returns the immediate churn probability.
TabFM overcomes the limitations of LLM by treating the data as a grid, preserving structural integrity without forcing it into a one-dimensional text string.
To effectively handle diverse table structures while enabling scalable zero-shot predictions, TabFM integrates the strengths of previous experimental architectures TabPFN and TabICL. TabPFN, developed by Prior Labs, was the first to prove that a transformation architecture could perform zero-shot classification on small tables, but was computationally difficult to scale to larger datasets.
TabICL, developed by the French National Institute for Digital Science and Technology, then addressed this bottleneck by introducing row compression, allowing contextual learning to efficiently process larger tables.
TabFM combines TabPFN’s deep feature contextualization with TabICL’s efficient compression to deliver a new hybrid design built on three key mechanisms:
1. Pay attention to rows and columns alternately: The raw table is first processed through a multi-layer attention module that alternates between columns (features) and rows (examples). By continuously engaging across these two dimensions, the model natively captures complex functional interactions. This deep contextualization does the heavy lifting, typically requiring tedious manual feature creation by data scientists.
2. Row compression: Following this contextualization, each row’s mutual participation information is compressed into a single dense vector representation. TabICL pioneered this by using CLS tokens to compress a row’s rich information into a single vector, “as opposed to TabPFN v2, v2.5, and v2.6, which participate through a complete cell grid across the network,” Kong explained. This significantly reduces the amount of computation.
3. In-context learning (ICL): A causal Transformer then operates on this set of compressed embeddings. This Transformer model uses TabICL’s attention mechanism to process these dense row vectors, significantly reducing computational costs and allowing the model to efficiently process large datasets.
TabFM’s main selling point is its pre-workout recipes. This model was fully trained on hundreds of millions of synthetic datasets. These datasets were dynamically generated using a structural causal model (SCM) incorporating various random functions. By training solely on synthetic SCMs, TabFM learned basic mathematical priors about how tabular features interact without ingesting sensitive real-world CSV files.
TabFM in operation
To test the model’s capabilities, Google researchers benchmarked TabFM on TabArena, a comprehensive evaluation suite across 51 diverse tabular datasets across 38 classification tasks and 13 regression tasks.
On these public benchmarks, TabFM’s zero-shot predictions already match or exceed tightly tuned supervised baselines. However, Google notes that this does not automatically mean that TabFM completely eliminates bespoke, hyper-optimized operating models for all enterprise workloads.
“Rather than replacing hyper-optimized production models, it is speed that can deliver real practical business value for lean engineering teams,” said Conn. “This enables data analysts and backend engineers to instantly launch high-quality baseline models without the need for a dedicated data science team to manage complex lifecycles.”
For advanced practitioners who want to squeeze out maximum precision, the research team also introduced the ‘TabFM-Ensemble’ configuration. TabFM further boosts performance by running the model through 32 different variations and blending the results.
How to get started, tradeoffs, and the future of cloud
The move to in-context learning of tables creates new economic trade-offs that engineering teams must consider.
Traditional algorithms are slow and expensive to train, but inference is lightning fast and cheap. TabFM reverses this dynamic. While training time is reduced to zero, inference becomes significantly heavier. The model must process the entire historical dataset as context for each single prediction, which requires more compute and memory at runtime.
In this new paradigm, “traditional machine learning training becomes a context window ‘prefill’ phase (KV cache),” Kong said. This prefilling cost is high, but it is paid only once per table, and the cache is reused for subsequent queries. “The problem is predictive delay, which no amount of caching can eliminate,” Kong added. Every time we make a new prediction, we have to go through a massive transformer. “Production APIs that require single-digit millisecond response times cannot tolerate the forward path overhead of TabFM.”
The barrier to entry is low for developers who want to evaluate this model now. Google designed TabFM as a drop-in replacement for traditional ML workflows and provides scikit-learn compatible APIs (TabFMClassifier and TabFMRegressor). It natively handles a mix of numeric and categorical columns, works directly with pandas DataFrame, and does not require manual ordinal encoders or numeric scalers. This library supports both JAX and PyTorch backends.
However, enterprise teams need to be aware of current limitations and licensing restrictions. The model architecture has a hard limit of 10 output classes for classification tasks and is optimized for tables with up to 500 features. More importantly, while Google released the underlying codebase under the permissive Apache 2.0 license, the pre-trained model weights are published on Hugging Face under the strict tabfm-non-commercial-v1.0 license. Developers can evaluate the model internally, but cannot yet deploy it in commercial products.
Looking ahead, Google is addressing commercial deployment friction through its cloud ecosystem. TabFM is directly integrated into Google BigQuery, allowing analysts to perform zero-shot predictions natively via the “AI.PREDICT” command. By placing the underlying model inference right next to the data warehouse, TabFM has the potential to make complex tabular machine learning as readily accessible as basic database queries.
In fact, TabFM shines in rapid prototyping, environments with high data drift, and small to medium-sized datasets of less than 100,000 rows. Conversely, for strict, ultra-low-latency APIs or large tables with more than 1 million rows, teams should stick to the traditional model. Currently, these tables require aggressive row sampling, which makes the underlying model less competitive.
