Enterprise data is often outdated by the time AI systems can use it, creating significant lag for agent AI that needs continuous access to the latest information. Snowflake has enhanced its platform to address this issue, introducing native Apache Kafka-compatible streaming capabilities and AI-powered features to streamline the data development lifecycle. These updates are intended to ease the infrastructure management burden for data engineering teams.
At the heart of this push is Snowflake Datastream, a new service designed to integrate streaming data directly into Snowflake. Reduce operational overhead by allowing your data to be placed as native Snowflake or Apache Iceberg tables and be queried within seconds. Data is managed at ingestion using security and lineage policies inherited from Snowflake’s Horizon catalog. CoCo, Snowflake’s conversational AI, simplifies Datastream setup and authentication and minimizes Kafka expertise.
Stream at AI pace
Agentic AI operates in a continuous decision-making loop and requires a constant flow of data. Organizations already using Kafka face the challenge of managing separate analytics platforms, leading to increased costs and data latency. Datastream aims to consolidate these systems into a single managed platform.
Enhancements to Snowflake Snowpipe Streaming, a direct ingestion API, include Kafka Connector 4.0, which provides up to 10 GB/s of server-side ingest per table and reduces client-side resource needs by up to 30%. New error logging captures failed rows to facilitate data quality control, and multilingual SDK support enables streaming from familiar stacks like Java and Python.
Elastic Channels (private preview) enables thousands of clients to simultaneously stream gigabytes per second. Durable Acknowledgment (Private Preview), on the other hand, is intended to eliminate data loss between ingest and commit and ensure that mission-critical pipelines do not send incomplete data to agents.
self-managed pipeline
Converting raw data streams into usable insights requires a continuous and reliable pipeline. Snowflake Dynamic Tables is now faster, with performance enhancements that make refreshes for common workloads up to 2.8x faster. Custom Incrementation (Public Preview) allows engineers to perform complex transformations using MERGE or INSERT statements while maintaining automation.
DCM Projects (Public Preview) provides an integrated workflow for defining your infrastructure and deploying changes across your environment. CoCo skills are also integrated to accelerate the setup and troubleshooting of Snowpipe streaming, dynamic tables, and dbt projects, allowing engineers to focus on pipeline logic.
Semantically access enterprise data
High-value data often resides in platforms such as SAP, Salesforce, and Workday. Restructuring this data for AI initiatives can be a major hurdle. Zero-copy integration exposes this intelligence directly to Snowflake without moving any data. SAP BDC Connect for Snowflake is now generally available, enabling bidirectional zero-copy integration with SAP ERP data. Salesforce Data 360 offers an enhanced connector experience where Workday data goes into private preview and appears as an externally managed Iceberg table.
These integrations inherit Snowflake’s governance boundaries and provide end-to-end lineage and access policies. CoCo skills manage the lifecycle management of these connections through natural language prompts.
Connect the rest with Openflow
For data from on-premises OLTP databases, SaaS applications, and legacy systems, Snowflake Openflow, a managed data integration service, is expanding. Its managed deployment is now generally available on Google Cloud in addition to AWS and Azure. Data Connectivity Proxy (coming soon on AWS) extends Openflow to private networks.
Openflow supports structured and unstructured data, batch and streaming. AI-assisted troubleshooting powered by CoCo is built into the connector monitoring dashboard to analyze logs and provide remediation steps. New connectors for Veeva, BigQuery, and MongoDB are in public preview.
Build and deploy at scale with Snowpark
Snowpark continues to bridge the gap between code prototypes and production environments for programmatic data transformation. Summit announcements include optimized ML batch inference (public preview), expanded data integration API with JDBC support (public preview), and file transformation for Apache Spark (coming soon to public preview).
Snowpark Directory Import is now generally available for deployment of simpler, multi-file Python projects. CoCo skills for Snowpark Python and Apache Spark are aimed at accelerating deployments and migrations, promising faster performance and lower costs.
Modernize with Snowflake AIM
Snowflake AIM (AI-powered migration) is now generally available and unifies migration, modernization, and virtualization. It combines IP from SnowConvert AI, Snowpark Migration Accelerator, and Datometry. The AIM Migration Agent, accessible through Snowflake CoCo, guides you through the migration process and identifies dependencies and risks before making changes to your production environment.
This approach significantly reduces the time and effort required for modernization projects.
The overarching theme across these updates is to help engineers spend less time maintaining systems and more time focusing on outcomes. The role of the data engineer is evolving from infrastructure management to building the data foundation that powers AI. Snowflake aims to provide visibility into complex data operations so data teams can focus on innovation.
