Dive deep into embedding images and vector searches using BigQuery in Google Cloud

Images by editor | chatgpt

# introduction

We were all there: scrolling endlessly through our online store and trying to find it perfection item. In today's lightning e-commerce world, immediate results are expected. That's exactly where AI is stepping in and shaking things.

At the heart of this revolution is the image embedded. It's a flashy term for simple ideas: you can search for products by them as well as keywords Visual similarity. Simply upload a photo and find the exact dress you saw on social media. This technology makes online shopping smarter, more intuitive, and ultimately helps businesses sell more.

Are you ready to see how it works? We'll show you how to leverage the power of BigQuery's machine learning capabilities to build your own AI-driven dress search using these incredible image embeddings.

# Image embedding magic

Essentially, image embedding is the process of converting an image into a numerical representation (vector) in a higher dimensional space. Semantically similar images (for example, blue ball gowns or navy blue dresses) have vectors “close” to each other in this space. This allows for powerful comparisons and searches that go beyond simple metadata.

Here are some images of the dresses that will be used in this demo to generate embeddings.

The demo shows the process of creating an image embedding model in Google Cloud.

The first step is to create a model: Name model image_embeddings_model Created with leverage multimodalembedding@001 endpoint in image_embedding Dataset.

CREATE OR REPLACE MODEL 
   `image_embedding.image_embeddings_model`
REMOTE WITH CONNECTION `[PROJECT_ID].us.llm-connection`
OPTIONS (
   ENDPOINT = 'multimodalembedding@001'
);

Creating an Object Table: To process images in BigQuery, create an external table called external_images_table in image_embedding A dataset that references all images stored in a Google Cloud Storage Bucket.

CREATE OR REPLACE EXTERNAL TABLE 
   `image_embedding.external_images_table` 
WITH CONNECTION `[PROJECT_ID].us.llm-connection` 
OPTIONS( 
   object_metadata="SIMPLE", 
   uris = ['gs://[BUCKET_NAME]/*'], 
   max_staleness = INTERVAL 1 DAY, 
   metadata_cache_mode="AUTOMATIC"
);

Creating embeddings: Once the table of models and objects is placed, generate an embed of the dress image using the above model and save it in the table dress_embeddings.

CREATE OR REPLACE TABLE `image_embedding.dress_embeddings` AS SELECT * 
FROM ML.GENERATE_EMBEDDING( 
   MODEL `image_embedding.image_embeddings_model`, 
   TABLE `image_embedding.external_images_table`, 
   STRUCT(TRUE AS flatten_json_output, 
   512 AS output_dimensionality) 
);

# Unleash the power of vector search

Generate image embeddings and use vector search to find the dress you're looking for. Unlike traditional searches that rely on accurate keyword matches, vector searches find items based on embedding similarity. This means that you can search for images using either a text description or other images.

// Dress up search with text

Perform a text search: Used here VECTOR_SEARCH It works within the big query to search for “blue dress” among all dresses. The text “blue dress” is converted to a vector and with the help of a vector search, you get a similar vector.

CREATE OR REPLACE TABLE `image_embedding.image_search_via_text` AS 
SELECT base.uri AS image_link, distance 
FROM 
VECTOR_SEARCH( 
   TABLE `image_embedding.dress_embeddings`, 
   'ml_generate_embedding_result', 
   ( 
      SELECT ml_generate_embedding_result AS embedding_col 
      FROM ML.GENERATE_EMBEDDING 
      ( 
         MODEL`image_embedding.image_embeddings_model` , 
            (
               SELECT "Blue dress" AS content
            ), 
            STRUCT 
         (
            TRUE AS flatten_json_output, 
            512 AS output_dimensionality
         ) 
      )
   ),
   top_k => 5 
)
ORDER BY distance ASC; 
SELECT * FROM `image_embedding.image_search_via_text`;

result: Query results provide An image_link And the distance of each result. You can see the results you get will give you the closest match in terms of search queries and available dresses.

// Dress up search through images

Next, we will consider how to use the images to find similar images. Find the dress that looks like the one shown in the image below.

External table of test images: You need to save the test image in a Google Cloud Storage Bucket and create an external table external_images_test_tablesaves the test image used for searching.

CREATE OR REPLACE EXTERNAL TABLE 
   `image_embedding.external_images_test_table` 
WITH CONNECTION `[PROJECT_ID].us.llm-connection` 
OPTIONS( 
   object_metadata="SIMPLE", 
   uris = ['gs://[BUCKET_NAME]/test-image-for-dress/*'], 
   max_staleness = INTERVAL 1 DAY, 
   metadata_cache_mode="AUTOMATIC"
);

Generates an embedding of the test image: Next, generate using this single test image embedding ML.GENERATE_EMBEDDING function.

CREATE OR REPLACE TABLE `image_embedding.test_dress_embeddings` AS 
SELECT * 
FROM ML.GENERATE_EMBEDDING
   ( 
      MODEL `image_embedding.image_embeddings_model`, 
      TABLE `image_embedding.external_images_test_table`, STRUCT(TRUE AS flatten_json_output, 
      512 AS output_dimensionality
   ) 
);

Vector search with image embeddingFinally, to perform a vector search using the test image embedding image_embedding.dress_embeddings table. ml_generate_embedding_result from image_embedding.test_dress_embeddings Used as an embed for queries.

SELECT base.uri AS image_link, distance 
FROM 
VECTOR_SEARCH( 
   TABLE `image_embedding.dress_embeddings`, 
   'ml_generate_embedding_result', 
   ( 
      SELECT * FROM `image_embedding.test_dress_embeddings`
   ),
   top_k => 5, 
   distance_type => 'COSINE', 
   options => '{"use_brute_force":true}' 
);

result: Image search query results showed the most visually similar dress. The best result was white-dress At a distance of 0.2243, followed by sky-blue-dress At a distance of 0.3645 polka-dot-dress Distance of 0.3828.

These results clearly demonstrate the ability to find visually similar items based on the input image.

// Impact

This demonstration effectively demonstrates how embedding images and vector searching in Google Cloud can revolutionize the way they interact with visual data. Applications are vast to accommodate content management systems that provide intelligent visual asset discovery from e-commerce platforms that enable “similar” functionality. By converting images into searchable vectors, these technologies unlock new dimensions, making them more intuitive, powerful, and visually intelligent.

These results are presented to the user and can quickly find the dress of your choice.

# Benefits of AI Dress Search

1. Enhanced user experience: Visual search provides a more intuitive and efficient way to find what users are looking for
2. Improved accuracy: Image embedding allows searches based on visual similarity, providing more relevant results than traditional keyword-based searches
3. Increased sales: By making it easier for customers to find the products they want, AI dress search can drive conversions and drive revenue

# Beyond the search for dresses

Combining the power of image embedding with BigQuery's robust data processing capabilities, you can create innovative AI-driven solutions that change the way you interact with visual content. From e-commerce to content moderation, the power of embedding images and big queries goes beyond searching for dresses.

Other potential applications are listed below:

• E-commerce: Product recommendations, visual search for other product categories
• Fashion Design: Trend Analysis, Design Inspiration
• Content Moderation: Identifying inappropriate content
• Copyright Pin-In Detection: Find Visually Similar Images to Protect Your Intellectual Property

For more information about embedding BigQuery, see Vector Search here.

Nivedita Kumari He is a veteran data analysis and AI expert with over 10 years of experience. In her current role, as a Google Data Analytics Customer Engineer, she is always involved with C-level executives, helping with data solutions architectural solutions and guides you on best practices for building data and machine learning solutions on Google Cloud. Nivedita holds a Masters degree in Technology Management, focusing on data analysis at the University of Illinois, Urbana-Champaign. She wants to democratize machine learning and AI, break down technological barriers, and ensure that everyone can become part of this transformative technology. She shares her knowledge and experience with the developer community by creating tutorials, guides, opinions and coding demonstrations. Connect to Nivedita on LinkedIn.