Twelvelabs video understanding model now available on Amazon Bedrock

It assumes that Twelvelabs' video understanding model is coming to Amazon Bedrock earlier this year. Today, models can search for videos, categorize scenes, and extract accurate and reliable insights.

Twelvelabs has introduced Marengo. It introduced a video embedding model that is skilled in performing tasks such as searching and classification, and a video language model that allows you to generate text based on video data. These models are trained on Amazon Sagemaker HyperPod and provide groundbreaking video analytics that provides text summaries, metadata generation, and creative optimization.

With Amazon Bedrock's Twelvelabs model, you can find specific moments using natural language video search features, such as “Show the first touchdown of the game” and “Find the scene where the main character meets first”, and jump straight to those exact moments. You can also build applications to understand video content by generating descriptive text such as titles, topics, overviews, chapters, highlights and more to discover insights and connections without the need for predefined labels or categories.

For example, you can find recurring themes in customer feedback, or spot product usage patterns that were previously unclear. Whether you have hundreds or thousands of hours of video content, you can turn your entire library into a searchable knowledge resource, while maintaining enterprise-grade security and performance.

Check out the Marengo and Pegasus video released by Twelvelabs.

These models allow you to transform video workflows across the industry. Media producers and editors can instantly find specific scenes and dialogues. This means you can focus on storytelling rather than sifting through hours of footage. While marketing teams streamline advertising workflows by quickly personalizing content and resonating with different audiences, security teams use technology to actively identify potential risks by discovering patterns across multiple video feeds.

Get started with Amazon Bedrock's Twelvelabs model
Before you begin, if you are new to using the Twelvelabs model, go to the Amazon Bedrock Console and select it Model Access Navigation pane at the bottom left. To access the latest Twelvelabs models, request access Marengo embedded 2.7 or Pegasus 1.2 in Twelvelabs Depending on the region available.

To use the Twelvelabs model with Amazon Bedrock, please select Chat/Text Playground under test Left navigation pane. choose Select a model,choice Twelvelabs As a category Pegasus As a model and then select Apply.

To search or generate text from a video, you must upload the video to an Amazon Simple Storage Service (Amazon S3) Bucket or an input Base64 format video string.

Use sample video generated by Amazon Nova Reel as a prompt.

Enter S3 URI Uploaded videos and S3 bucket ownerand then run the prompt: “Please tell me about the video in the timeline.”

You can get the following model response:

The video opens with an underwater scene featuring a sandy seabed and swaying seaweed, creating a serene atmosphere. At around 5 seconds, a raccoon appears, swimming towards the camera with a curious expression. By 10 seconds, the raccoon is seen swimming near the seabed. The focus then shifts to a group of koi fish swimming gracefully near the seabed from 11 to 13 seconds.

At 14 seconds, the scene transitions to an ukulele resting on the seabed, with the camera zooming in to highlight its details. The raccoon reappears at 20 seconds, swimming among the koi fish, and by 23 seconds, it begins to play the ukulele. The raccoon continues to play the ukulele from 29 to 41 seconds, with the koi fish swimming around it, adding a lively and colorful touch to the underwater setting.

From 42 to 49 seconds, the raccoon is shown playing the ukulele with the koi fish swimming around it. The video concludes with a screen displaying "Video generated with Amazon Novo Reel 1.1" and "Available in Amazon Bedrock" from 50 to 54 seconds, followed by the AWS logo from 55 to 57 seconds.

The Twelvelabs model can be easily integrated into your application using the Amazon Bedrock Invoke API. It provides a unified interface for AI interactions in conversations.

Here is an example of how to use the AWS SDK for Python (Boto3) in the TWELVELABS PEGASUS model:

import boto3
import json
import base64

AWS_REGION = "us-west-2"
MODEL_ID = "us.twelvelabs.pegasus-1-2-v1:0" # OR "eu.twelvelabs.pegasus-1-2-v1:0" if it is doing cross region inference in europe
VIDEO_PATH = "sample.mp4"

def read_file(file_path: str) -> str:
    """Read a file and return as base64 encoded string."""
    try:
        with open(file_path, 'rb') as file:
            file_content = file.read()
            return base64.b64encode(file_content).decode('utf-8')
    except Exception as e:
        raise Exception(f"Error reading file {file_path}: {str(e)}")


bedrock_runtime = boto3.client(
    service_name="bedrock-runtime",
    region_name=AWS_REGION
)

request_body = {
    "inputPrompt": "tell me about the video",
    "mediaSource": {
        "base64String": read_file(VIDEO_PATH)
    }
}

response = bedrock_runtime.invoke_model(
    modelId=MODEL_ID,
    body=json.dumps(request_body),
    contentType="application/json",
    accept="application/json"
)

response_body = json.loads(response['body'].read())
print(json.dumps(response_body, indent=2))

The Twelvelabs Marengo Embed 2.7 model generates vector embeddings from video, text, audio, or image inputs. These embeddings can be used for similarity search, clustering, and other machine learning (ML) tasks. This model supports asynchronous inference via the Bedrock Startasyncinvoke API.

For video sources, you can request the JSON format for the Twelvelabs Marengo Embed 2.7 model as follows: StartAsyncInvoke API.

{
    "modelId": "twelvelabs.marengo-embed-2-7-v1:0",
    "modelInput": {
        "inputType": "video",
        "mediaSource": {
            "s3Location": {
                "uri": "s3://your-video-object-s3-path",
                "bucketOwner": "your-video-object-s3-bucket-owner-account"
            }
        }
    },
    "outputDataConfig": {
        "s3OutputDataConfig": {
            "s3Uri": "s3://your-bucket-name"
        }
    }
}

You can deliver responses to the specified S3 position.

{
    "embedding": [0.345, -0.678, 0.901, ...],
    "embeddingOption": "visual-text",
    "startSec": 0.0,
    "endSec": 5.0
}

To get started, take a look at a wide range of code examples for multiple use cases and various programming languages. For more information, see Twelvelabs Pegasus 1.2 and Twelvelabs Marengo embedded 2.7 in their AWS documentation.

Now available
The Twelvelabs model is generally available in Amazon Bedrock: Marengo models in the US East (N. Virginia), Europe (Ireland), Asia Pacific (Seoul) regions, and Pegasus models in the US West (Oregon), and the European (Ireland) regions where inferences can be made beyond territory from the US and European territories. Check the entire region list for future updates. For more information, see the Twelvelabs and Amazon Bedrock Pricing pages on the Amazon Bedrock product page.

Try the Amazon Bedrock console now to your Twelvelabs model and send feedback to AWS Re:Amazon Bedrock posts or regular AWS support contacts.

– Channy

Updated July 16, 2025 – Screenshots and code parts have been revised.