AWS Bedrock Model

Connect AI Squared to AWS Bedrock Model

This guide will help you configure the AWS Bedrock Model Connector in AI Squared to access your AWS Bedrock Model Endpoint.

Prerequisites

Before proceeding, ensure you have the necessary access key, secret access key, and region from AWS.

Step-by-Step Guide to Connect to an AWS Bedrock Model Endpoint

Step 1: Navigate to AWS Console

Start by logging into your AWS Management Console.

Step 2: Locate AWS Configuration Details

Once you’re in the AWS console, you’ll find the necessary configuration details:

Access Key and Secret Access Key:
- Click on your username at the top right corner of the AWS Management Console.
- Choose “Security Credentials” from the dropdown menu.
- In the “Access keys” section, you can create or view your access keys.
- If you haven’t created an access key pair before, click on “Create access key” to generate a new one. Make sure to copy the Access Key ID and Secret Access Key as they are shown only once.
Region:
- The AWS region can be selected from the top right corner of the AWS Management Console. Choose the region where your AWS Bedrock resources is located and note down the region.
Inference Profile ARN:
- The Inference Profile ARN is in the Cross-region inference page and can be found in your selected model.
Model ID:
- The AWS Model Id can be found in your selected models catalog.

Step 3: Configure AWS Bedrock Model Connector in Your Application

Now that you have gathered all the necessary details enter the following information:

Access Key ID: Your AWS IAM user’s Access Key ID.
Secret Access Key: The corresponding Secret Access Key.
Region: The AWS region where your Bedrock model are located.
Inference Profile ARN: Inference Profile ARN for Model in AWS Bedrock.
Model ID: The Model ID.

Sample Request and Response

Jamba Models

Jamba 1.5 Large

Request:

{
  "messages": [
    {
      "role": "user",
      "content": "hello"
    }
  ],
  "max_tokens": 100
}

Response:

{
  "id": "chatcmpl",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": " Hello!",
        "tool_calls": null
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 10,
    "total_tokens": 22
  },
  "meta": {
    "requestDurationMillis": 113
  },
  "model": "jamba-1.5-large"
}

Jamba 1.5 Mini

Request:

{
  "messages": [
    {
      "role": "user",
      "content": "hello"
    }
  ],
  "max_tokens": 100
}

Response:

{
  "id": "chatcmpl",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": " Hello!",
        "tool_calls": null
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 10,
    "total_tokens": 22
  },
  "meta": {
    "requestDurationMillis": 113
  },
  "model": "jamba-1.5-mini"
}

Amazon Models

Nova Lite

Request:

{
  "inferenceConfig": {
    "max_new_tokens": 100
  },
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "output": {
    "message": {
      "content": [
        {
          "text": "Hello!"
        }
      ],
      "role": "assistant"
    }
  },
  "stopReason": "end_turn",
  "usage": {
    "inputTokens": 1,
    "outputTokens": 51,
    "totalTokens": 52,
    "cacheReadInputTokenCount": 0,
    "cacheWriteInputTokenCount": 0
  }
}

Nova Micro

Request:

{
  "inferenceConfig": {
    "max_new_tokens": 100
  },
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "output": {
    "message": {
      "content": [
        {
          "text": "Hello!"
        }
      ],
      "role": "assistant"
    }
  },
  "stopReason": "end_turn",
  "usage": {
    "inputTokens": 1,
    "outputTokens": 51,
    "totalTokens": 52,
    "cacheReadInputTokenCount": 0,
    "cacheWriteInputTokenCount": 0
  }
}

Nova Pro

Request:

{
  "inferenceConfig": {
    "max_new_tokens": 100
  },
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "output": {
    "message": {
      "content": [
        {
          "text": "Hello!"
        }
      ],
      "role": "assistant"
    }
  },
  "stopReason": "end_turn",
  "usage": {
    "inputTokens": 1,
    "outputTokens": 51,
    "totalTokens": 52,
    "cacheReadInputTokenCount": 0,
    "cacheWriteInputTokenCount": 0
  }
}

Titan Text G1 - Premier

Request:

{
  "inputText": "hello",
  "textGenerationConfig": {
    "maxTokenCount": 100,
    "stopSequences": []
  }
}

Response:

{
  "inputTextTokenCount": 3,
  "results": [
    {
      "tokenCount": 13,
      "outputText": "\nBot: Hi there! How can I help you?",
      "completionReason": "FINISH"
    }
  ]
}

Titan Text G1 - Express

Request:

{
  "inputText": "hello",
  "textGenerationConfig": {
    "maxTokenCount": 100,
    "stopSequences": []
  }
}

Response:

{
  "inputTextTokenCount": 3,
  "results": [
    {
      "tokenCount": 13,
      "outputText": "\nBot: Hi there! How can I help you?",
      "completionReason": "FINISH"
    }
  ]
}

Titan Text G1 - Lite

Request:

{
  "inputText": "hello",
  "textGenerationConfig": {
    "maxTokenCount": 100,
    "stopSequences": []
  }
}

Response:

{
  "inputTextTokenCount": 3,
  "results": [
    {
      "tokenCount": 13,
      "outputText": "\nBot: Hi there! How can I help you?",
      "completionReason": "FINISH"
    }
  ]
}

Anthropic Models

Claude 3.7 Sonnet

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hello!"
    }
  ],
  "model": "claude-3-7-sonnet-20250219",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 12,
    "output_tokens": 6
  }
}

Claude 3.5 Haiku

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_02ABC1234",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hi there!"
    }
  ],
  "model": "claude-3-5-haiku-20240305",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 9,
    "output_tokens": 5
  }
}

Claude 3.5 Sonnet v2

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_03XYZ5678",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hello, friend!"
    }
  ],
  "model": "claude-3-5-sonnet-20240315-v2",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 9,
    "output_tokens": 6
  }
}

Claude 3.5 Sonnet

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_bdrk",
  "type": "message",
  "role": "assistant",
  "model": "claude-3-5-sonnet-20240307",
  "content": [
    {
      "type": "text",
      "text": "Hello!"
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 8,
    "output_tokens": 12
  }
}

Claude 3 Opus

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_05OPQ2345",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hey there!"
    }
  ],
  "model": "claude-3-opus-20240229",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 9,
    "output_tokens": 6
  }
}

Claude 3 Haiku

Request:

{
  "anthropic_version": "bedrock-2023-05-31",
  "max_tokens": 100,
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "msg_06RST6789",
  "type": "message",
  "role": "assistant",
  "content": [
    {
      "type": "text",
      "text": "Hello, world!"
    }
  ],
  "model": "claude-3-haiku-20240305",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 9,
    "output_tokens": 6
  }
}

Cohere Models

Command R

Request:

{ 
  "message": "Hi" 
}

Response:

{
  "response_id": "123D7",
  "text": "Hi there!",
  "generation_id": "e70d12",
  "chat_history": [
    {
      "role": "USER",
      "message": "Hi"
    },
    {
      "role": "CHATBOT",
      "message": "Hi there!"
    }
  ],
  "finish_reason": "COMPLETE"
}

Command R+

Request:

{ 
  "message": "Hi" 
}

Response:

{
  "response_id": "123D7",
  "text": "Hi there!",
  "generation_id": "e70d12",
  "chat_history": [
    {
      "role": "USER",
      "message": "Hi"
    },
    {
      "role": "CHATBOT",
      "message": "Hi there!"
    }
  ],
  "finish_reason": "COMPLETE"
}

Command Light

Request:

{
  "prompt": "hello",
  "max_tokens": 100
}

Response:

{
  "id": "5e820f61f54d",
  "generations": [
    {
      "id": "5e820f61f54d",
      "text": " Hello!",
      "finish_reason": "COMPLETE"
    }
  ],
  "prompt": "hello"
}

Command

Request:

{
  "prompt": "hello",
  "max_tokens": 100
}

Response:

{
  "id": "5e820f61f54d",
  "generations": [
    {
      "id": "5e820f61f54d",
      "text": " Hello!",
      "finish_reason": "COMPLETE"
    }
  ],
  "prompt": "hello"
}

DeepSeek Models

Meta Models

Llama 3.3 70B Instruct

Llama 3.2 11B Instruct

Llama 3.2 1B Instruct

Llama 3.2 3B Instruct

Llama 3.2 90B Instruct

Llama 3.1 70B Instruct

Llama 3.1 8B Instruct

Llama 3 70B Instruct

Llama 3 8B Instruct

Mistral AI Models

Pixtral Large (25.02)

Request:

{
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "hello"
        }
      ]
    }
  ]
}

Response:

{
  "id": "model_id",
  "object": "chat.completion",
  "created": 1745858024,
  "model": "pixtral-large-2502",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 33,
    "total_tokens": 38
  }
}

Mistral Large (24.02)

Mistral 7B Instruct

Mixtral 8x7B Instruct

Mistral Small (24.02)

AI/ML Sources

Vector Search Sources

AI Modeling

Data Apps

Connect AI Squared to AWS Bedrock Model

Prerequisites

Step-by-Step Guide to Connect to an AWS Bedrock Model Endpoint

Step 1: Navigate to AWS Console

Step 2: Locate AWS Configuration Details

Step 3: Configure AWS Bedrock Model Connector in Your Application

Sample Request and Response

AI/ML Sources

Vector Search Sources

AI Modeling

Data Apps

​Connect AI Squared to AWS Bedrock Model

​Prerequisites

​Step-by-Step Guide to Connect to an AWS Bedrock Model Endpoint

​Step 1: Navigate to AWS Console

​Step 2: Locate AWS Configuration Details

​Step 3: Configure AWS Bedrock Model Connector in Your Application

​Sample Request and Response

Connect AI Squared to AWS Bedrock Model

Prerequisites

Step-by-Step Guide to Connect to an AWS Bedrock Model Endpoint

Step 1: Navigate to AWS Console

Step 2: Locate AWS Configuration Details

Step 3: Configure AWS Bedrock Model Connector in Your Application

Sample Request and Response