14.3 HeatWave GenAI External LLMs

HeatWave GenAI supports external LLMs provided by Amazon Bedrock. When external LLMs are used, data leaves your HeatWave Cluster to be processed by Amazon Bedrock.

The following are the external LLMs provided by Amazon Bedrock that are currently supported by HeatWave on AWS:

• cohere.command-r-v1:0
• cohere.command-r-plus-v1:0
• cohere.command-text-v14
• cohere.command-light-text-v14
• meta.llama3-70b-instruct-v1:0
• meta.llama3-1-70b-instruct-v1:0
• meta.llama3-1-405b-instruct-v1:0

Note:

Not all LLMs provided by Amazon Bedrock are available in all AWS regions. Also, HeatWave GenAI does not support cross-region inference, which means an external LLM provided by Amazon Bedrock is not supported by HeatWave GenAI in your AWS region if it is only accessible through cross-region inference in that region. See Supported foundation models in Amazon Bedrock for details.

To use these models with the HeatWave GenAI routines, you need:

• An IAM policy to access the Amazon Bedrock service
• An IAM Role to access the Amazon Bedrock service

Parent topic: HeatWave GenAI