diff --git a/docs/remote_inference_blueprints/bedrock_connector_cohere_rerank_blueprint.md b/docs/remote_inference_blueprints/bedrock_connector_cohere_rerank_blueprint.md new file mode 100644 index 0000000000..d2e8199707 --- /dev/null +++ b/docs/remote_inference_blueprints/bedrock_connector_cohere_rerank_blueprint.md @@ -0,0 +1,332 @@ +# Bedrock connector blueprint example for Cohere Rerank model + +This blueprint shows how to create a connector for the [Cohere Rerank v3.5](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank.html) model on Amazon Bedrock. The Cohere Rerank model calculates relevance scores for documents with respect to a query, enabling you to reorder search results for improved accuracy in RAG pipelines and enterprise search. + +For a complete end-to-end tutorial including search pipeline setup, see [Reranking search results using Cohere Rerank on Amazon Bedrock](https://opensearch.org/docs/latest/tutorials/reranking/reranking-cohere-bedrock/). + +## 1. Add connector endpoint to trusted URLs: + +Note: no need to do this after 2.11.0 + +```json +PUT /_cluster/settings +{ + "persistent": { + "plugins.ml_commons.trusted_connector_endpoints_regex": [ + "^https://bedrock-runtime\\..*[a-z0-9-]\\.amazonaws\\.com/.*$" + ] + } +} +``` + +## 2. Create connector for Amazon Bedrock: + +### 2.1 Self-managed OpenSearch with AWS credentials + +If you are using self-managed OpenSearch, you should supply AWS credentials: + +```json +POST /_plugins/_ml/connectors/_create +{ + "name": "Amazon Bedrock Connector: Cohere Rerank", + "description": "Connector for Cohere Rerank v3.5 model on Amazon Bedrock", + "version": 1, + "protocol": "aws_sigv4", + "credential": { + "access_key": "", + "secret_key": "", + "session_token": "" + }, + "parameters": { + "service_name": "bedrock", + "endpoint": "bedrock-runtime", + "region": "", + "model_name": "cohere.rerank-v3-5:0" + }, + "actions": [ + { + "action_type": "predict", + "method": "POST", + "headers": { + "x-amz-content-sha256": "required", + "content-type": "application/json" + }, + "url": "https://${parameters.endpoint}.${parameters.region}.amazonaws.com/model/${parameters.model_name}/invoke", + "request_body": "{ \"documents\": ${parameters.documents}, \"query\": \"${parameters.query}\", \"api_version\": 2 }", + "pre_process_function": "connector.pre_process.cohere.rerank", + "post_process_function": "connector.post_process.cohere.rerank" + } + ] +} +``` + +> **Security note:** Do not commit real AWS credentials to version control. For production deployments, consider using IAM roles (section 2.2) or [AWS Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html) to manage credentials securely. + +### 2.2 AWS OpenSearch Service with IAM role + +If using the AWS OpenSearch Service, you can provide an IAM role ARN that allows access to the Amazon Bedrock service. +Refer to this [AWS doc](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html). + +```json +POST /_plugins/_ml/connectors/_create +{ + "name": "Amazon Bedrock Connector: Cohere Rerank", + "description": "Connector for Cohere Rerank v3.5 model on Amazon Bedrock", + "version": 1, + "protocol": "aws_sigv4", + "credential": { + "roleArn": "" + }, + "parameters": { + "service_name": "bedrock", + "endpoint": "bedrock-runtime", + "region": "", + "model_name": "cohere.rerank-v3-5:0" + }, + "actions": [ + { + "action_type": "predict", + "method": "POST", + "headers": { + "x-amz-content-sha256": "required", + "content-type": "application/json" + }, + "url": "https://${parameters.endpoint}.${parameters.region}.amazonaws.com/model/${parameters.model_name}/invoke", + "request_body": "{ \"documents\": ${parameters.documents}, \"query\": \"${parameters.query}\", \"api_version\": 2 }", + "pre_process_function": "connector.pre_process.cohere.rerank", + "post_process_function": "connector.post_process.cohere.rerank" + } + ] +} +``` + +Sample response: +```json +{ + "connector_id": "aB1cD2eF3gH4iJ5kL6mN" +} +``` + +### 2.3 Using custom Painless scripts (for OpenSearch versions before built-in function support) + +If your OpenSearch version does not support the built-in `connector.pre_process.cohere.rerank` and `connector.post_process.cohere.rerank` functions, you can use custom Painless scripts instead: + +```json +POST /_plugins/_ml/connectors/_create +{ + "name": "Amazon Bedrock Connector: Cohere Rerank", + "description": "Connector for Cohere Rerank v3.5 model on Amazon Bedrock", + "version": 1, + "protocol": "aws_sigv4", + "credential": { + "access_key": "", + "secret_key": "", + "session_token": "" + }, + "parameters": { + "service_name": "bedrock", + "endpoint": "bedrock-runtime", + "region": "", + "model_name": "cohere.rerank-v3-5:0" + }, + "actions": [ + { + "action_type": "predict", + "method": "POST", + "headers": { + "x-amz-content-sha256": "required", + "content-type": "application/json" + }, + "url": "https://${parameters.endpoint}.${parameters.region}.amazonaws.com/model/${parameters.model_name}/invoke", + "request_body": "{ \"documents\": ${parameters.documents}, \"query\": \"${parameters.query}\", \"api_version\": 2 }", + "pre_process_function": "\n def query_text = params.query_text;\n def text_docs = params.text_docs;\n def textDocsBuilder = new StringBuilder('[');\n for (int i=0; i