[Bug]: BEDROCK_MODEL_ID silently falls back to a US-only inference profile (i18n)

[tadmas2020]

Description

When bedrockLlmModelId is not set in config, the BEDROCK_MODEL_ID environment variable is not emitted to context-manager. The hardcoded Python default (a us. prefixed inference profile) is used instead, so the query path does not work in regions where that inference profile is unavailable.

Because the variable is not emitted at all, the model in use cannot be read from the CloudFormation template, which makes the missing configuration undetectable.

Environment

• Commit ID: Observed on 42b5c6b (tag v0.2.2). Still unfixed on e265573 (tag v0.3.1 = main); the affected lines have only moved from 546 to 610.
• Environment: ap-northeast-1 deployment

Step to reproduce

• Deploy to ap-northeast-1 without setting bedrockLlmModelId
• Inspect the context-manager task definition environment (BEDROCK_MODEL_ID is absent)
• Run a query from the Playground
• The Bedrock call targets us.anthropic.claude-sonnet-5 and fails

A root cause

When the config path was introduced for non-US regions (#94 / #93), only the embedding side was fixed and the LLM side was left behind. The two are now asymmetric on adjacent lines of the same environment block.

infra/lib/stacks/services/serve-stack.ts

...(props.bedrockLlmModelId && {
  BEDROCK_MODEL_ID: props.bedrockLlmModelId,
}),
// Query embedding + graphrag lexical retriever MUST use the same model
// doc-kg-build ingested with. Config-resolved (#94) so a non-US deploy
// can set a region-appropriate model; shared ts-shared constant is the
// fallback, keeping producers and consumers on one value.
BEDROCK_EMBED_MODEL_ID:
  props.bedrockEmbedModelId ?? DEFAULT_BEDROCK_MODEL_ID,

The embedding side is always emitted with a shared-constant fallback; the LLM side is a conditional spread.

The Python-side fallback is a literal in two places.

# packages/context-manager/src/coa_serve/clients/bedrock.py:321
self._model_id = model_id or os.environ.get("BEDROCK_MODEL_ID", "us.anthropic.claude-sonnet-5")

# packages/context-manager/src/coa_serve/config.py:222
bedrock_model_id=os.environ.get("BEDROCK_MODEL_ID", "us.anthropic.claude-sonnet-5"),

The fix has three parts.

1. Add a shared constant for the serve query LLM. None of the three existing constants in libs/ts-shared/src/constants.ts covers this path.

   Existing constant	Value	Purpose
   DEFAULT_BEDROCK_MODEL_ID	us.cohere.embed-v4:0	Embeddings
   DEFAULT_BEDROCK_CHAT_MODEL_ID	us.anthropic.claude-haiku-4-5-20251001-v1:0	LLM text paths (source enrichment, etc.)
   DEFAULT_BEDROCK_INDUCTION_MODEL_ID	us.anthropic.claude-sonnet-5	Ontology induction, grounding rerank

   /**
    * Default Bedrock model ID for the serve query LLM (context-manager
    * BEDROCK_MODEL_ID) — MUST match the Python single source of truth.
    * Overridable per deployment via the `bedrockLlmModelId` config key (#94).
    */
   export const DEFAULT_BEDROCK_QUERY_MODEL_ID = "us.anthropic.claude-sonnet-5";

2. Add a single source of truth on the Python side as well and point both literals above at it, following the same "keep TS and Python in sync" convention documented on the three existing constants.

3. Emit BEDROCK_MODEL_ID unconditionally, matching the format and comment style of the adjacent BEDROCK_EMBED_MODEL_ID.

   // Config-resolved (#94) so a non-US deploy can set a region-appropriate
   // model; shared ts-shared constant is the fallback, keeping the CDK default
   // and the Python default on one value.
   BEDROCK_MODEL_ID:
     props.bedrockLlmModelId ?? DEFAULT_BEDROCK_QUERY_MODEL_ID,

Context

This change does not alter the model ID value. Deployments that leave the config unset get the same value as before, so a non-US region still has to set it explicitly. The goal is to restore consistency with the embedding side and to make a missing setting visible.

infra/lib/types.ts declares bedrockLlmModelId?: string (optional, with the comment "Defaults to us.anthropic.claude-sonnet-5 at runtime when omitted"), unchanged in v0.3.1.

Additional Information / References

• [Bug]: Deploying to a non-US region requires source edits — model IDs have no config path #94 [Bug]: Deploying to a non-US region requires source edits — model IDs have no config path (CLOSED) — this issue is the remaining part of [Bug]: Deploying to a non-US region requires source edits — model IDs have no config path #94; the LLM side was never fixed
• [Feature]: Embedding model ID is hardcoded to a US-only inference profile, blocking deployment outside US regions #93 [Feature]: Embedding model ID is hardcoded to a US-only inference profile (CLOSED) — the embedding-side counterpart

[tadmas2020]

Related: #201 — make preflight checks only the model IDs written in /coa/config, so a key omitted from config is never validated even though it falls back to a us. profile. Fixing that requires resolving the effective value per key, which for bedrockLlmModelId depends on the shared constant proposed here. #201 therefore has to land together with, or after, this issue.