Add Cohere Embed v4 support for Bedrock code indexing and fix credential refresh for long-running sessions

[cscvenkatmadurai]

Problem

When indexing large codebases with the Bedrock embedding provider, two issues arise:

1. Missing model: Cohere Embed v4 (cohere.embed-v4:0) is now available on AWS Bedrock but isn't supported as an embedding model for code indexing. It offers 1536-dimensional embeddings with 128k context — a significant upgrade over Cohere v3.

2. Credential expiry on large codebases: The BedrockEmbedder uses fromEnv() as the credential fallback when no AWS profile is specified. fromEnv() only reads static environment variables and doesn't support auto-refresh. For large codebases where indexing takes >1 hour, temporary AWS credentials (from SSO, STS, or credential_process like ada) expire mid-session, causing indexing failures.

User Benefit

• Access to the latest Cohere Embed v4 model for higher quality code embeddings via Bedrock
• Reliable credential auto-refresh during long-running indexing sessions, supporting AWS SSO, IAM Identity Center, ECS, EC2 IMDS, and credential_process-based profiles
• No more indexing failures on large codebases due to expired credentials

Proposed Solution

1. Add Cohere Embed v4 (cohere.embed-v4:0)

• Add model to Bedrock embedding profiles with dimension 1536
• Add v4-specific request format (embedding_types: ["float"])
• Add v4-specific response parsing (embeddings.float[0] instead of embeddings[0])
• Text-only support initially; multimodal image support planned as follow-up

2. Fix credential fallback

• Replace fromEnv() with fromNodeProviderChain() — the AWS SDK v3 default credential chain
• fromNodeProviderChain() checks env vars first (backward compatible), then SSO, INI, IMDS, ECS, etc.
• Includes built-in memoization for efficient credential caching with auto-refresh

Files Changed (3 files, ~10 lines production code + tests)

• src/services/code-index/embedders/bedrock.ts — Credential fix + Cohere v4 request/response
• src/shared/embeddingModels.ts — New model profile entry
• src/services/code-index/embedders/__tests__/bedrock.spec.ts — 5 new tests

Roadmap Alignment

This aligns with: "Expand robust support for a wide variety of AI providers and models."

Related: aws-samples/bedrock-access-gateway#212

[roomote]

I've analyzed this issue and here's my proposed implementation plan:

Changes (3 files)

1. src/shared/embeddingModels.ts - Add Cohere Embed v4 model profile

Add "cohere.embed-v4:0": { dimension: 1536, scoreThreshold: 0.4 } to the bedrock section of EMBEDDING_MODEL_PROFILES.

2. src/services/code-index/embedders/bedrock.ts - Two changes

a) Fix credential fallback (line 2, line 41):

• Change import { fromEnv, fromIni } to import { fromNodeProviderChain, fromIni } from @aws-sdk/credential-providers
• Replace fromEnv() with fromNodeProviderChain() in the constructor fallback (line 41)
• fromNodeProviderChain() checks env vars first (backward compatible), then SSO, INI, IMDS, ECS, etc., and includes built-in credential caching with auto-refresh -- fixing the expiry issue for long-running indexing sessions

b) Add Cohere v4-specific request/response handling (lines 212-255):

• In _invokeEmbeddingModel, update the cohere.embed branch to detect v4 vs v3:
  • For v4 (model.includes("embed-v4")): add embedding_types: ["float"] to the request body
  • For v4 response parsing: extract from responseBody.embeddings.float[0] instead of responseBody.embeddings[0]
  • v3 behavior remains unchanged

3. src/services/code-index/embedders/__tests__/bedrock.spec.ts - Add tests

Add tests covering:

1. Cohere v4 request format includes embedding_types: ["float"]
2. Cohere v4 response parsing reads from embeddings.float[0]
3. Cohere v3 request format remains unchanged (no embedding_types)
4. Cohere v3 response parsing still reads from embeddings[0]
5. fromNodeProviderChain is used when no profile is specified (instead of fromEnv)

Also update the existing mock to include fromNodeProviderChain alongside fromEnv (for backward compat in existing tests).

Rationale

• The @aws-sdk/credential-providers package (already at v3.922.0) exports fromNodeProviderChain, so no new dependency is needed
• The v4 request/response format differs from v3, so we need model-version detection in the existing cohere.embed branches
• All changes are backward compatible -- existing v3 behavior and fromIni profile-based auth are untouched

Please mention @roomote in a comment saying that you approve this plan, and I'll proceed with the implementation.

[cscvenkatmadurai]

Claiming

I have an implementation ready on branch feat/bedrock-cohere-v4-and-credentials with all tests passing. Will submit a PR shortly.

[cscvenkatmadurai]

Research: Cohere Embed v4 API Validation (LT-1)

Validated the Cohere Embed v4 model on Bedrock with a real API call:

Model Status

Field	Value
Model ID	cohere.embed-v4:0
Provider	Cohere
Input Modalities	TEXT, IMAGE
Output Modalities	EMBEDDING
Inference Types	ON_DEMAND, INFERENCE_PROFILE
Lifecycle Status	ACTIVE

API Test

Request:

{"texts": ["hello world"], "input_type": "search_document", "embedding_types": ["float"]}

Response:

{
  "embeddings": {"float": [[-0.0189, 0.0125, -0.0135, 0.0136, -0.0027, ...]]},
  "response_type": "embeddings_by_type"
}

• Default dimension: 1536
• v4 nests embeddings under the type key (float), unlike v3's flat {embeddings: [[...]]}
• Multimodal (TEXT + IMAGE) — text-only supported in this PR, image support planned as follow-up

[cscvenkatmadurai]

Research: AWS SDK v3 Credential Caching Behavior (LT-3)

Tested how fromIni() and fromNodeProviderChain() handle credential caching with credential_process-based profiles (like Amazon's ada tool).

Key Finding: ignoreCache is irrelevant for credential_process

fromIni() always re-executes credential_process on every direct call (~1.2s per invocation), regardless of ignoreCache:

Test	Setting	Call 1	Call 2	Behavior
1	ignoreCache: true	1434ms	1203ms	Ada re-invoked both times
2	No ignoreCache	1236ms	1199ms	Ada also re-invoked both times
3	fromNodeProviderChain()	2ms	0ms	Built-in memoize(), cached

Why This Matters

• ignoreCache controls parsed INI file contents caching, NOT credential_process subprocess caching
• fromIni() has no built-in memoization — the BedrockRuntimeClient itself handles credential caching internally
• fromNodeProviderChain() wraps providers with memoize() for efficient caching + auto-refresh

Decision

• ❌ Removed ignoreCache: true from the design — unnecessary for credential_process
• ✅ Keep fromIni({ profile }) for profile-based credentials — SDK client caches internally
• ✅ Use fromNodeProviderChain() for no-profile fallback — built-in memoization + broad credential source support

[cscvenkatmadurai]

Research: Parsing Logic Validation (LT-4)

Validated the proposed parsing logic before implementation using simulated v4 and v3 responses.

Model ID Matching ✅

Model ID	Matches v4?	Matches v3?
cohere.embed-v4:0	✅ true	false
cohere.embed-english-v3	false	✅ true
cohere.embed-multilingual-v3	false	✅ true

Response Parsing ✅

Format	Expression	Result
v4: {embeddings: {float: [[...]]}}	embeddings?.float?.[0]	✅ Extracts array
v3: {embeddings: [[...]]}	embeddings[0]	✅ Extracts array

⚠️ Critical: If/Else Ordering

"cohere.embed-v4:0".startsWith("cohere.embed") is true — so the v4-specific check MUST come before the generic Cohere check in the if/else chain. Otherwise v4 models get routed through the v3 code path with wrong request/response format.

// ✅ CORRECT order:
if (model.startsWith("cohere.embed-v4")) { /* v4 format */ }
else if (model.startsWith("cohere.embed")) { /* v3 format */ }

// ❌ WRONG order (v4 would never match):
if (model.startsWith("cohere.embed")) { /* v3 format - catches v4 too! */ }
else if (model.startsWith("cohere.embed-v4")) { /* never reached */ }

Request Body Difference

Model	v4 adds	Body
v4	embedding_types: ["float"]	{"texts":[...],"input_type":"search_document","embedding_types":["float"]}
v3	(nothing extra)	{"texts":[...],"input_type":"search_document"}

All 9 test assertions passed. Implementation matches design exactly.

[cscvenkatmadurai]

Detailed Design

Overview

Two targeted changes to the Bedrock embedder in the code indexing system:

1. Add Cohere Embed v4 model (cohere.embed-v4:0) — 1536-dim, 128k context, text-only for now
2. Fix credential refresh — replace fromEnv() with fromNodeProviderChain() for reliable auto-refresh

Architecture

ConfigManager → ServiceFactory → BedrockEmbedder → BedrockRuntimeClient → AWS Bedrock API
                     ↑                    ↑
              embeddingModels.ts    bedrock.ts (changes here)

Component 1: Model Profiles (src/shared/embeddingModels.ts)

bedrock: {
    // ... existing models ...
    // Cohere Embed v4 (supports only text for now; multimodal image support planned)
    "cohere.embed-v4:0": { dimension: 1536, scoreThreshold: 0.4 },
    // Cohere Embed v3 models
    "cohere.embed-english-v3": { dimension: 1024, scoreThreshold: 0.4 },
    "cohere.embed-multilingual-v3": { dimension: 1024, scoreThreshold: 0.4 },
}

Component 2: Credential Fix (src/services/code-index/embedders/bedrock.ts)

// BEFORE:
import { fromEnv, fromIni } from "@aws-sdk/credential-providers"
const credentials = this.profile ? fromIni({ profile: this.profile }) : fromEnv()

// AFTER:
import { fromIni, fromNodeProviderChain } from "@aws-sdk/credential-providers"
const credentials = this.profile ? fromIni({ profile: this.profile }) : fromNodeProviderChain()

fromNodeProviderChain() checks env vars first (backward compatible), then SSO, INI, IMDS, ECS — with built-in memoize() for auto-refresh.

Component 3: v4 Request/Response (src/services/code-index/embedders/bedrock.ts)

Request (v4 adds embedding_types):

} else if (model.startsWith("cohere.embed-v4")) {
    requestBody = {
        texts: [text],
        input_type: "search_document",
        embedding_types: ["float"],  // NEW for v4
    }
} else if (model.startsWith("cohere.embed")) {
    // Existing v3 format (unchanged)
    requestBody = { texts: [text], input_type: "search_document" }
}

Response (v4 nests under type key):

} else if (model.startsWith("cohere.embed-v4")) {
    // v4: { embeddings: { float: [[...]] } }
    return { embedding: responseBody.embeddings?.float?.[0] || responseBody.embeddings?.[0] }
} else if (model.startsWith("cohere.embed")) {
    // v3: { embeddings: [[...]] }
    return { embedding: responseBody.embeddings[0] }
}

Out of Scope

• Multimodal/image embedding: Requires pipeline-wide changes (scanner, file watcher, parsers)
• Qwen on Bedrock: Only has text generation models, not embeddings
• Configurable dimensions: v4 supports 256/512/1024/1536 but we use default 1536
• Retry on auth failure: SDK client handles auto-refresh; existing 3-retry backoff covers edge cases

Testing

• 5 new unit tests (credential selection, v4 request/response, v3 regression)
• All 143 related tests pass
• TypeScript compiles cleanly, 14/14 packages lint clean

[cscvenkatmadurai]

Implementation Plan

Checklist (all completed ✅)

• Step 0: Learning tasks — validate assumptions before coding
  • LT-1: Verify Cohere v4 model access on Bedrock ✅
  • LT-3: Verify ignoreCache behavior with credential_process ✅
  • LT-4: Validate parsing logic with simulated responses ✅
• Step 1: Fix credential provider — replace fromEnv() → fromNodeProviderChain()
• Step 2: Add Cohere Embed v4 model profile (cohere.embed-v4:0, dim 1536)
• Step 3: Add v4 request/response handling in _invokeEmbeddingModel()
• Step 4: Update unit tests for credential changes (mock fromNodeProviderChain instead of fromEnv)
• Step 5: Add unit tests for Cohere v4 (request format, response parsing, v3 regression)
• Step 6: Post-implementation validation (all 143 tests pass, TS compiles, 14/14 lint clean)

Files Changed (3 files, +153/-6 lines)

File	Changes
src/services/code-index/embedders/bedrock.ts	Import fix + credential fallback + v4 request/response branches
src/shared/embeddingModels.ts	Added cohere.embed-v4:0 profile entry
src/services/code-index/embedders/__tests__/bedrock.spec.ts	5 new tests: credential selection (2), v4 embedding (2), v3 regression (1)

Test Results

bedrock.spec.ts:          30 passed (30)  ✅
service-factory.spec.ts:  73 passed (73)  ✅
config-manager.spec.ts:   40 passed (40)  ✅
TypeScript compile:       0 errors        ✅
Lint (14 packages):       0 warnings      ✅

Draft PR

#11824 — ready for review after assignment