feat(bedrock): enable prompt caching toggle for custom ARN and default to ON #10697
+5
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #10576
Description
This PR attempts to address Issue #10576. Feedback and guidance are welcome.
Changes:
Enable prompt caching toggle for custom ARN users: Changed
supportsPromptCachefromfalsetotruefor custom ARN model info. Custom ARN is treated as an advanced option where users know what they are doing, and their custom ARN often points to newer Claude models that support prompt caching.Default prompt caching to ON: Changed the default value of
awsUsePromptCachefromfalsetotrueby using?? trueinstead of|| false. This ensures users get cost savings and latency improvements by default, while the nullish coalescing operator (??) properly respects explicitly setfalsevalues.Files modified:
webview-ui/src/components/ui/hooks/useSelectedModel.ts- Enable supportsPromptCache for custom-arnwebview-ui/src/components/settings/providers/Bedrock.tsx- Default checkbox to trueTest Procedure
cd src && npx vitest run api/providers/__tests__/bedrock*- all 143 tests passnpx tsc --noEmit- no type errorsPre-Submission Checklist
Documentation Updates
Additional Notes
This change treats custom ARN as an advanced option where users know what they are doing. Since custom ARNs often point to newer Claude models that support prompt caching, it makes sense to show the toggle and let users decide. This is a re-implementation of the closed PR #10577 per request in the issue comments.
Important
Enable prompt caching toggle for custom ARN users and default prompt caching to ON in
useSelectedModel.tsandBedrock.tsx.supportsPromptCachefor custom ARN inuseSelectedModel.ts, allowing users to toggle prompt caching.awsUsePromptCachetotrueinBedrock.tsx, using?? trueto respect explicitfalsevalues.useSelectedModel.ts: ChangesupportsPromptCachetotruefor custom ARN.Bedrock.tsx: Update prompt caching checkbox default totrue.This description was created by
for 62214b3. You can customize this summary. It will automatically update as commits are pushed.