vercel/ai

Description

AWS Bedrock with the new Kimi K2 Thinking model (https://aws.amazon.com/blogs/aws/amazon-bedrock-adds-fully-managed-open-weight-models/) seems to end the conversation abruptly. It looks like it sometimes parses tool calls as "text-delta" instead of an actual tool call, which shows up as text in my chat app.

Here is a sample log:

{"type":"start","messageId":"yhfl7aguy69p78kdaupavgr0"}{"type":"start","messageId":"zdeo0nspwe3nd4cv0n9pl4q9"}{"type":"start-step"}{"type":"reasoning-start","id":"1"}{"type":"reasoning-delta","id":"1","delta":" The user is asking if they have"}{"type":"reasoning-delta","id":"1","delta":" any \"cursor files\" in their repository. This likely refers to Cursor editor configuration files (like .cursorrules) or possibly files related to cursor/cursor-based operations in the code.\n\nLet me"}{"type":"reasoning-delta","id":"1","delta":" search for files with \""}{"type":"reasoning-delta","id":"1","delta":"cursor\" in the name or path."}{"type":"reasoning-end","id":"1"}{"type":"tool-input-start","toolCallId":"tooluse_OEppyvDpRkWByqqI-Ixz0A","toolName":"glob_search"}{"type":"tool-input-delta","toolCallId":"tooluse_OEppyvDpRkWByqqI-Ixz0A","inputTextDelta":"{\"pattern\": \"**/*cursor*\"}"}{"type":"tool-input-available","toolCallId":"tooluse_OEppyvDpRkWByqqI-Ixz0A","toolName":"glob_search","input":{"pattern":"**/*cursor*"}}{"type":"text-start","id":"4"}{"type":"text-delta","id":"4","delta":"{\"pattern\": "}{"type":"text-delta","id":"4","delta":"\"**/*cursor*\"}"}{"type":"text-end","id":"4"}{"type":"tool-output-available","toolCallId":"tooluse_OEppyvDpRkWByqqI-Ixz0A","output":{"files":["FILE1","FILE2"]}}{"type":"finish-step"}{"type":"start-step"}{"type":"reasoning-start","id":"1"}{"type":"reasoning-delta","id":"1","delta":" Yes, you have cursor files in your repo:\n1. FILE1\n2. FILE2\n\nLet me read the"}{"type":"reasoning-delta","id":"1","delta":" TypeScript file to see what it contains."}{"type":"reasoning-end","id":"1"}{"type":"text-start","id":"2"}{"type":"text-delta","id":"2","delta":"{\"target_file\": "}{"type":"text-delta","id":"2","delta":"\"FILE1\"}  "}{"type":"text-delta","id":"2","delta":"<|tool_calls_section_end|>"}{"type":"text-end","id":"2"}{"type":"finish-step"}{"type":"finish","finishReason":"stop"}

I also had to add a custom transform because every time it made a tool call, it would add the tool markers in the "text" field as well. Here's a sample:

   () =>
            new TransformStream({
              transform(chunk, controller) {
                if (chunk.type === "text-delta") {
                  // Remove Kimi K2 tool call markers
                  const cleanedText = chunk.text
                    .replace(/<\|tool_call_begin\|>/g, "")
                    .replace(/<\|tool_call_argument_begin\|>/g, "")
                    .replace(/<\|tool_call_end\|>/g, "")
                    .replace(/functions\.\w+:\d+/g, "")
                    .trim()

                  // Only enqueue if there's actual text left after cleaning
                  if (cleanedText) {
                    controller.enqueue({
                      ...chunk,
                      text: cleanedText,
                    })
                  }
                } else {
                  // Pass through all other chunk types unchanged
                  controller.enqueue(chunk)
                }
              },
            }),