Repository navigation
Throw ToolExecutionStopped from stopped HTTP responses - #390
Merged
Merged
Conversation
A session with tools that asked MLXLanguageModel or LlamaLanguageModel for a Generable type went straight to constrained JSON generation, so the model never called a tool. Run the tool loop for every response type, then generate the structured content from the tool calls and outputs. Resolves #388.
The OpenAI, Open Responses, Anthropic, Gemini, and Ollama models now end a stopped response with the shared stoppedResponse helper, and their streams throw ToolExecutionStopped from stoppedSnapshot, when the response type can't be empty. Previously they threw a decoding error, and Ollama crashed on a force cast for any type other than String.
A stopped respond call and a stopped stream now end with the same empty values: an empty string for String, and otherwise an empty object, null, or an empty array. A stopped number or Boolean response throws ToolExecutionStopped in both paths, because zero or false would look like the model's answer.
# Conflicts: # Sources/AnyLanguageModel/ToolExecution.swift # Tests/AnyLanguageModelTests/LlamaLanguageModelTests.swift # Tests/AnyLanguageModelTests/LocalGenerationUsageTests.swift # Tests/AnyLanguageModelTests/MLXLanguageModelTests.swift
mattt
marked this pull request as ready for review
November 5, 2026 19:22
A stopped number or Boolean stream throws ToolExecutionStopped and can end without a snapshot, and the session counts streamed usage only from snapshots. The error now carries the requests' usage, and the session counts it against what the stream's snapshots reported, so each request counts once. A handoff to another model adds the entries and usage of the rounds before it to the error.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes the HTTP models end a stopped structured response the same way as MLX and llama.cpp. When a tool execution delegate returns
.stopand the response type has required properties, the OpenAI, Open Responses, Anthropic, and Gemini models threw a decoding error, and Ollama crashed on a force cast for any type other thanString.Each model's
respondstop path now uses the sharedstoppedResponsehelper, which moves to its own file, andstoppedSnapshotthrowsLanguageModelSession.ToolExecutionStoppedwith the provider name and the stopped tool calls. In both paths, aStringresponse ends with an empty string, and other types end with the first of{},null, or[]that they decode from; a stopped number or Boolean response throws, because zero orfalsewould look like the model's answer.A stream that throws this error can end without a snapshot, so the error also carries the usage of the stream's requests. The session counts that usage against what the snapshots reported, so each request counts once. After a model handoff, the error includes the entries and usage of the rounds before the handoff.