Fix: Remove unnecessary max_tokens cap in AnthropicClient
Removes the hacky min() workaround that was capping max_tokens to DEFAULT_MAX_TOKENS (8192) in the AnthropicClient. This fix allows the client to respect the max_tokens parameter passed by callers, particularly for edge extraction operations that may require higher token limits (e.g., 16384). The new implementation aligns with how other LLM clients (OpenAI, Gemini) handle max_tokens by using the provided value or falling back to the instance max_tokens without an arbitrary cap. Resolves TODO in anthropic_client.py:207-208.
This commit is contained in:
parent
8d99984204
commit
db7c179991
1 changed files with 3 additions and 6 deletions
|
|
@ -204,12 +204,9 @@ class AnthropicClient(LLMClient):
|
|||
user_messages = [{'role': m.role, 'content': m.content} for m in messages[1:]]
|
||||
user_messages_cast = typing.cast(list[MessageParam], user_messages)
|
||||
|
||||
# TODO: Replace hacky min finding solution after fixing hardcoded EXTRACT_EDGES_MAX_TOKENS = 16384 in
|
||||
# edge_operations.py. Throws errors with cheaper models that lower max_tokens.
|
||||
max_creation_tokens: int = min(
|
||||
max_tokens if max_tokens is not None else self.config.max_tokens,
|
||||
DEFAULT_MAX_TOKENS,
|
||||
)
|
||||
# Use the provided max_tokens or fall back to the instance max_tokens
|
||||
# This aligns with how other LLM clients handle max_tokens
|
||||
max_creation_tokens: int = max_tokens or self.max_tokens
|
||||
|
||||
try:
|
||||
# Create the appropriate tool based on whether response_model is provided
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue