fix: handle reasoning_effort gracefully across models (#1447)
<!-- .github/pull_request_template.md --> ## Description The async LLM client fails with non-reasoning models like gpt-4o with the error: `Completion error: litellm.BadRequestError: OpenAIException - Unrecognized request argument supplied: reasoning_effort`. This PR add a sets the `drop_params` config to True to drop all unsupported model configs instead of catching with a try-except block. Additionally, the `reasoning_effort` wasn't being set for the sync client. It adds the parameter for both async & sync for consistency. ## Type of Change <!-- Please check the relevant option --> - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Code refactoring - [ ] Performance improvement - [ ] Other (please specify): ## Changes Made - Set `litellm.drop_params=True` to auto-drop unsupported parameters - Changed `reasoning_effort` from `extra_body` to a direct parameter - Add `reasoning_effort` to the sync client - Removed redundant retry for `reasoning_effort` ## Testing <!-- Describe how you tested your changes --> ## Screenshots/Videos (if applicable) <!-- Add screenshots or videos to help explain your changes --> ## Pre-submission Checklist <!-- Please check all boxes that apply before submitting your PR --> - [ ] **I have tested my changes thoroughly before submitting this PR** - [ ] **This PR contains minimal changes necessary to address the issue/feature** - [ ] My code follows the project's coding standards and style guidelines - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have added necessary documentation (if applicable) - [ ] All new and existing tests pass - [ ] I have searched existing PRs to ensure this change hasn't been submitted already - [ ] I have linked any relevant issues in the description - [ ] My commits have clear and descriptive messages ## Related Issues <!-- Link any related issues using "Fixes #issue_number" or "Relates to #issue_number" --> ## Additional Notes <!-- Add any additional notes, concerns, or context for reviewers --> ## DCO Affirmation I affirm that all code in every commit of this pull request conforms to the terms of the Topoteretes Developer Certificate of Origin.
This commit is contained in:
commit
38b83a5ec1
2 changed files with 7 additions and 29 deletions
|
|
@ -178,7 +178,7 @@ You can also cognify your files and query using cognee UI.
|
|||
|
||||
### Running the UI
|
||||
|
||||
Try cognee UI by running ``` cognee-cli -ui ``` command on your terminal.
|
||||
Try cognee UI by setting LLM_API_KEY and running ``` cognee-cli -ui ``` command on your terminal.
|
||||
|
||||
## Understand our architecture
|
||||
|
||||
|
|
|
|||
|
|
@ -29,6 +29,9 @@ observe = get_observe()
|
|||
|
||||
logger = get_logger()
|
||||
|
||||
# litellm to drop unsupported params, e.g., reasoning_effort when not supported by the model.
|
||||
litellm.drop_params = True
|
||||
|
||||
|
||||
class OpenAIAdapter(LLMInterface):
|
||||
"""
|
||||
|
|
@ -132,39 +135,13 @@ class OpenAIAdapter(LLMInterface):
|
|||
api_version=self.api_version,
|
||||
response_model=response_model,
|
||||
max_retries=self.MAX_RETRIES,
|
||||
extra_body={"reasoning_effort": "minimal"},
|
||||
reasoning_effort="minimal",
|
||||
)
|
||||
except (
|
||||
ContentFilterFinishReasonError,
|
||||
ContentPolicyViolationError,
|
||||
InstructorRetryException,
|
||||
) as error:
|
||||
if (
|
||||
isinstance(error, InstructorRetryException)
|
||||
and "content management policy" not in str(error).lower()
|
||||
):
|
||||
logger.debug(
|
||||
"LLM Model does not support reasoning_effort parameter, trying call without the parameter."
|
||||
)
|
||||
return await self.aclient.chat.completions.create(
|
||||
model=self.model,
|
||||
messages=[
|
||||
{
|
||||
"role": "user",
|
||||
"content": f"""{text_input}""",
|
||||
},
|
||||
{
|
||||
"role": "system",
|
||||
"content": system_prompt,
|
||||
},
|
||||
],
|
||||
api_key=self.api_key,
|
||||
api_base=self.endpoint,
|
||||
api_version=self.api_version,
|
||||
response_model=response_model,
|
||||
max_retries=self.MAX_RETRIES,
|
||||
)
|
||||
|
||||
):
|
||||
if not (self.fallback_model and self.fallback_api_key):
|
||||
raise ContentPolicyFilterError(
|
||||
f"The provided input contains content that is not aligned with our content policy: {text_input}"
|
||||
|
|
@ -246,6 +223,7 @@ class OpenAIAdapter(LLMInterface):
|
|||
api_base=self.endpoint,
|
||||
api_version=self.api_version,
|
||||
response_model=response_model,
|
||||
reasoning_effort="minimal",
|
||||
max_retries=self.MAX_RETRIES,
|
||||
)
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue