Commit graph

206 commits

Author SHA1 Message Date
Lucas Oliveira
0b40b606c2
fix: chat blocking async requests (#619)
* fix chat blocking async requests

* fix delete user conversation not being async
2025-12-05 17:48:51 -03:00
Lucas Oliveira
9b08f1fcee create lightweight health check based on query param 2025-12-03 17:11:51 -03:00
Edwin Jose
c83ea54745
Merge branch 'main' into onboarding-fqs 2025-12-02 16:52:28 -05:00
Lucas Oliveira
ed77a9cba0
feat: add Scarf telemetry to OpenRAG (#548)
* added telemetry utils

* added telemetry to openrag

* fixed http timeout

* Added OS and GPU logging

* Track task fail and cancel

* Updated messages to be more readable
2025-12-02 16:52:51 -03:00
Sebastián Estévez
48d5e7c45e
Merge branch 'main' into onboarding-fqs 2025-12-02 13:15:31 -05:00
phact
0a2a991519 search embeding model support for watsonx models 2025-12-02 11:58:01 -05:00
phact
0fc0be855c onboarding filter creation + sticky filters 2025-12-02 02:31:59 -05:00
Eric Hare
7a11ebb189
fix: Respect selected embedding model when restoring flow 2025-12-01 14:04:28 -08:00
phact
c106438d75 wip 2025-12-01 16:36:35 -05:00
Lucas Oliveira
cebb38eb04
fix: make backend reset langflow flows after upgrade, backup flows every 5 mins (#483)
* Changed backend to mount config at volume

* update lock

* Changed backend to reapply settings after detecting that flow is reset

* Added periodic backup for flows, make better reset

* tui warning

* Changed settings page to alert user that he has to disable lock flow

* Changed flows to be locked

* Do periodic backup only if onboarding is done

* Change backup function to only back up flows if flow lock is disabled

* Added session manager to reapply all settings

---------

Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
2025-11-26 22:53:33 -03:00
Edwin Jose
998d8b9195 Add fail-safe mode to embedding and OpenSearch components
Introduces a 'fail_safe_mode' option to the Embedding Model and OpenSearch (Multi-Model Multi-Embedding) components, allowing errors to be logged and None returned instead of raising exceptions. Refactors embedding model fetching logic for better error handling and updates component metadata, field order, and dependencies. Also adds 'className' fields and updates frontend node folder IDs for improved UI consistency.
2025-11-26 01:15:30 -05:00
Edwin Jose
0f2012bbb9 Update Langflow tweaks and add provider credentials to headers
Replaces all references to 'OpenSearchHybrid-Ve6bS' with 'OpenSearchVectorStoreComponentMultimodalMultiEmbedding-By9U4' in main.py, processors, and file service. Adds a utility for injecting provider credentials into Langflow request headers and integrates it into chat and file services for improved credential handling.
2025-11-25 22:50:38 -05:00
Edwin Jose
07b84e373a Update config import to use settings module
Replaced imports from config_manager with settings in chat_service.py and langflow_file_service.py to use get_openrag_config from config.settings. This change ensures consistency with the updated configuration structure.
2025-11-25 19:40:24 -05:00
Edwin Jose
284b982dbf Add SELECTED_EMBEDDING_MODEL global variable support
Introduces SELECTED_EMBEDDING_MODEL as a global environment variable in docker-compose files and ensures it is passed in API headers for Langflow-related services. Updates settings and onboarding logic to set this variable without triggering flow updates, improving embedding model configuration consistency across services.
2025-11-25 17:47:54 -05:00
Lucas Oliveira
141a7da339 update template when provider updates 2025-11-19 16:18:26 -03:00
Lucas Oliveira
c295431484
fix: refactor models validation to fix bugs related to ollama, watsonx and openai (#406)
* Fixed models service to try api key with first available model

* fixed ibm onboarding to not disable query when no data is available

* make ibm query disabled when not configured

* enable ollama query only when configured or endpoint present

* enable get openai models query when already configured

* just enable get from env when not configured

* Simplify ollama models validation

* fix max_tokens error on gpt 4o
2025-11-14 18:09:47 -03:00
Cole Goldsmith
1385fd5d5c
better settings form validation, grouped model selection (#383)
* better form validation, grouped model selection

* bump version

* fix fe build issue

* fix test

* change linting error

* Fixed integration tests

* fixed tests

* sample commit

---------

Co-authored-by: Lucas Oliveira <lucas.edu.oli@hotmail.com>
2025-11-11 22:39:59 -03:00
Lucas Oliveira
37faf94979
feat: adds anthropic provider, splits onboarding editing into two, support provider changing with generic llm and embedding components (#373)
* Added flows with new components

* commented model provider assignment

* Added agent component display name

* commented provider assignment, assign provider on the generic component, assign custom values

* fixed ollama not showing loading steps, fixed loading steps never being removed

* made embedding and llm model optional on onboarding call

* added isEmbedding handling on useModelSelection

* added isEmbedding on onboarding card, separating embedding from non embedding card

* Added one additional step to configure embeddings

* Added embedding provider config

* Changed settings.py to return if not embedding

* Added editing fields to onboarding

* updated onboarding and flows_service to change embedding and llm separately

* updated templates that needs to be changed with provider values

* updated flows with new components

* Changed config manager to not have default models

* Changed flows_service settings

* Complete steps if not embedding

* Add more onboarding steps

* Removed one step from llm steps

* Added Anthropic as a model for the language model on the frontend

* Added anthropic models

* Added anthropic support on Backend

* Fixed provider health and validation

* Format settings

* Change anthropic logo

* Changed button to not jump

* Changed flows service to make anthropic work

* Fixed some things

* add embedding specific global variables

* updated flows

* fixed ingestion flow

* Implemented anthropic on settings page

* add embedding provider logo

* updated backend to work with multiple provider config

* update useUpdateSettings with new settings type

* updated provider health banner to check for health with new api

* changed queries and mutations to use new api

* changed embedding model input to work with new api

* Implemented provider based config on the frontend

* update existing design

* fixed settings configured

* fixed provider health query to include health check for both the providers

* Changed model-providers to show correctly the configured providers

* Updated prompt

* updated openrag agent

* Fixed settings to allow editing providers and changing llm and embedding models

* updated settings

* changed lf ver

* bump openrag version

* added more steps

* update settings to create the global variables

* updated steps

* updated default prompt

---------

Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
2025-11-11 19:22:16 -03:00
Lucas Oliveira
a5d25e0c0b
fix: disable upload message when ingesting on onboarding, wait for file to be ingested, added knowledge filters on nudges (#345)
* Removed upload start message

* Made onboarding upload refetch nudges and only finish when document is ingested

* Implemented query filters on nudges

* changed get to post

* Implemented filtering for documents that are not sample data on nudges

---------

Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
2025-11-11 18:20:39 -03:00
Lucas Oliveira
e02ea85431
Changed default llm model to be gpt 4o (#334) 2025-10-31 12:17:47 -03:00
Lucas Oliveira
16dbc31cc6
Delete unused models (#333) 2025-10-30 15:06:44 -03:00
Lucas Oliveira
cece8a91d5
check if model is embedding by testing it (#332) 2025-10-30 15:03:23 -03:00
Lucas Oliveira
b9ea9c99f1
fix: fixed bugs on ollama integration, added ingestion on onboarding (#330)
* Updated ollama components

* Changed ollama display name to be correct

* Changed prompt of provider validation

* removed event dispatched from file upload

* Changed onboarding to upload the entire knowledge

* Changed default models for ollama
2025-10-30 09:02:06 -03:00
phact
3998014561 fix document processing embedding model bug 2025-10-13 11:31:56 -04:00
phact
a7c5a9f8f3 fis: keyword type field name for search 2025-10-11 02:10:01 -04:00
phact
a424bb422a improve embedding generation timeout hadling w/ retry and error handling 2025-10-11 01:06:14 -04:00
phact
0c696afef8 make overquery optional 2025-10-11 00:59:45 -04:00
phact
aff70096ce .keyword fix 2025-10-11 00:48:09 -04:00
phact
12ae6d3fb1 ingest flow works multi-embedding 2025-10-10 22:14:51 -04:00
phact
6266e5c18d persist dimensions 2025-10-10 14:16:58 -04:00
phact
c8c719c7eb filter query optimization 2025-10-10 10:28:11 -04:00
phact
102c17d7ec WIP, non-langflow mode only 2025-10-10 09:43:37 -04:00
Lucas Oliveira
45c80204a7
changed ibm validation to include generating the bearer token (#240) 2025-10-09 13:58:53 -03:00
Lucas Oliveira
66f8d1b5bf Added validation for api keys 2025-10-08 17:59:30 -03:00
phact
330b16ae06 preserve file name for upload 2025-10-07 02:00:57 -04:00
Edwin Jose
e7799e1a2c Quote OWNER_NAME to handle spaces in headers
OWNER_NAME is now wrapped in double quotes to prevent issues with spaces and special characters when used in headers. This change improves reliability when passing user names containing spaces.
2025-10-06 23:15:13 -04:00
Edwin Jose
88f0bbc1f4
Merge branch 'main' into add-mcp-agent-flows 2025-10-06 17:27:36 -04:00
phact
7763dc5d48 fix: file tuple for ingest 2025-10-06 16:25:05 -04:00
Edwin Jose
42a198f96a Add CONNECTOR_TYPE_URL env and flow support
Introduces the CONNECTOR_TYPE_URL environment variable to docker-compose files and assets, updates the OpenRAG URL ingestion flow to use it, and ensures it is set in the auth service global variables. This enables explicit configuration and handling of URL-based connectors in the OpenRAG system.
2025-10-06 16:24:15 -04:00
Lucas Oliveira
05284f3d2e Merge remote-tracking branch 'origin/main' into fix/tasks_ingest 2025-10-06 11:58:45 -03:00
Edwin Jose
336e7bb501 Integrate OpenSearch hybrid search and update ingestion flow
Replaces the File component with a new OpenSearch hybrid search component in the ingestion flow, adds support for document metadata, and updates flow edges for DataFrame operations. Updates OpenSearch component implementation with advanced authentication, metadata handling, and vector store features. Docker Compose files and related service references are also updated to support the new OpenSearch integration.
2025-10-04 02:09:29 -04:00
Lucas Oliveira
ea61aa7136 added url ingest id to everything 2025-10-03 17:29:28 -03:00
Edwin Jose
5ea70cc779
Refactor file upload task to use filename mapping (#195)
Changed the handling of original filenames in Langflow upload tasks to use a mapping from file paths to original filenames instead of a list. Updated both the API router and TaskService to support this change, improving reliability when associating uploaded files with their original names.
2025-10-03 17:17:05 -03:00
Lucas Oliveira
589839fedc implement duplicate handling on backend and frontend 2025-10-03 15:34:54 -03:00
Lucas Oliveira
11464f031e Merge remote-tracking branch 'origin/main' into lfx-openrag-update-flows 2025-10-02 17:40:19 -03:00
Edwin Jose
e3eb4dc3d1 Update chat_service.py 2025-10-02 15:51:38 -04:00
Lucas Oliveira
52896a8da7
fix: set defined models for openai, make default embedding be used when disable ingest with LF feature flag is enabled (#128)
* hard-coded openai models

* ensure index if disable ingest with langflow is active

* update backend to not update embedding model when flag is disabled

* initialize index on startup when feature flag is enabled

* put config.yaml on docker compose
2025-10-01 16:50:21 -03:00
Edwin Jose
14b0f30107
Merge branch 'main' into lfx-openrag-update-flows 2025-09-30 15:52:18 -04:00
Mike Fortman
d015ed1b0c
add container utils (#151)
* add container utils

* added localhost url to settings

* added localhost_url as a constant

* added localhost_url to get settings query

* make ollama onboarding have localhost url by default

* make endpoint be changed in models service and in onboarding backend instead of onboarding screen

* fixed embedding dimensions to get stripped model

* make config come as localhost but global variable be set as the transformed endpoint

* remove setting ollama url since it comes from the global variable

* use localhost again on ollama

---------

Co-authored-by: Lucas Oliveira <lucas.edu.oli@hotmail.com>
2025-09-30 16:34:55 -03:00
Lucas Oliveira
530a223518 fixed model change not working 2025-09-29 17:39:49 -03:00