Introduces opensearch_multimodel.py, a new component supporting multi-model hybrid search and ingestion in OpenSearch with dynamic vector fields, parallel embedding, and advanced filtering. Refactors embedding generation in opensearch.py to use tenacity-based retry logic and IBM/Watsonx rate limiting. Updates related flow JSONs to integrate the new component.
* Changed backend to mount config at volume
* update lock
* Changed backend to reapply settings after detecting that flow is reset
* Added periodic backup for flows, make better reset
* tui warning
* Changed settings page to alert user that he has to disable lock flow
* Changed flows to be locked
* Do periodic backup only if onboarding is done
* Change backup function to only back up flows if flow lock is disabled
* Added session manager to reapply all settings
---------
Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
Introduces a 'fail_safe_mode' option to the Embedding Model component, allowing errors to be logged and None returned instead of raising exceptions. Refactors embedding initialization logic for OpenAI, Ollama, and IBM watsonx.ai providers to support this mode, and updates UI configuration and metadata accordingly.
Replaces OpenSearchHybrid with OpenSearchVectorStoreComponentMultimodalMultiEmbedding in ingestion_flow.json, updating all relevant edges and embedding connections. Updates docker-compose.yml to use local builds for backend, frontend, and langflow, and improves environment variable handling for API keys. This refactor enables multi-model and multimodal embedding support for document ingestion and search.
Switches Docker Compose services to local builds for backend, frontend, and langflow. Updates embedding model component to support IBM watsonx.ai features, including input token truncation and original text output, adds new dependencies, and improves configuration options in ingestion and agent flows.
* Added flows with new components
* commented model provider assignment
* Added agent component display name
* commented provider assignment, assign provider on the generic component, assign custom values
* fixed ollama not showing loading steps, fixed loading steps never being removed
* made embedding and llm model optional on onboarding call
* added isEmbedding handling on useModelSelection
* added isEmbedding on onboarding card, separating embedding from non embedding card
* Added one additional step to configure embeddings
* Added embedding provider config
* Changed settings.py to return if not embedding
* Added editing fields to onboarding
* updated onboarding and flows_service to change embedding and llm separately
* updated templates that needs to be changed with provider values
* updated flows with new components
* Changed config manager to not have default models
* Changed flows_service settings
* Complete steps if not embedding
* Add more onboarding steps
* Removed one step from llm steps
* Added Anthropic as a model for the language model on the frontend
* Added anthropic models
* Added anthropic support on Backend
* Fixed provider health and validation
* Format settings
* Change anthropic logo
* Changed button to not jump
* Changed flows service to make anthropic work
* Fixed some things
* add embedding specific global variables
* updated flows
* fixed ingestion flow
* Implemented anthropic on settings page
* add embedding provider logo
* updated backend to work with multiple provider config
* update useUpdateSettings with new settings type
* updated provider health banner to check for health with new api
* changed queries and mutations to use new api
* changed embedding model input to work with new api
* Implemented provider based config on the frontend
* update existing design
* fixed settings configured
* fixed provider health query to include health check for both the providers
* Changed model-providers to show correctly the configured providers
* Updated prompt
* updated openrag agent
* Fixed settings to allow editing providers and changing llm and embedding models
* updated settings
* changed lf ver
* bump openrag version
* added more steps
* update settings to create the global variables
* updated steps
* updated default prompt
---------
Co-authored-by: Sebastián Estévez <estevezsebastian@gmail.com>
Replaces the File component with a new OpenSearch hybrid search component in the ingestion flow, adds support for document metadata, and updates flow edges for DataFrame operations. Updates OpenSearch component implementation with advanced authentication, metadata handling, and vector store features. Docker Compose files and related service references are also updated to support the new OpenSearch integration.
Added OWNER, OWNER_NAME, OWNER_EMAIL, and CONNECTOR_TYPE environment variables to docker-compose.yml and updated LANGFLOW_VARIABLES_TO_GET_FROM_ENVIRONMENT accordingly. Modified ingestion_flow.json to adjust node selection and className, and cleared a sensitive value. Added logging for metadata tweaks in langflow_file_service.py for better traceability.
Updated the OpenSearchVectorStoreComponent to improve document metadata ingestion, including support for Data objects in docs_metadata. Added new edges and nodes to ingestion_flow.json for dynamic metadata input. Changed Dockerfile.langflow to use the fix-file-component branch.
This commit modifies the ingestion flow JSON by adding support for user-defined metadata through the "docs_metadata" input field, improving the context for document ingestion. Additionally, it updates the selection state of a node and adjusts the viewport settings for better layout. These changes enhance the overall functionality and clarity of the component configurations, aligning with best practices for robust async development.
This commit modifies the ingestion flow JSON by adjusting the position of nodes for better layout, adding a new input field for ingestion metadata, and updating the code hash. These changes improve the clarity and functionality of the component configurations, aligning with best practices for robust async development and enhancing the overall user experience.
This commit updates the ingestion flow JSON by modifying the last updated timestamp, changing the code hash, and adding the "anyio" dependency to the metadata. Additionally, it refines the File component's description and input options for advanced document processing, ensuring better clarity and functionality. These changes align with best practices for robust async development and enhance the overall user experience.
This commit modifies the ingestion flow JSON by adding the "lf_version" field to components, updating the last updated timestamp, and changing the selection state of a node. Additionally, it refines the file path handling and improves the overall clarity of the configuration. These changes align with best practices for robust async development and enhance the functionality of the ingestion flow.
This commit updates the ingestion flow JSON by modifying the SplitText component to include new input options such as "text_key" and "keep_separator", along with a detailed documentation link. Additionally, it refines the File component's description and adds new input fields for advanced document processing, improving clarity and functionality. These changes align with best practices for robust async development and enhance the overall user experience.
This commit refines the ingestion flow JSON by reintroducing an edge connection between the File and SplitText nodes, ensuring proper data flow. It also updates the last updated timestamp and modifies the selection state of a node, contributing to improved functionality and user experience. These changes align with best practices for async development and enhance the overall robustness of the ingestion flow.
This commit modifies the ingestion flow configuration by adding a new input option for file paths and updating the last updated timestamp. It also refines the OpenSearchHybridComponent to enhance authentication handling and input configurations. These changes improve the functionality and user experience of the ingestion flow, aligning with best practices for async development.
This commit updates the ingestion flow JSON configuration, including modifications to the last updated timestamps, dimensions of UI components, and selected output types. Additionally, it enhances the OpenSearchHybridComponent by refining input options and ensuring proper handling of authentication settings. These changes improve the overall functionality and user experience of the ingestion flow.
This commit modifies the ingestion flow JSON by updating the last_updated timestamp, removing the obsolete File Path input field, and adjusting the file_path array to be empty. Additionally, it corrects the height measurement of a UI component and changes the selected state of a node. These updates contribute to a cleaner and more efficient configuration, aligning with best practices for async code development.
This commit updates the ingestion flow JSON configuration to enhance the OpenSearch hybrid component. Key changes include renaming the component to OpenSearchHybrid, updating input fields for better clarity, and improving the handling of JWT authentication. Additionally, the commit introduces new parameters for search configuration, such as engine selection and filter expressions, while ensuring the overall structure adheres to robust async coding practices and well-documented code.
This commit introduces a new JSON configuration file for the OpenSearch ingestion flow, detailing the data processing pipeline. The flow includes components for splitting text, generating embeddings, and ingesting data into OpenSearch, enhancing the capabilities for Retrieval Augmented Generation (RAG) tasks. The configuration is designed to support various input types and provides detailed metadata for each component, ensuring robust and well-documented integration.
