diff --git a/docs/docs/_partial-integrate-chat.mdx b/docs/docs/_partial-integrate-chat.mdx
index a7bd565d..fa302b55 100644
--- a/docs/docs/_partial-integrate-chat.mdx
+++ b/docs/docs/_partial-integrate-chat.mdx
@@ -2,7 +2,7 @@ import Icon from "@site/src/components/icon/icon";
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
-1. Open the **OpenRAG OpenSearch Agent** flow in the Langflow visual editor: From the **Chat** window, click **Settings**, click **Edit in Langflow**, and then click **Proceed**.
+1. Open the **OpenRAG OpenSearch Agent** flow in the Langflow visual editor: Click **Settings**, click **Edit in Langflow**, and then click **Proceed**.
2. Optional: If you don't want to use the Langflow API key that is generated automatically when you install OpenRAG, you can create a [Langflow API key](https://docs.langflow.org/api-keys-and-authentication).
This key doesn't grant access to OpenRAG; it is only for authenticating with the Langflow API.
diff --git a/docs/docs/_partial-temp-knowledge.mdx b/docs/docs/_partial-temp-knowledge.mdx
index 7ecdf99c..da537c1e 100644
--- a/docs/docs/_partial-temp-knowledge.mdx
+++ b/docs/docs/_partial-temp-knowledge.mdx
@@ -1,5 +1,5 @@
import Icon from "@site/src/components/icon/icon";
-When using the OpenRAG **Chat**, click in the chat input field to upload a file to the current chat session.
+When using the OpenRAG **Chat**, click **Add** in the chat input field to upload a file to the current chat session.
Files added this way are processed and made available to the agent for the current conversation only.
These files aren't stored in the knowledge base permanently.
\ No newline at end of file
diff --git a/docs/docs/core-components/agents.mdx b/docs/docs/core-components/agents.mdx
index 32387a59..e902f10f 100644
--- a/docs/docs/core-components/agents.mdx
+++ b/docs/docs/core-components/agents.mdx
@@ -21,14 +21,18 @@ You can customize these flows and create your own flows using OpenRAG's embedded
All OpenRAG flows are designed to be modular, performant, and provider-agnostic.
To modify a flow in OpenRAG, click **Settings**.
-From here, you can quickly edit commonly used parameters, such as the **Language model** and **Agent Instructions**.
-To further explore and edit the flow, click **Edit in Langflow** to launch the embedded [Langflow visual editor](https://docs.langflow.org/concepts-overview) where you can fully [customize the flow](https://docs.langflow.org/concepts-flows) to suit your use case.
+From here, you can manage model provider configurations and edit commonly used parameters, such as the **Language model** and **Agent Instructions**.
+To further explore and edit a flow, click **Edit in Langflow** to launch the embedded [Langflow visual editor](https://docs.langflow.org/concepts-overview) where you can fully [customize the flow](https://docs.langflow.org/concepts-flows) to suit your use case.
-For example, to view and edit the built-in **Chat** flow (the **OpenRAG OpenSearch Agent** flow), do the following:
+For example, the following steps explain how to view and edit the built-in **Agent** flow, which is the **OpenRAG OpenSearch Agent** flow used for the OpenRAG **Chat**:
-1. In OpenRAG, click **Chat**.
+1. In OpenRAG, click **Settings**, and then find the **Agent** section.
-2. Click **Settings**, and then click **Edit in Langflow** to launch the Langflow visual editor in a new browser window.
+2. If you only need to edit the language model or agent instructions, edit those fields directly on the **Settings** page.
+Language model changes are saved automatically.
+To apply new instructions, click **Save Agent Instructions**.
+
+3. To edit all flow settings and components with full customization capabilities, click **Edit in Langflow** to launch the Langflow visual editor in a new browser tab.
If prompted to acknowledge that you are entering Langflow, click **Proceed**.
@@ -36,13 +40,12 @@ For example, to view and edit the built-in **Chat** flow (the **OpenRAG OpenSear
-3. Modify the flow as desired, and then press Command+S (Ctrl+S) to save your changes.
+4. Modify the flow as desired, and then press Command+S (Ctrl+S) to save your changes.
- You can close the Langflow browser window, or leave it open if you want to continue experimenting with the flow editor.
+ You can close the Langflow browser tab, or leave it open if you want to continue experimenting with the flow editor.
- :::tip
- If you modify the built-in **Chat** flow, make sure you click in the **Conversations** tab to start a new conversation. This ensures that the chat doesn't persist any context from the previous conversation with the original flow settings.
- :::
+5. After you modify any **Agent** flow settings, go to the OpenRAG **Chat**, and then click **Start new conversation** in the **Conversations** list.
+This ensures that the chat doesn't persist any context from the previous conversation with the original flow settings.
### Revert a built-in flow to its original configuration {#revert-a-built-in-flow-to-its-original-configuration}
diff --git a/docs/docs/core-components/knowledge-filters.mdx b/docs/docs/core-components/knowledge-filters.mdx
index 2197b5f7..f2a2c2b6 100644
--- a/docs/docs/core-components/knowledge-filters.mdx
+++ b/docs/docs/core-components/knowledge-filters.mdx
@@ -26,36 +26,52 @@ After uploading your own documents, it is recommended that you create your own f
To create a knowledge filter, do the following:
-1. Click **Knowledge**, and then click **Knowledge Filters**.
+1. Click **Knowledge**, and then click **Knowledge Filters**.
-2. Enter a **Name** and **Description**, and then click **Create Filter**.
+2. Enter a **Name**.
- By default, new filters match all documents in your knowledge base.
- Modify the filter to customize it.
+3. Optional: Click the filter icon next to the filter name to select a different icon and color for the filter.
+This is purely cosmetic, but it can help you visually distinguish different sets of filters, such as different projects or sources.
-3. To modify the filter, click **Knowledge**, and then click your new filter. You can edit the following settings:
+4. Optional: Enter a **Description**.
- * **Search Query**: Enter text for semantic search, such as `financial reports from Q4`.
- * **Data Sources**: Select specific data sources or folders to include.
+5. Customize the filter settings.
+
+ By default, filters match all documents in your knowledge base.
+ Use the filter settings to narrow the scope of documents that the filter captures:
+
+ * **Search Query**: Enter a natural language text string for semantic search.
+ When you apply a filter that has a **Search Query**, only documents matching the search query are included.
+ It is recommended that you use the **Score Threshold** setting to avoid returning irrelevant documents.
+ * **Data Sources**: Select specific files and folders to include in the filter.
+ This is useful if you want to create a filter for a specific project or topic and you know the specific documents you want to include.
+ Similarly, if you upload a folder of documents, you might want to create a filter that only includes the documents from that folder.
* **Document Types**: Filter by file type.
* **Owners**: Filter by the user that uploaded the documents.
+ **Anonymous User** means a document was uploaded in an OpenRAG environment where OAuth isn't configured.
* **Connectors**: Filter by [upload source](/ingestion), such as the local file system or a Google Drive OAuth connector.
- * **Response Limit**: Set the maximum number of results to return from the knowledge base. The default is `10`.
- * **Score Threshold**: Set the minimum relevance score for similarity search. The default score is `0`.
+ * **Response Limit**: Set the maximum number of results to return from the knowledge base. The default is `10`, which means the filter returns only the top 10 most relevant documents.
+ * **Score Threshold**: Set the minimum relevance score for similarity search. The default score is `0`. A threshold is recommended to avoid returned irrelevant documents.
-4. To save your changes, click **Update Filter**.
+6. Click **Create Filter**.
+
+## Edit a filter
+
+To modify a filter, click **Knowledge**, and then click the filter you want to edit in the **Knowledge Filters** list.
+On the filter settings pane, edit the filter as desired, and then click **Update Filter**.
## Apply a filter {#apply-a-filter}
-* **Apply a global filter**: Click **Knowledge**, and then enable the toggle next to your preferred filter. Only one filter can be the global filter. The global filter applies to all chat sessions.
-
-* **Apply a chat filter**: In the **Chat** window, click **Filter**, and then select the filter to apply.
+In the OpenRAG **Chat**, click **Filter**, and then select the filter to apply.
Chat filters apply to one chat session only.
+You can also use filters when [browsing the **Knowledge** page](/knowledge#browse-knowledge).
+This is a helpful way to test filters and manage knowledge bases that have many documents.
+
## Delete a filter
1. Click **Knowledge**.
-2. Click the filter that you want to delete.
+2. In the **Knowledge Filters** list, click the filter that you want to delete.
-3. Click **Delete Filter**.
\ No newline at end of file
+3. In the filter settings pane, click **Delete Filter**.
\ No newline at end of file
diff --git a/docs/docs/core-components/knowledge.mdx b/docs/docs/core-components/knowledge.mdx
index 1a603bd0..7d6a527b 100644
--- a/docs/docs/core-components/knowledge.mdx
+++ b/docs/docs/core-components/knowledge.mdx
@@ -23,7 +23,22 @@ You can configure how documents are ingested and how the **Chat** interacts with
The **Knowledge** page lists the documents OpenRAG has ingested into your OpenSearch database, specifically in an [OpenSearch index](https://docs.opensearch.org/latest/getting-started/intro/#index) named `documents`.
To explore the raw contents of your knowledge base, click **Knowledge** to get a list of all ingested documents.
-Click a document to view the chunks produced from splitting the document during ingestion.
+
+Click a document to view the chunks produced from splitting the document during ingestion as well as technical details related to chunking.
+
+For each document, the **Knowledge** page provides metadata, including the size, type, user that uploaded the document, the number of chunks created from the document, and the embedding model and dimensions used to embed the document.
+
+The search field at the top of the **Knowledge** page allows you to search for specific documents by name, contents, or with a knowledge filter:
+
+* To search with a text string, enter your search string in the search field, and then press Enter.
+
+* To apply a [knowledge filter](/knowledge-filters) when browsing your knowledge base, click the filter in the **Knowledge Filters** list.
+The filter settings pane opens, and the filter appears in the search field.
+To remove the filter, close the filter settings pane or clear the filter from the search field.
+
+ If a knowledge filter contains a search query, that query is applied in addition to any additional string you enter in the search field.
+
+When you search, the **Avg score** column shows how relevant each document is to your search query.
### Default documents {#default-documents}
@@ -81,8 +96,10 @@ The default embedding dimension is `1536`, and the default model is the OpenAI `
If you want to use an unsupported model, you must manually set the model in your [OpenRAG `.env` file](/reference/configuration).
If you use an unsupported embedding model that doesn't have defined dimensions in `settings.py`, then OpenRAG falls back to the default dimensions (1536) and logs a warning. OpenRAG's OpenSearch instance and flows continue to work, but [similarity search](https://www.ibm.com/think/topics/vector-search) quality can be affected if the actual model dimensions aren't 1536.
-To change the embedding model after onboarding, it is recommended that you modify the embedding model setting in the OpenRAG **Settings** page or in your [OpenRAG `.env` file](/reference/configuration).
-This will automatically update all relevant [OpenRAG flows](/agents) to use the new embedding model configuration.
+To change the embedding model after onboarding, it is recommended that you modify the embedding model configuration on the OpenRAG **Settings** page or in your [OpenRAG `.env` file](/reference/configuration).
+This ensures that all relevant [OpenRAG flows](/agents) are updated to use the new embedding model configuration.
+
+If you edit these settings in the `.env` or `docker-compose` files, you must [stop and restart the OpenRAG containers](/manage-services#stop-and-start-containers) to apply the changes.
### Set Docling parameters
@@ -90,32 +107,36 @@ OpenRAG uses [Docling](https://docling-project.github.io/docling/) for document
When you [upload documents](/ingestion), Docling processes the files, splits them into chunks, and stores them as separate, structured documents in your OpenSearch knowledge base.
+#### Select a Docling implementation
+
You can use either Docling Serve or OpenRAG's built-in Docling ingestion pipeline to process documents.
* **Docling Serve ingestion**: By default, OpenRAG uses [Docling Serve](https://github.com/docling-project/docling-serve).
This means that OpenRAG starts a `docling serve` process on your local machine and runs Docling ingestion through an API service.
* **Built-in Docling ingestion**: If you want to use OpenRAG's built-in Docling ingestion pipeline instead of the separate Docling Serve service, set `DISABLE_INGEST_WITH_LANGFLOW=true` in your [OpenRAG environment variables](/reference/configuration#document-processing-settings).
+The built-in pipeline uses the Docling processor directly instead of through the Docling Serve API.
+For the underlying functionality, see [`processors.py`](https://github.com/langflow-ai/openrag/blob/main/src/models/processors.py#L58) in the OpenRAG repository.
- The built-in pipeline uses the Docling processor directly instead of through the Docling Serve API.
+#### Configure Docling ingestion settings
- For the underlying functionality, see [`processors.py`](https://github.com/langflow-ai/openrag/blob/main/src/models/processors.py#L58) in the OpenRAG repository.
-
-To modify the Docling ingestion and embedding parameters, click **Settings** in the OpenRAG user interface.
+To modify the Docling document processing and embedding parameters, click **Settings** in OpenRAG, and then find the **Knowledge Ingest** section.
:::tip
-OpenRAG warns you if `docling serve` isn't running.
+The TUI warns you if `docling serve` isn't running.
For information about starting and stopping OpenRAG native services, like Docling, see [Manage OpenRAG services](/manage-services).
:::
+You can edit the following parameters:
+
* **Embedding model**: Select the model to use to generate vector embeddings for your documents.
This is initially set during installation.
- The recommended way to change this setting is in the OpenRAG **Settings** or your [OpenRAG `.env` file](/reference/configuration).
- This will automatically update all relevant [OpenRAG flows](/agents) to use the new embedding model configuration.
+ The recommended way to change this setting is in the OpenRAG **Settings** or your [OpenRAG `.env` file](/reference/configuration).
+ This ensures that all relevant [OpenRAG flows](/agents) are updated to use the new embedding model configuration.
If you uploaded documents prior to changing the embedding model, you can [create filters](/knowledge-filters) to separate documents embedded with different models, or you can reupload all documents to regenerate embeddings with the new model.
- If you want to use multiple embeddings models, similarity search (in the **Chat**) can take longer as it searching each model's embeddings separately.
+ If you want to use multiple embeddings models, similarity search (in the **Chat**) can take longer as it searches each model's embeddings separately.
* **Chunk size**: Set the number of characters for each text chunk when breaking down a file.
Larger chunks yield more context per chunk, but can include irrelevant information. Smaller chunks yield more precise semantic search, but can lack context.
@@ -125,7 +146,7 @@ The default value is 1000 characters, which is usually a good balance between co
Use larger overlap values for documents where context is most important. Use smaller overlap values for simpler documents or when optimization is most important.
The default value is 200 characters, which represents an overlap of 20 percent if the **Chunk size** is 1000. This is suitable for general use. For faster processing, decrease the overlap to approximately 10 percent. For more complex documents where you need to preserve context across chunks, increase it to approximately 40 percent.
-* **Table Structure**: Enables Docling's [`DocumentConverter`](https://docling-project.github.io/docling/reference/document_converter/) tool for parsing tables. Instead of treating tables as plain text, tables are output as structured table data with preserved relationships and metadata. This option is enabled by default.
+* **Table structure**: Enables Docling's [`DocumentConverter`](https://docling-project.github.io/docling/reference/document_converter/) tool for parsing tables. Instead of treating tables as plain text, tables are output as structured table data with preserved relationships and metadata. This option is enabled by default.
* **OCR**: Enables Optical Character Recognition (OCR) processing when extracting text from images and ingesting scanned documents. This setting is best suited for processing text-based documents faster with Docling's [`DocumentConverter`](https://docling-project.github.io/docling/reference/document_converter/). Images are ignored and not processed.
@@ -147,7 +168,12 @@ To change this location, modify the **Documents Paths** variable in either the [
This is a destructive operation that cannot be undone.
:::
-To clear your entire knowledge base, [reset your OpenRAG containers](/manage-services#reset-containers) or [reinstall OpenRAG](/reinstall).
+To delete documents from your knowledge base, click **Knowledge**, use the checkboxes to select one or more documents, and then click **Delete**.
+If you select the checkbox at the top of the list, all documents are selected and your entire knowledge base will be deleted.
+
+To delete an individual document, you can also click **More** next to that document, and then select **Delete**.
+
+To completely clear your entire knowledge base and OpenSearch index, [reset your OpenRAG containers](/manage-services#reset-containers) or [reinstall OpenRAG](/reinstall).
## See also