From d578e7832e67882d6d3da4721629f2861bca494f Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Thu, 4 Dec 2025 17:17:00 -0800 Subject: [PATCH] revised setup, explain uv cache, rename svc mgmt --- docs/docs/core-components/ingestion.mdx | 8 +- docs/docs/core-components/knowledge.mdx | 25 ++- docs/docs/get-started/docker.mdx | 9 +- docs/docs/get-started/install-options.mdx | 15 +- docs/docs/get-started/install-uv.mdx | 4 +- docs/docs/get-started/install-uvx.mdx | 34 +-- docs/docs/get-started/install-windows.mdx | 6 +- docs/docs/get-started/install.mdx | 237 ++++++++++---------- docs/docs/get-started/manage-containers.mdx | 32 --- docs/docs/get-started/manage-services.mdx | 103 +++++++++ docs/docs/get-started/quickstart.mdx | 35 +-- docs/docs/get-started/reinstall.mdx | 2 +- docs/docs/get-started/tui.mdx | 8 +- docs/docs/get-started/uninstall.mdx | 6 +- docs/docs/get-started/upgrade.mdx | 10 +- docs/docs/reference/configuration.mdx | 45 ++-- docs/docs/support/troubleshoot.mdx | 30 ++- docs/sidebars.js | 6 +- 18 files changed, 347 insertions(+), 268 deletions(-) delete mode 100644 docs/docs/get-started/manage-containers.mdx create mode 100644 docs/docs/get-started/manage-services.mdx diff --git a/docs/docs/core-components/ingestion.mdx b/docs/docs/core-components/ingestion.mdx index 3e991176..33fb1420 100644 --- a/docs/docs/core-components/ingestion.mdx +++ b/docs/docs/core-components/ingestion.mdx @@ -64,11 +64,11 @@ To enable multiple connectors, you must register an app and generate credentials -If you use the TUI to manage your OpenRAG containers, provide OAuth credentials in the **Advanced Setup**. +If you use the TUI to manage your OpenRAG services, provide OAuth credentials in the **Advanced Setup**. You can do this during [installation](/install#setup), or you can add the credentials afterwards: -1. If OpenRAG is running, stop it: Go to [**Status**](/install#tui-container-management), and then click **Stop Services**. +1. If OpenRAG is running, stop it: Go to [**Status**](/manage-services#tui-container-management), and then click **Stop Services**. 2. Click **Advanced Setup**, and then add the OAuth credentials for the cloud storage providers that you want to use: @@ -83,12 +83,12 @@ These are the URLs your OAuth provider will redirect back to after users authent OpenRAG regenerates the [`.env`](/reference/configuration) file with the given credentials. -5. Click **Start Container Services**. +5. Click **Start All Services**. -If you [install OpenRAG with self-managed containers](/docker), set OAuth credentials in the `.env` file for Docker Compose. +If you [installed OpenRAG with self-managed services](/docker), set OAuth credentials in the `.env` file for Docker Compose. You can do this during [initial set up](/docker#install-openrag-with-docker-compose), or you can add the credentials afterwards: diff --git a/docs/docs/core-components/knowledge.mdx b/docs/docs/core-components/knowledge.mdx index 3edf5a55..86604cbb 100644 --- a/docs/docs/core-components/knowledge.mdx +++ b/docs/docs/core-components/knowledge.mdx @@ -32,20 +32,21 @@ It is recommended that you keep these documents, and use [filters](/knowledge-fi ## OpenSearch authentication and document access {#auth} -When you [install OpenRAG](/install), you can choose between two setup modes: **Basic Setup** and **Advanced Setup**. -The mode you choose determines how OpenRAG authenticates with OpenSearch and controls access to documents: +When you [install OpenRAG](/install-options), you provide the initial configuration values for your OpenRAG services. +This includes authentication credentials for OpenSearch and OAuth connectors. +This configuration determines how OpenRAG authenticates with OpenSearch and controls access to documents in your knowledge base: -* **Basic Setup (no-auth mode)**: If you choose **Basic Setup**, then OpenRAG is installed in no-auth mode. -This mode uses one, anonymous JWT token for OpenSearch authentication. -There is no differentiation between users. -All users that access your OpenRAG instance can access all documents uploaded to your OpenSearch knowledge base. +* **No-auth mode (basic setup)**: If you choose **Basic Setup** in the [TUI](/tui), or your `.env` file doesn't include OAuth credentials, then OpenRAG runs in no-auth mode. -* **Advanced Setup (OAuth mode)**: If you choose **Advanced Setup**, then OpenRAG is installed in OAuth mode. -This mode uses a unique JWT token for each OpenRAG user, and each document is tagged with user ownership. Documents are filtered by user owner. -This means users see only the documents that they uploaded or have access to. + This mode uses one anonymous JWT token for OpenSearch authentication. + There is no differentiation between users; all users that access your OpenRAG instance can access all documents uploaded to your knowledge base. -You can enable OAuth mode after installation. -For more information, see [Ingest files with OAuth connectors](/ingestion#oauth-ingestion). +* **OAuth mode (advanced setup)**: If you choose **Advanced Setup** in the [TUI](/tui), or your `.env` file includes OAuth credentials, then OpenRAG runs in OAuth mode. + + This mode uses a unique JWT token for each OpenRAG user, and each document is tagged with user ownership. + Documents are filtered by user owner; users see only the documents that they uploaded or have access to through their cloud storage accounts. + + You can enable OAuth mode after installation, as explained in [Ingest files with OAuth connectors](/ingestion#oauth-ingestion). ## OpenSearch indexes @@ -119,7 +120,7 @@ To modify the Docling ingestion and embedding parameters, click - 1. To install OpenRAG with **Basic Setup**, click **Basic Setup** or press 1. - 2. Click **Generate Passwords** to generate passwords for OpenSearch and Langflow. + 1. In the TUI, click **Basic Setup** or press 1. - The OpenSearch password is required. The Langflow admin password is optional. - If no Langflow admin password is generated, Langflow runs in [autologin mode](https://docs.langflow.org/api-keys-and-authentication#langflow-auto-login) with no password required. + 2. Enter administrator passwords for the OpenRAG OpenSearch and Langflow services, or click **Generate Passwords** to generate passwords automatically. + + The OpenSearch password is required. + + The Langflow password is optional. + If the Langflow password is empty, Langflow runs in [autologin mode](https://docs.langflow.org/api-keys-and-authentication#langflow-auto-login) without password authentication. + + 3. Optional: Enter your OpenAI API key, or leave this field empty if you want to configure model provider credentials later during application onboarding. - 3. Optional: Paste your OpenAI API key in the OpenAI API key field. You can also provide this during onboarding or choose a different model provider. 4. Click **Save Configuration**. - Your passwords are saved in the `.env` file used to start OpenRAG. - 5. To start OpenRAG, click **Start All Services**. - Startup pulls container images and runs them, so it can take some time. - When startup is complete, the TUI displays the following: - ```bash + + Your passwords and API key, if provided, are stored in the `.env` file in your OpenRAG installation directory. + If you modified any credentials that were pulled from an existing `.env` file, those values are updated in the `.env` file. + + 5. Click **Start All Services** to start the OpenRAG services that run in containers. + + This process can take some time while OpenRAG pulls and runs the container images. + If all services start successfully, the TUI prints a confirmation message: + + ```text Services started successfully Command completed successfully ``` - 6. To start the Docling service, under **Native Services**, click **Start**. - 7. To open the OpenRAG application, navigate to the TUI main menu, and then click **Open App**. - Alternatively, in your browser, navigate to `localhost:3000`. + + 6. Under [**Native Services**](/manage-services), click **Start** to start the Docling service. + + 7. Launch the OpenRAG application: + + * From the TUI main menu, click **Open App**. + * In your browser, navigate to `localhost:3000`. + 8. Continue with [application onboarding](#application-onboarding). + - 1. To install OpenRAG with **Advanced Setup**, click **Advanced Setup** or press 2. - 2. Click **Generate Passwords** to generate passwords for OpenSearch and Langflow. + 1. In the TUI, click **Advanced Setup** or press 2. - The OpenSearch password is required. The Langflow admin password is optional. - If no Langflow admin password is generated, Langflow runs in [autologin mode](https://docs.langflow.org/api-keys-and-authentication#langflow-auto-login) with no password required. + 2. Enter administrator passwords for the OpenRAG OpenSearch and Langflow services, or click **Generate Passwords** to generate passwords automatically. - 3. Paste your OpenAI API key in the OpenAI API key field. - 4. If you want to upload documents from external storage, such as Google Drive, add the required OAuth credentials for the connectors that you want to use. These settings can be populated automatically if OpenRAG detects these credentials in a `.env` file in the OpenRAG installation directory. + The OpenSearch password is required. + + The Langflow password is optional. + If the Langflow password is empty, Langflow runs in [autologin mode](https://docs.langflow.org/api-keys-and-authentication#langflow-auto-login) without password authentication. + + 3. Optional: Enter your OpenAI API key, or leave this field empty if you want to configure model provider credentials later during application onboarding. + + 4. To upload documents from external storage, such as Google Drive, add the required OAuth credentials for the connectors that you want to use. These settings can be populated automatically if OpenRAG detects these credentials in a `.env` file in the OpenRAG installation directory. * **Amazon**: Provide your AWS Access Key ID and AWS Secret Access Key with access to your S3 instance. For more information, see the AWS documentation on [Configuring access to AWS applications](https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-your-applications.html). * **Google**: Provide your Google OAuth Client ID and Google OAuth Client Secret. You can generate these in the [Google Cloud Console](https://console.cloud.google.com/apis/credentials). For more information, see the [Google OAuth client documentation](https://developers.google.com/identity/protocols/oauth2). @@ -122,113 +170,54 @@ If OpenRAG detects OAuth credentials, it recommends **Advanced Setup**. 5. The OpenRAG TUI presents redirect URIs for your OAuth app. These are the URLs your OAuth provider will redirect back to after user sign-in. Register these redirect values with your OAuth provider as they are presented in the TUI. + 6. Click **Save Configuration**. - 7. To start OpenRAG, click **Start All Services**. - Startup pulls container images and runs them, so it can take some time. - When startup is complete, the TUI displays the following: - ```bash + + Your passwords, API key (if provided), and OAuth credentials (if provided) are stored in the `.env` file in your OpenRAG installation directory. + If you modified any credentials that were pulled from an existing `.env` file, those values are updated in the `.env` file. + + 7. Click **Start All Services** to start the OpenRAG services that run in containers. + + This process can take some time while OpenRAG pulls and runs the container images. + If all services start successfully, the TUI prints a confirmation message: + + ```text Services started successfully Command completed successfully ``` - 8. To start the Docling service, under **Native Services**, click **Start**. - 9. To open the OpenRAG application, navigate to the TUI main menu, and then click **Open App**. - Alternatively, in your browser, navigate to `localhost:3000`. + + 8. Under [**Native Services**](/manage-services), click **Start** to start the Docling service. + + 9. Launch the OpenRAG application: + + * From the TUI main menu, click **Open App**. + * In your browser, navigate to `localhost:3000`. 10. If you enabled OAuth connectors, you must sign in to your OAuth provider before being redirected to your OpenRAG instance. - 11. Two additional variables are available for **Advanced Setup** at this point. - Only change these variables if you have a non-default network configuration for your deployment, such as using a reverse proxy or custom domain. + 11. If required, you can edit the following additional environment variables. + Only change these variables if your OpenRAG deployment has a non-default network configuration, such as a reverse proxy or custom domain. * `LANGFLOW_PUBLIC_URL`: Sets the base address to access the Langflow web interface. This is where users interact with flows in a browser. - * `WEBHOOK_BASE_URL`: Sets the base address of the OpenRAG OAuth connector endpoint. - Supported webhook endpoints: + * `WEBHOOK_BASE_URL`: Sets the base address for the following OpenRAG OAuth connector endpoints: - Amazon S3: Not applicable. - - Google Drive: `/connectors/google_drive/webhook` - - OneDrive: `/connectors/onedrive/webhook` - - SharePoint: `/connectors/sharepoint/webhook` + - Google Drive: `WEBHOOK_BASE_URL/connectors/google_drive/webhook` + - OneDrive: `WEBHOOK_BASE_URL/connectors/onedrive/webhook` + - SharePoint: `WEBHOOK_BASE_URL/connectors/sharepoint/webhook` 12. Continue with [application onboarding](#application-onboarding). + +The first time you start the OpenRAG application, you must complete application onboarding to select language and embedding models that are essential for OpenRAG features like the [**Chat**](/chat). + -## Manage OpenRAG containers with the TUI {#tui-container-management} - -After installation, the TUI can deploy, manage, and upgrade your OpenRAG containers. - -### Diagnostics - -The **Diagnostics** menu provides health monitoring for your container runtimes and monitoring of your OpenSearch security. - -### Status {#status} - -The **Status** menu displays information on your container deployment. -Here you can check container health, find your service ports, view logs, and upgrade your containers. - -* **Logs**: To view streaming logs, select the container you want to view, and press l. -To copy the logs, click **Copy to Clipboard**. - -* **Upgrade**: Check for updates. For more information, see [upgrade OpenRAG](/upgrade). - -* **Factory Reset**: This is a destructive action that [resets your containers](#reset-containers). - -* **Native services**: [View and manage OpenRAG services](#start-all-services) that run directly on your local machine instead of a container. - -### Reset containers {#reset-containers} - -Reset your OpenRAG deployment by recreating the containers and removing some related data. - -:::warning -This is a destructive action that destroys the following: - -* All OpenRAG containers, volumes, and local images -* Any additional Docker objects -* The contents of OpenRAG's `config` and `./opensearch-data` directories -* The `conversations.json` file - -This operation _doesn't_ remove the `.env` file or the contents of the `./openrag-documents` directory. -::: - -1. To destroy and recreate your OpenRAG containers, go to the TUI [**Status** menu](#status), and then click **Factory Reset**. - - This function runs the following commands _and_ deletes the contents of OpenRAG's `config` and `./opensearch-data` directories. - - ```bash - docker compose down --volumes --remove-orphans --rmi local - docker system prune -f - ``` - -2. If you reset your containers as part of reinstalling OpenRAG, continue the [reinstallation process](/reinstall) after resetting the containers. -Otherwise, in the TUI **Setup** menu, repeat the [setup process](#setup) to start the services and launch the OpenRAG app. Your OpenRAG passwords, OAuth credentials (if previously set), and onboarding configuration are restored from the `.env` file. - -### Start all services {#start-all-services} - -Through the TUI, you can view and manage OpenRAG services that run in containers and directly on your local machine. - -#### Start containers - -On the TUI main page or the **Setup** menu, click **Start All Services** to start the OpenRAG containers and launch OpenRAG itself. - -When you start all services, the following processes happen: - -1. OpenRAG automatically detects your container runtime, and then checks if your machine has compatible GPU support by checking for `CUDA`, `NVIDIA_SMI`, and Docker/Podman runtime support. This check determines which Docker Compose file OpenRAG uses. - -2. OpenRAG pulls the OpenRAG container images with `docker compose pull` if any images are missing. - -3. OpenRAG deploys the containers with `docker compose up -d`. - -#### Start native services (Docling) - -A _native service_ in OpenRAG is a service that runs locally on your machine, not within a container. For example, the `docling serve` process is an OpenRAG native service because this document processing service runs on your local machine, separate from the OpenRAG containers. - -From the **Status** menu, you can view the status, port, and process ID (PID) of the OpenRAG native services. -You can also click **Stop** or **Restart** to stop and start OpenRAG native services. - ## Next steps -* [Manage OpenRAG containers](/manage-containers) +* [Manage OpenRAG services](/manage-services) * [Chat](/chat) * [Upload documents](/ingestion) \ No newline at end of file diff --git a/docs/docs/get-started/manage-containers.mdx b/docs/docs/get-started/manage-containers.mdx deleted file mode 100644 index 584701fa..00000000 --- a/docs/docs/get-started/manage-containers.mdx +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: Manage OpenRAG containers -slug: /manage-containers ---- - -Container management is an essential part of maintaining your OpenRAG deployment. - -## Manage containers with the TUI - -If you [installed OpenRAG](/install-options) with the automated installer script, `uv`, or `uvx`, you can use the [Terminal User Interface (TUI)](/tui) to manage your containers: - -* Start and stop OpenRAG containers. -* Start and stop OpenRAG's native services (Docling). -* View the status of your OpenRAG services. -* Access container logs for troubleshooting. -* Upgrade your OpenRAG containers to the latest version. -* Reset your OpenRAG containers to an initial state (destructive). - -## Manage containers with Docker or Podman - -If you [deployed OpenRAG with self-managed containers](/docker), run Docker or Podman commands to manage your OpenRAG containers. - -* Start containers -* Stop containers -* View container status -* Access container logs for troubleshooting -* Upgrade your OpenRAG containers to the latest version -* Reset your OpenRAG containers to an initial state (destructive) - -## See also - -* [Uninstall OpenRAG](/uninstall) \ No newline at end of file diff --git a/docs/docs/get-started/manage-services.mdx b/docs/docs/get-started/manage-services.mdx new file mode 100644 index 00000000..082ee59a --- /dev/null +++ b/docs/docs/get-started/manage-services.mdx @@ -0,0 +1,103 @@ +--- +title: Manage OpenRAG containers and services +slug: /manage-services +--- + +Service management is an essential part of maintaining your OpenRAG deployment. + +Most OpenRAG services run in containers. + +_Native services_, like Docling, run directly on the local machine where you installed OpenRAG. + +## Manage containers and services with the TUI {#tui-container-management} + +If you [installed OpenRAG](/install-options) with the automated installer script, `uv`, or `uvx`, you can use the [Terminal User Interface (TUI)](/tui) to manage your services: + +* Start and stop OpenRAG container-based services. +* Start and stop OpenRAG's native services (Docling). +* View the status of your OpenRAG services. +* Access container logs for troubleshooting. +* Upgrade your OpenRAG containers to the latest version. +* Reset your OpenRAG containers to an initial state (destructive). + +### Diagnostics + +The **Diagnostics** menu provides health monitoring for your container runtimes and monitoring of your OpenSearch security. + +### Status {#status} + +The TUI's **Status** menu provides information about your OpenRAG services, including health, ports, logs, and controls: + +* **Logs**: To view streaming logs, select the container you want to view, and press l. +To copy the logs, click **Copy to Clipboard**. + +* **Upgrade**: Check for updates to OpenRAG. For more information, see [upgrade OpenRAG](/upgrade). + +* **Factory Reset**: This is a destructive action that [resets your containers](#reset-containers). + +* **Native services**: [View and manage OpenRAG's native services](#start-all-services) that run directly on the local machine instead of a container. + +### Reset containers {#reset-containers} + +Reset your OpenRAG deployment by recreating the containers and removing some related data. + +:::warning +This is a destructive action that destroys the following: + +* All OpenRAG containers, volumes, and local images +* Any additional Docker objects +* The contents of OpenRAG's `config` and `./opensearch-data` directories +* The `conversations.json` file + +This operation _doesn't_ remove the `.env` file or the contents of the `./openrag-documents` directory. +::: + +1. To destroy and recreate your OpenRAG containers, go to the TUI [**Status** menu](#status), and then click **Factory Reset**. + + This function runs the following commands _and_ deletes the contents of OpenRAG's `config` and `./opensearch-data` directories. + + ```bash + docker compose down --volumes --remove-orphans --rmi local + docker system prune -f + ``` + +2. If you reset your containers as part of reinstalling OpenRAG, continue the [reinstallation process](/reinstall) after resetting the containers. +Otherwise, in the TUI **Setup** menu, repeat the [setup process](#setup) to start the services and launch the OpenRAG app. Your OpenRAG passwords, OAuth credentials (if previously set), and onboarding configuration are restored from the `.env` file. + +### Start all services {#start-all-services} + +Through the TUI, you can stop and start OpenRAG services. + +#### Start containers + +On the TUI main page or the **Setup** menu, click **Start All Services** to start the OpenRAG containers. + +When you start all services, the following processes happen: + +1. OpenRAG automatically detects your container runtime, and then checks if your machine has compatible GPU support by checking for `CUDA`, `NVIDIA_SMI`, and Docker/Podman runtime support. This check determines which Docker Compose file OpenRAG uses. + +2. OpenRAG pulls the OpenRAG container images with `docker compose pull` if any images are missing. + +3. OpenRAG deploys the containers with `docker compose up -d`. + +#### Start native services (Docling) + +A _native service_ in OpenRAG is a service that runs locally on your machine, not within a container. For example, the `docling serve` process is an OpenRAG native service because this document processing service runs on your local machine, separate from the OpenRAG containers. + +From the **Status** menu, you can view the status, port, and process ID (PID) of the OpenRAG native services. +You can also click **Stop** or **Restart** to stop and start OpenRAG native services. + +## Manage containers with Docker or Podman + +If you [deployed OpenRAG with self-managed containers](/docker), run Docker or Podman commands to manage your OpenRAG containers. + +* Start containers +* Stop containers +* View container status +* Access container logs for troubleshooting +* Upgrade your OpenRAG containers to the latest version +* Reset your OpenRAG containers to an initial state (destructive) + +## See also + +* [Uninstall OpenRAG](/uninstall) \ No newline at end of file diff --git a/docs/docs/get-started/quickstart.mdx b/docs/docs/get-started/quickstart.mdx index 2359d6ae..5789e3ed 100644 --- a/docs/docs/get-started/quickstart.mdx +++ b/docs/docs/get-started/quickstart.mdx @@ -33,10 +33,9 @@ For other providers, see the complete [installation guide](/install). ## Install OpenRAG For this quickstart, install OpenRAG with the automatic installer script and basic setup. +The script installs OpenRAG dependencies, including Docker or Podman, and then it installs and runs OpenRAG with [`uvx`](https://docs.astral.sh/uv/guides/tools/#running-tools). -This script installs OpenRAG dependencies, including Docker or Podman, installs OpenRAG with [`uvx`](https://docs.astral.sh/uvx/getting-started/installation/), and creates a `.env` file and `docker-compose` files in the current working directory. - -1. Create a directory to store the OpenRAG configuration files, and then change to that directory: +1. Create a directory for your OpenRAG installation, and then change to that directory: ```bash mkdir openrag-workspace @@ -53,39 +52,41 @@ This script installs OpenRAG dependencies, including Docker or Podman, installs You might be prompted to install certain dependencies if they aren't already present in your environment. The entire process can take a few minutes. - Once the environment is ready, the OpenRAG Terminal User Interface (TUI) starts. + Once the environment is ready, the OpenRAG [Terminal User Interface (TUI)](/tui) starts. ![OpenRAG TUI Interface](@site/static/img/OpenRAG_TUI_2025-09-10T13_04_11_757637.svg) -3. Click **Basic Setup**. +3. In the TUI, click **Basic Setup**. -4. Create passwords for your OpenRAG installation's OpenSearch and Langflow services. You can click **Generate Passwords** to automatically generate passwords. +4. Click **Generate Passwords** to create administrator passwords for your OpenRAG OpenSearch and Langflow services. - The OpenSearch password is required. The Langflow admin password is optional. - If you don't generate a Langflow admin password, Langflow runs in [autologin mode](https://docs.langflow.org/api-keys-and-authentication#langflow-auto-login) with no password required. +5. Leave the **OpenAI API key** field empty. Your passwords are saved in the `.env` file that is used to start OpenRAG. You can find this file in your OpenRAG installation directory. -5. Click **Save Configuration**, and then click **Start All Services**. +6. Click **Save Configuration**, and then click **Start All Services**. - Wait a few minutes while the startup process pulls and runs the necessary container images. - Proceed when you see the following messages in the TUI: + This process can take some time while OpenRAG pulls and runs the container images. + If all services start successfully, the TUI prints a confirmation message: - ```bash + ```text Services started successfully Command completed successfully ``` -6. To open the OpenRAG application, go to the TUI main menu, and then click **Open App**. -Alternatively, in your browser, navigate to `localhost:3000`. + Your [OpenRAG configuration](/reference/configuration) is stored in a `.env` file that is created automatically in the directory where you ran the installer script. + Container definitions are stored in a `docker-compose.yml` file in the same directory. -7. Select the **OpenAI** model provider, enter your OpenAI API key, and then click **Complete**. +7. Under [**Native Services**](/manage-services), click **Start** to start the Docling service. - For this quickstart, you can use the default options for the model settings. +8. From the TUI main menu, click **Open App** to launch the OpenRAG application and start the application onboarding process. -8. Click through the overview slides for a brief introduction to OpenRAG and basic setup, or click