From bb63309d1cb43baff7c4730feae1ce112a05e801 Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Tue, 13 Jan 2026 09:18:08 -0800 Subject: [PATCH 1/5] issue 762 and 765 --- docs/docs/_partial-prereq-common.mdx | 4 +++- docs/docs/get-started/docker.mdx | 33 ++++++++++++++++++---------- docs/docs/support/troubleshoot.mdx | 27 +++++++++++++++++++++++ 3 files changed, 52 insertions(+), 12 deletions(-) diff --git a/docs/docs/_partial-prereq-common.mdx b/docs/docs/_partial-prereq-common.mdx index 4682d8cc..621273c7 100644 --- a/docs/docs/_partial-prereq-common.mdx +++ b/docs/docs/_partial-prereq-common.mdx @@ -15,4 +15,6 @@ If a provider offers only one type, you must select two providers. ::: -* Optional: Install GPU support with an NVIDIA GPU, [CUDA](https://docs.nvidia.com/cuda/) support, and compatible NVIDIA drivers on the OpenRAG host machine. If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment. \ No newline at end of file +* Optional: Install GPU support with an NVIDIA GPU, [CUDA](https://docs.nvidia.com/cuda/) support, and compatible NVIDIA drivers on the OpenRAG host machine. +If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment that is suitable for most use cases. +The default CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers. \ No newline at end of file diff --git a/docs/docs/get-started/docker.mdx b/docs/docs/get-started/docker.mdx index 811d3388..2f12d353 100644 --- a/docs/docs/get-started/docker.mdx +++ b/docs/docs/get-started/docker.mdx @@ -116,17 +116,7 @@ The following variables are required or recommended: 3. Deploy the OpenRAG containers locally using the appropriate Docker Compose configuration for your environment: - * **GPU-accelerated deployment**: If your host machine has an NVIDIA GPU with CUDA support and compatible NVIDIA drivers, use the base `docker-compose.yml` file with the `docker-compose.gpu.yml` override. - - ```bash title="Docker" - docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d - ``` - - ```bash title="Podman" - podman compose -f docker-compose.yml -f docker-compose.gpu.yml up -d - ``` - - * **CPU-only deployment** (default): If your host machine doesn't have NVIDIA GPU support, use the base `docker-compose.yml` file. + * **CPU-only deployment** (default, recommended): If your host machine doesn't have NVIDIA GPU support, use the base `docker-compose.yml` file. ```bash title="Docker" docker compose up -d @@ -136,6 +126,27 @@ The following variables are required or recommended: podman compose up -d ``` + * **GPU-accelerated deployment**: If your host machine has an NVIDIA GPU with CUDA support and compatible NVIDIA drivers, use the base `docker-compose.yml` file with the `docker-compose.gpu.yml` override: + + ```bash title="Docker" + docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d + ``` + + ```bash title="Podman" + podman compose -f docker-compose.yml -f docker-compose.gpu.yml up -d + ``` + + tip::: + GPU acceleration isn't required for most use cases. + OpenRAG's CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers. + + GPU acceleration is required only for specific use cases, typically involving customization of the ingestion flows or ingestion logic. + For example, writing alternate ingest logic in OpenRAG that uses GPUs directly in the container, or customizing the ingestion flows to use Langflow's Docling component with GPU acceleration instead of the `docling serve` service. + + If you are deploying OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, you must edit the `docker-compose.gpu.yml` file before deploying the OpenRAG containers. + For more information, see [Some images failed to pull on Amazon EC2 with GPU support](/support/troubleshoot#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support). + ::: + 4. Wait for the OpenRAG containers to start, and then confirm that all containers are running: ```bash title="Docker" diff --git a/docs/docs/support/troubleshoot.mdx b/docs/docs/support/troubleshoot.mdx index f4d69e6e..fa1cdaa4 100644 --- a/docs/docs/support/troubleshoot.mdx +++ b/docs/docs/support/troubleshoot.mdx @@ -35,6 +35,33 @@ open "/Applications/Python VERSION/Install Certificates.command" Replace `VERSION` with your installed Python version, such as `3.13`. +### Some images failed to pull on Amazon EC2 with GPU support {#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support} + +When installing OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, some Docker images might fail to pull. + +To resolve this issue, do the following: + +1. [Stop the OpenRAG containers](/manage-services#stop-and-start-containers). + +2. In your `~/.openrag` directory, edit the `docker-compose.gpu.yml` file as follows: + + ```yaml + services: + openrag-backend: + environment: + - NVIDIA_DRIVER_CAPABILITIES=compute,utility + - NVIDIA_VISIBLE_DEVICES=all + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all + capabilities: [gpu] + ``` + +3. [Restart the OpenRAG containers](/manage-services#stop-and-start-containers). + ### Application onboarding gets stuck on Google Chrome If the OpenRAG onboarding process gets stuck when using Google Chrome, try clearing your browser's cache. From ebaa1f1a4eecaa45d77c0a6a99177b52175b97fe Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Tue, 13 Jan 2026 09:20:39 -0800 Subject: [PATCH 2/5] punctuation --- docs/docs/get-started/docker.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/get-started/docker.mdx b/docs/docs/get-started/docker.mdx index 2f12d353..acd4a971 100644 --- a/docs/docs/get-started/docker.mdx +++ b/docs/docs/get-started/docker.mdx @@ -116,7 +116,7 @@ The following variables are required or recommended: 3. Deploy the OpenRAG containers locally using the appropriate Docker Compose configuration for your environment: - * **CPU-only deployment** (default, recommended): If your host machine doesn't have NVIDIA GPU support, use the base `docker-compose.yml` file. + * **CPU-only deployment** (default, recommended): If your host machine doesn't have NVIDIA GPU support, use the base `docker-compose.yml` file: ```bash title="Docker" docker compose up -d From 42df43d21ad4fd0f0bd7a98a1391e77a00117082 Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Tue, 13 Jan 2026 09:24:52 -0800 Subject: [PATCH 3/5] gpu mode tip partial --- docs/docs/_partial-gpu-mode-tip.mdx | 8 ++++++++ docs/docs/get-started/docker.mdx | 12 +++--------- docs/docs/get-started/tui.mdx | 6 ++++++ 3 files changed, 17 insertions(+), 9 deletions(-) create mode 100644 docs/docs/_partial-gpu-mode-tip.mdx diff --git a/docs/docs/_partial-gpu-mode-tip.mdx b/docs/docs/_partial-gpu-mode-tip.mdx new file mode 100644 index 00000000..6a828ef8 --- /dev/null +++ b/docs/docs/_partial-gpu-mode-tip.mdx @@ -0,0 +1,8 @@ +GPU acceleration isn't required for most use cases. +OpenRAG's CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers. + +GPU acceleration is required only for specific use cases, typically involving customization of the ingestion flows or ingestion logic. +For example, writing alternate ingest logic in OpenRAG that uses GPUs directly in the container, or customizing the ingestion flows to use Langflow's Docling component with GPU acceleration instead of OpenRAG's `docling serve` service. + +To deploy OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, you must edit the `docker-compose.gpu.yml` file before starting the OpenRAG containers. +For more information, see [Some images failed to pull on Amazon EC2 with GPU support](/support/troubleshoot#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support). \ No newline at end of file diff --git a/docs/docs/get-started/docker.mdx b/docs/docs/get-started/docker.mdx index acd4a971..fa6b966a 100644 --- a/docs/docs/get-started/docker.mdx +++ b/docs/docs/get-started/docker.mdx @@ -12,6 +12,7 @@ import PartialPrereqWindows from '@site/docs/_partial-prereq-windows.mdx'; import PartialPrereqPython from '@site/docs/_partial-prereq-python.mdx'; import PartialInstallNextSteps from '@site/docs/_partial-install-next-steps.mdx'; import PartialOllamaModels from '@site/docs/_partial-ollama-models.mdx'; +import PartialGpuModeTip from '@site/docs/_partial-gpu-mode-tip.mdx'; To manage your own OpenRAG services, deploy OpenRAG with Docker or Podman. @@ -136,15 +137,8 @@ The following variables are required or recommended: podman compose -f docker-compose.yml -f docker-compose.gpu.yml up -d ``` - tip::: - GPU acceleration isn't required for most use cases. - OpenRAG's CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers. - - GPU acceleration is required only for specific use cases, typically involving customization of the ingestion flows or ingestion logic. - For example, writing alternate ingest logic in OpenRAG that uses GPUs directly in the container, or customizing the ingestion flows to use Langflow's Docling component with GPU acceleration instead of the `docling serve` service. - - If you are deploying OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, you must edit the `docker-compose.gpu.yml` file before deploying the OpenRAG containers. - For more information, see [Some images failed to pull on Amazon EC2 with GPU support](/support/troubleshoot#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support). + :::tip + ::: 4. Wait for the OpenRAG containers to start, and then confirm that all containers are running: diff --git a/docs/docs/get-started/tui.mdx b/docs/docs/get-started/tui.mdx index 2a6c36dc..7a192c67 100644 --- a/docs/docs/get-started/tui.mdx +++ b/docs/docs/get-started/tui.mdx @@ -3,6 +3,8 @@ title: Use the TUI slug: /tui --- +import PartialGpuModeTip from '@site/docs/_partial-gpu-mode-tip.mdx'; + The OpenRAG Terminal User Interface (TUI) provides a simplified and guided experience for configuring, managing, and monitoring your OpenRAG deployment directly from the terminal. ![OpenRAG TUI Interface](@site/static/img/openrag_tui_dec_2025.png) @@ -36,6 +38,10 @@ In the TUI, click **Status**, and then click **Switch to GPU Mode** or **Switch This change requires restarting all OpenRAG services because each mode has its own `docker-compose` file. +:::tip + +::: + ## Exit the OpenRAG TUI To exit the OpenRAG TUI, press q on the TUI main page. From a3304594e55e62481ad2c8a0d900777d24bfc2fc Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Tue, 13 Jan 2026 09:59:10 -0800 Subject: [PATCH 4/5] indentation --- docs/docs/get-started/docker.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/docs/get-started/docker.mdx b/docs/docs/get-started/docker.mdx index fa6b966a..1fc4e798 100644 --- a/docs/docs/get-started/docker.mdx +++ b/docs/docs/get-started/docker.mdx @@ -129,13 +129,13 @@ The following variables are required or recommended: * **GPU-accelerated deployment**: If your host machine has an NVIDIA GPU with CUDA support and compatible NVIDIA drivers, use the base `docker-compose.yml` file with the `docker-compose.gpu.yml` override: - ```bash title="Docker" - docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d - ``` + ```bash title="Docker" + docker compose -f docker-compose.yml -f docker-compose.gpu.yml up -d + ``` - ```bash title="Podman" - podman compose -f docker-compose.yml -f docker-compose.gpu.yml up -d - ``` + ```bash title="Podman" + podman compose -f docker-compose.yml -f docker-compose.gpu.yml up -d + ``` :::tip From 70bf08e7ec4e38d261ac248d85f2c1217dc76c62 Mon Sep 17 00:00:00 2001 From: April M <36110273+aimurphy@users.noreply.github.com> Date: Thu, 15 Jan 2026 07:12:06 -0800 Subject: [PATCH 5/5] remove ec2 troubleshooting --- docs/docs/_partial-gpu-mode-tip.mdx | 5 +---- docs/docs/support/troubleshoot.mdx | 27 --------------------------- 2 files changed, 1 insertion(+), 31 deletions(-) diff --git a/docs/docs/_partial-gpu-mode-tip.mdx b/docs/docs/_partial-gpu-mode-tip.mdx index 6a828ef8..d9d229fb 100644 --- a/docs/docs/_partial-gpu-mode-tip.mdx +++ b/docs/docs/_partial-gpu-mode-tip.mdx @@ -2,7 +2,4 @@ GPU acceleration isn't required for most use cases. OpenRAG's CPU-only deployment doesn't prevent you from using GPU acceleration in external services, such as Ollama servers. GPU acceleration is required only for specific use cases, typically involving customization of the ingestion flows or ingestion logic. -For example, writing alternate ingest logic in OpenRAG that uses GPUs directly in the container, or customizing the ingestion flows to use Langflow's Docling component with GPU acceleration instead of OpenRAG's `docling serve` service. - -To deploy OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, you must edit the `docker-compose.gpu.yml` file before starting the OpenRAG containers. -For more information, see [Some images failed to pull on Amazon EC2 with GPU support](/support/troubleshoot#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support). \ No newline at end of file +For example, writing alternate ingest logic in OpenRAG that uses GPUs directly in the container, or customizing the ingestion flows to use Langflow's Docling component with GPU acceleration instead of OpenRAG's `docling serve` service. \ No newline at end of file diff --git a/docs/docs/support/troubleshoot.mdx b/docs/docs/support/troubleshoot.mdx index fa1cdaa4..f4d69e6e 100644 --- a/docs/docs/support/troubleshoot.mdx +++ b/docs/docs/support/troubleshoot.mdx @@ -35,33 +35,6 @@ open "/Applications/Python VERSION/Install Certificates.command" Replace `VERSION` with your installed Python version, such as `3.13`. -### Some images failed to pull on Amazon EC2 with GPU support {#some-images-failed-to-pull-on-amazon-ec2-with-gpu-support} - -When installing OpenRAG on an Amazon EC2 instance with GPU acceleration enabled, some Docker images might fail to pull. - -To resolve this issue, do the following: - -1. [Stop the OpenRAG containers](/manage-services#stop-and-start-containers). - -2. In your `~/.openrag` directory, edit the `docker-compose.gpu.yml` file as follows: - - ```yaml - services: - openrag-backend: - environment: - - NVIDIA_DRIVER_CAPABILITIES=compute,utility - - NVIDIA_VISIBLE_DEVICES=all - deploy: - resources: - reservations: - devices: - - driver: nvidia - count: all - capabilities: [gpu] - ``` - -3. [Restart the OpenRAG containers](/manage-services#stop-and-start-containers). - ### Application onboarding gets stuck on Google Chrome If the OpenRAG onboarding process gets stuck when using Google Chrome, try clearing your browser's cache.