Deploy with Docker

There are two different Docker Compose files. They deploy the same applications and containers, but to different environments.

• docker-compose.yml is an OpenRAG deployment with GPU support for accelerated AI processing.

• docker-compose-cpu.yml is a CPU-only version of OpenRAG for systems without GPU support. Use this Docker compose file for environments where GPU drivers aren't available.

Both Docker deployments depend on docling serve to be running on port 5001 on the host machine. This enables Mac MLX support for document processing. Installing OpenRAG with the TUI starts docling serve automatically, but for a Docker deployment you must manually start the docling serve process.

Prerequisites

• Python Version 3.10 to 3.13
• uv
• Podman (recommended) or Docker installed
• Docker Compose installed. If you're using Podman, use podman-compose or alias Docker compose commands to Podman commands.
• Create an OpenAI API key. This key is required to start OpenRAG, but you can choose a different model provider during Application Onboarding.
• Optional: GPU support requires an NVIDIA GPU with CUDA support and compatible NVIDIA drivers installed on the OpenRAG host machine. If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment.

Deploy OpenRAG with Docker Compose

To install OpenRAG with Docker Compose, do the following:

1. Clone the OpenRAG repository.

   git clone https://github.com/langflow-ai/openrag.git
   cd openrag

2. Install dependencies.

   uv sync

3. Copy the example .env file included in the repository root. The example file includes all environment variables with comments to guide you in finding and setting their values.

   cp .env.example .env

   Alternatively, create a new .env file in the repository root.

   touch .env

4. Set environment variables. The Docker Compose files will be populated with values from your .env. The following values are required to be set:

   OPENSEARCH_PASSWORD=your_secure_password
   OPENAI_API_KEY=your_openai_api_key
   LANGFLOW_SUPERUSER=admin
   LANGFLOW_SUPERUSER_PASSWORD=your_langflow_password
   LANGFLOW_SECRET_KEY=your_secret_key

   For more information on configuring OpenRAG with environment variables, see Environment variables.

5. Start docling serve on the host machine. Both Docker deployments depend on docling serve to be running on port 5001 on the host machine. This enables Mac MLX support for document processing.

   uv run python scripts/docling_ctl.py start --port 5001

6. Confirm docling serve is running.

   uv run python scripts/docling_ctl.py status

   Successful result:

   Status: running
   Endpoint: http://127.0.0.1:5001
   Docs: http://127.0.0.1:5001/docs
   PID: 27746

7. Deploy OpenRAG with Docker Compose based on your deployment type.

   For GPU-enabled systems, run the following commands:

   docker compose build
   docker compose up -d

   For environments without GPU support, run:

   docker compose -f docker-compose-cpu.yml up -d

   The OpenRAG Docker Compose file starts five containers:

   Container Name	Default Address	Purpose
   OpenRAG Backend	http://localhost:8000	FastAPI server and core functionality.
   OpenRAG Frontend	http://localhost:3000	React web interface for users.
   Langflow	http://localhost:7860	AI workflow engine and flow management.
   OpenSearch	http://localhost:9200	Vector database for document storage.
   OpenSearch Dashboards	http://localhost:5601	Database administration interface.

8. Verify installation by confirming all services are running.

   docker compose ps

   You can now access the application at:

   • Frontend: http://localhost:3000
   • Backend API: http://localhost:8000
   • Langflow: http://localhost:7860

9. Continue with Application Onboarding.

To stop docling serve when you're done with your OpenRAG deployment, run:

uv run python scripts/docling_ctl.py stop

Application onboarding

The first time you start OpenRAG, whether using the TUI or a .env file, you must complete application onboarding.

Most values from onboarding can be changed later in the OpenRAG Settings page, but there are important restrictions.

The language model provider and embeddings model provider can only be selected at onboarding, and you must use the same provider for your language model and embedding model. To change your provider selection later, you must completely reinstall OpenRAG.

The language model can be changed later in Settings, but the embeddings model cannot be changed later.

OpenAI

1. Enable Get API key from environment variable to automatically enter your key from the TUI-generated .env file.
2. Under Advanced settings, select your Embedding Model and Language Model.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. Continue with the Quickstart.

IBM watsonx.ai

1. Complete the fields for watsonx.ai API Endpoint, IBM API key, and IBM Project ID. These values are found in your IBM watsonx deployment.
2. Under Advanced settings, select your Embedding Model and Language Model.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. Continue with the Quickstart.

Ollama

tip

Ollama is not included with OpenRAG. To install Ollama, see the Ollama documentation.

1. Enter your Ollama server's base URL address. The default Ollama server address is http://localhost:11434. OpenRAG automatically transforms localhost to access services outside of the container, and sends a test connection to your Ollama server to confirm connectivity.
2. Select the Embedding Model and Language Model your Ollama server is running. OpenRAG retrieves the available models from your Ollama server.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. Continue with the Quickstart.

Container management commands

Manage your OpenRAG containers with the following commands. These commands are also available in the TUI's Status menu.

Upgrade containers

Upgrade your containers to the latest version while preserving your data.

docker compose pull
docker compose up -d --force-recreate

Rebuild containers (destructive)

Reset state by rebuilding all of your containers. Your OpenSearch and Langflow databases will be lost. Documents stored in the ./documents directory will persist, since the directory is mounted as a volume in the OpenRAG backend container.

docker compose up --build --force-recreate --remove-orphans

Remove all containers and data (destructive)

Completely remove your OpenRAG installation and delete all data. This deletes all of your data, including OpenSearch data, uploaded documents, and authentication.

docker compose down --volumes --remove-orphans --rmi local
docker system prune -f