Skip to main content

# Install OpenRAG containers

OpenRAG has two Docker Compose files. Both files deploy the same applications and containers locally, but they are for different environments.

- `docker-compose.yml` is an OpenRAG deployment with GPU support for accelerated AI processing. This Docker Compose file requires an NVIDIA GPU with CUDA support.

- `docker-compose-cpu.yml` is a CPU-only version of OpenRAG for systems without NVIDIA GPU support. Use this Docker Compose file for environments where GPU drivers aren't available.

## Prerequisites​

- Install Python Version 3.10 to 3.13
- Install uv
- Install Podman (recommended) or Docker
- Install Docker Compose. If using Podman, use podman-compose or alias Docker compose commands to Podman commands.
- Create an OpenAI API key. This key is required to start OpenRAG, but you can choose a different model provider during Application Onboarding.
- Optional: Install GPU support with an NVIDIA GPU, CUDA support, and compatible NVIDIA drivers on the OpenRAG host machine. If you don't have GPU capabilities, OpenRAG provides an alternate CPU-only deployment.

## Install OpenRAG with Docker Compose​

To install OpenRAG with Docker Compose, do the following:

1. Clone the OpenRAG repository.

       git clone https://github.com/langflow-ai/openrag.git
       cd openrag

2. Install dependencies.

       uv sync

3. Copy the example `.env` file included in the repository root. The example file includes all environment variables with comments to guide you in finding and setting their values.

       cp .env.example .env

   Alternatively, create a new `.env` file in the repository root.

       touch .env

4. The Docker Compose files are populated with the values from your `.env` file. The following values must be set:

       OPENSEARCH_PASSWORD=your_secure_password
       OPENAI_API_KEY=your_openai_api_key
       LANGFLOW_SECRET_KEY=your_secret_key

   `OPENSEARCH_PASSWORD` can be automatically generated when using the TUI, but for a Docker Compose installation, you can set it manually instead. To generate an OpenSearch admin password, see the OpenSearch documentation.

   The `OPENAI_API_KEY` is found in your OpenAI account.

   `LANGFLOW_SECRET_KEY` is automatically generated when using the TUI, and Langflow will also auto-generate it if not set. For more information, see the Langflow documentation.

   The following Langflow configuration values are optional but important to consider:

       LANGFLOW_SUPERUSER=admin
       LANGFLOW_SUPERUSER_PASSWORD=your_langflow_password

   `LANGFLOW_SUPERUSER` defaults to `admin`. You can omit it or set it to a different username. `LANGFLOW_SUPERUSER_PASSWORD` is optional. If omitted, Langflow runs in autologin mode with no password required. If set, Langflow requires password authentication.

   For more information on configuring OpenRAG with environment variables, see Environment variables.

5. Start `docling serve` on the host machine. OpenRAG Docker installations require that `docling serve` is running on port 5001 on the host machine. This enables Mac MLX support for document processing.

       uv run python scripts/docling_ctl.py start --port 5001

6. Confirm `docling serve` is running.

       uv run python scripts/docling_ctl.py status

   Make sure the response shows that `docling serve` is running, for example:

       Status: running
       Endpoint: http://127.0.0.1:5001
       Docs: http://127.0.0.1:5001/docs
       PID: 27746

7. Deploy OpenRAG locally with Docker Compose based on your deployment type.

   - docker-compose.yml
   - docker-compose-cpu.yml

       docker compose build
       docker compose up -d

       docker compose -f docker-compose-cpu.yml up -d

   The OpenRAG Docker Compose file starts five containers:

   Container Name | Default Address | Purpose
   --- | --- | ---
   OpenRAG Backend | http://localhost:8000 | FastAPI server and core functionality.
   OpenRAG Frontend | http://localhost:3000 | React web interface for users.
   Langflow | http://localhost:7860 | AI workflow engine and flow management.
   OpenSearch | http://localhost:9200 | Vector database for document storage.
   OpenSearch Dashboards | http://localhost:5601 | Database administration interface.

8. Verify installation by confirming all services are running.

       docker compose ps

   You can now access OpenRAG at the following endpoints:

   - Frontend: http://localhost:3000
   - Backend API: http://localhost:8000
   - Langflow: http://localhost:7860

9. Continue with Application Onboarding.

To stop `docling serve` when you're done with your OpenRAG deployment, run:

    uv run python scripts/docling_ctl.py stop

## Application onboarding​

The first time you start OpenRAG, whether using the TUI or a `.env` file, it's recommended that you complete application onboarding.

To skip onboarding, click Skip onboarding.

warning

Most values from onboarding can be changed later in the OpenRAG Settings page, but there are important restrictions.

The language model provider and embeddings model provider can only be selected at onboarding. To change your provider selection later, you must reinstall OpenRAG.

You must use the same provider for your language model and embedding model, unless you're using Ollama.

Choose one LLM provider and complete only those steps:

- OpenAI
- IBM watsonx.ai
- Ollama

1. Enable Get API key from environment variable to automatically enter your key from the TUI-generated `.env` file. Alternatively, paste an OpenAI API key into the field.
2. Under Advanced settings, select your Embedding Model and Language Model.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. To complete the onboarding tasks, click What is OpenRAG, and then click Add a Document.
6. Continue with the Quickstart.

1. Complete the fields for watsonx.ai API Endpoint, IBM Project ID, and IBM API key. These values are found in your IBM watsonx deployment.
2. Under Advanced settings, select your Embedding Model and Language Model.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. To complete the onboarding tasks, click What is OpenRAG, and then click Add a Document.
6. Continue with the Quickstart.

tip

Ollama is not included with OpenRAG. To install Ollama, see the Ollama documentation.

1. Enter your Ollama server's base URL address. The default Ollama server address is `http://localhost:11434`. OpenRAG automatically transforms `localhost` to access services outside of the container, and sends a test connection to your Ollama server to confirm connectivity.
2. Select the Embedding Model and Language Model your Ollama server is running. OpenRAG retrieves the available models from your Ollama server.
3. To load 2 sample PDFs, enable Sample dataset. This is recommended, but not required.
4. Click Complete.
5. To complete the onboarding tasks, click What is OpenRAG, and then click Add a Document.
6. Continue with the Quickstart.

## Container management commands​

Manage your OpenRAG containers with the following commands. These commands are also available in the TUI's Status menu.

### Upgrade containers​

Upgrade your containers to the latest version while preserving your data.

    docker compose pull
    docker compose up -d --force-recreate

### Rebuild containers (destructive)​

Reset state by rebuilding all of your containers. Your OpenSearch and Langflow databases will be lost. Documents stored in the `./documents` directory will persist, since the directory is mounted as a volume in the OpenRAG backend container.

    docker compose up --build --force-recreate --remove-orphans

### Remove all containers and data (destructive)​

Completely remove your OpenRAG installation and delete all data. This deletes all of your data, including OpenSearch data, uploaded documents, and authentication.

    docker compose down --volumes --remove-orphans --rmi local
    docker system prune -f