Migrate from pip to uv package manager for faster builds
• Replace pip with uv in Dockerfile
• Remove constraints-offline.txt
• Add uv.lock for dependency pinning
• Use uv sync --frozen for builds
(cherry picked from commit 466de2070d)
This commit is contained in:
yangdx
Raphaël MANSUY
parent
8c3a325193
commit
aa61e82820
3 changed files with 1738 additions and 3305 deletions
94
Dockerfile.offline
Normal file
94
Dockerfile.offline
Normal file
|
|
@ -0,0 +1,94 @@
|
|||
# Frontend build stage
|
||||
FROM oven/bun:1 AS frontend-builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy frontend source code
|
||||
COPY lightrag_webui/ ./lightrag_webui/
|
||||
|
||||
# Build frontend assets for inclusion in the API package
|
||||
RUN cd lightrag_webui \
|
||||
&& bun install --frozen-lockfile \
|
||||
&& bun run build
|
||||
|
||||
# Python build stage - using uv for faster package installation
|
||||
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder
|
||||
|
||||
ENV DEBIAN_FRONTEND=noninteractive
|
||||
ENV UV_SYSTEM_PYTHON=1
|
||||
ENV UV_COMPILE_BYTECODE=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install system deps (Rust is required by some wheels)
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
curl \
|
||||
build-essential \
|
||||
pkg-config \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
|
||||
|
||||
ENV PATH="/root/.cargo/bin:/root/.local/bin:${PATH}"
|
||||
|
||||
# Ensure shared data directory exists for uv caches
|
||||
RUN mkdir -p /root/.local/share/uv
|
||||
|
||||
# Copy project metadata and sources
|
||||
COPY pyproject.toml .
|
||||
COPY setup.py .
|
||||
COPY uv.lock .
|
||||
COPY lightrag/ ./lightrag/
|
||||
|
||||
# Include pre-built frontend assets from the previous stage
|
||||
COPY --from=frontend-builder /app/lightrag/api/webui ./lightrag/api/webui
|
||||
|
||||
# Install base and API extras so CLI helpers work during build
|
||||
RUN uv sync --frozen --no-dev --extra api
|
||||
|
||||
# Prepare offline cache directory and pre-populate tiktoken data
|
||||
# Use uv run to execute commands from the virtual environment
|
||||
RUN mkdir -p /app/data/tiktoken \
|
||||
&& uv run lightrag-download-cache --cache-dir /app/data/tiktoken || status=$?; \
|
||||
if [ -n "${status:-}" ] && [ "$status" -ne 0 ] && [ "$status" -ne 2 ]; then exit "$status"; fi
|
||||
|
||||
# Final stage
|
||||
FROM python:3.12-slim
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install uv for package management
|
||||
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
|
||||
|
||||
ENV UV_SYSTEM_PYTHON=1
|
||||
|
||||
# Copy installed packages and application code
|
||||
COPY --from=builder /root/.local /root/.local
|
||||
COPY --from=builder /app/.venv /app/.venv
|
||||
COPY --from=builder /app/lightrag ./lightrag
|
||||
COPY pyproject.toml .
|
||||
COPY setup.py .
|
||||
COPY uv.lock .
|
||||
|
||||
# Ensure the installed scripts are on PATH
|
||||
ENV PATH=/app/.venv/bin:/root/.local/bin:$PATH
|
||||
|
||||
# Install dependencies with uv sync (uses locked versions from uv.lock)
|
||||
# IMPORTANT: Must be done BEFORE creating data/ directory to avoid setuptools error
|
||||
RUN uv sync --frozen --no-dev --extra api
|
||||
|
||||
# Create persistent data directories AFTER package installation
|
||||
RUN mkdir -p /app/data/rag_storage /app/data/inputs /app/data/tiktoken
|
||||
|
||||
# Copy offline cache into the newly created directory
|
||||
COPY --from=builder /app/data/tiktoken /app/data/tiktoken
|
||||
|
||||
# Point to the prepared cache
|
||||
ENV TIKTOKEN_CACHE_DIR=/app/data/tiktoken
|
||||
ENV WORKING_DIR=/app/data/rag_storage
|
||||
ENV INPUT_DIR=/app/data/inputs
|
||||
|
||||
# Expose API port
|
||||
EXPOSE 9621
|
||||
|
||||
ENTRYPOINT ["python", "-m", "lightrag.api.lightrag_server"]
|
||||
160
docs/UV_LOCK_GUIDE.md
Normal file
160
docs/UV_LOCK_GUIDE.md
Normal file
|
|
@ -0,0 +1,160 @@
|
|||
# uv.lock Update Guide
|
||||
|
||||
## What is uv.lock?
|
||||
|
||||
`uv.lock` is uv's lock file. It captures the exact version of every dependency, including transitive ones, much like:
|
||||
- Node.js `package-lock.json`
|
||||
- Rust `Cargo.lock`
|
||||
- Python Poetry `poetry.lock`
|
||||
|
||||
Keeping `uv.lock` in version control guarantees that everyone installs the same dependency set.
|
||||
|
||||
## When does uv.lock change?
|
||||
|
||||
### Situations where it does *not* change automatically
|
||||
|
||||
- Running `uv sync --frozen`
|
||||
- Building Docker images that call `uv sync --frozen`
|
||||
- Editing source code without touching dependency metadata
|
||||
|
||||
### Situations where it will change
|
||||
|
||||
1. **`uv lock` or `uv lock --upgrade`**
|
||||
|
||||
```bash
|
||||
uv lock # Resolve according to current constraints
|
||||
uv lock --upgrade # Re-resolve and upgrade to the newest compatible releases
|
||||
```
|
||||
|
||||
Use these commands after modifying `pyproject.toml`, when you want fresh dependency versions, or if the lock file was deleted or corrupted.
|
||||
|
||||
2. **`uv add`**
|
||||
|
||||
```bash
|
||||
uv add requests # Adds the dependency and updates both files
|
||||
uv add --dev pytest # Adds a dev dependency
|
||||
```
|
||||
|
||||
`uv add` edits `pyproject.toml` and refreshes `uv.lock` in one step.
|
||||
|
||||
3. **`uv remove`**
|
||||
|
||||
```bash
|
||||
uv remove requests
|
||||
```
|
||||
|
||||
This removes the dependency from `pyproject.toml` and rewrites `uv.lock`.
|
||||
|
||||
4. **`uv sync` without `--frozen`**
|
||||
|
||||
```bash
|
||||
uv sync
|
||||
```
|
||||
|
||||
Normally this only installs what is already locked. However, if `pyproject.toml` and `uv.lock` disagree or the lock file is missing, uv will regenerate and update `uv.lock`. In CI and production builds you should prefer `uv sync --frozen` to prevent unintended updates.
|
||||
|
||||
## Example workflows
|
||||
|
||||
### Scenario 1: Add a new dependency
|
||||
|
||||
```bash
|
||||
# Recommended: let uv handle both files
|
||||
uv add fastapi
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Add fastapi dependency"
|
||||
|
||||
# Manual alternative
|
||||
# 1. Edit pyproject.toml
|
||||
# 2. Regenerate the lock file
|
||||
uv lock
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Add fastapi dependency"
|
||||
```
|
||||
|
||||
### Scenario 2: Relax or tighten a version constraint
|
||||
|
||||
```bash
|
||||
# 1. Edit the requirement in pyproject.toml,
|
||||
# e.g. openai>=1.0.0,<2.0.0 -> openai>=1.5.0,<2.0.0
|
||||
|
||||
# 2. Re-resolve the lock file
|
||||
uv lock
|
||||
|
||||
# 3. Commit both files
|
||||
git add pyproject.toml uv.lock
|
||||
git commit -m "Update openai to >=1.5.0"
|
||||
```
|
||||
|
||||
### Scenario 3: Upgrade everything to the newest compatible versions
|
||||
|
||||
```bash
|
||||
uv lock --upgrade
|
||||
git diff uv.lock
|
||||
git add uv.lock
|
||||
git commit -m "Upgrade dependencies to latest compatible versions"
|
||||
```
|
||||
|
||||
### Scenario 4: Teammate syncing the project
|
||||
|
||||
```bash
|
||||
git pull # Fetch latest code and lock file
|
||||
uv sync --frozen # Install exactly what uv.lock specifies
|
||||
```
|
||||
|
||||
## Using uv.lock in Docker
|
||||
|
||||
```dockerfile
|
||||
RUN uv sync --frozen --no-dev --extra api
|
||||
```
|
||||
|
||||
`--frozen` guarantees reproducible builds because uv will refuse to deviate from the locked versions.
|
||||
`--extra api` install API server
|
||||
|
||||
## Frequently asked questions
|
||||
|
||||
- **`uv.lock` is almost 1 MB. Does that matter?**
|
||||
No. The file is read only during dependency resolution.
|
||||
|
||||
- **Should we commit `uv.lock`?**
|
||||
Yes. Commit it so collaborators and CI jobs share the same dependency graph.
|
||||
|
||||
- **Deleted the lock file by accident?**
|
||||
Run `uv lock` to regenerate it from `pyproject.toml`.
|
||||
|
||||
- **Can `uv.lock` and `requirements.txt` coexist?**
|
||||
They can, but maintaining both is redundant. Prefer relying on `uv.lock` alone whenever possible.
|
||||
|
||||
- **How do I inspect locked versions?**
|
||||
```bash
|
||||
uv tree
|
||||
grep -A5 'name = "openai"' uv.lock
|
||||
```
|
||||
|
||||
## Best practices
|
||||
|
||||
### Recommended
|
||||
|
||||
1. Commit `uv.lock` alongside `pyproject.toml`.
|
||||
2. Use `uv sync --frozen` in CI, Docker, and other reproducible environments.
|
||||
3. Use plain `uv sync` during local development if you want uv to reconcile the lock for you.
|
||||
4. Run `uv lock --upgrade` periodically to pick up the latest compatible releases.
|
||||
5. Regenerate the lock file immediately after changing dependency constraints.
|
||||
|
||||
### Avoid
|
||||
|
||||
1. Running `uv sync` without `--frozen` in CI or production pipelines.
|
||||
2. Editing `uv.lock` by hand—uv will overwrite manual edits.
|
||||
3. Ignoring lock file diffs in code reviews—unexpected dependency changes can break builds.
|
||||
|
||||
## Summary
|
||||
|
||||
| Command | Updates `uv.lock` | Typical use |
|
||||
|-----------------------|-------------------|-------------------------------------------|
|
||||
| `uv lock` | ✅ Yes | After editing constraints |
|
||||
| `uv lock --upgrade` | ✅ Yes | Upgrade to the newest compatible versions |
|
||||
| `uv add <pkg>` | ✅ Yes | Add a dependency |
|
||||
| `uv remove <pkg>` | ✅ Yes | Remove a dependency |
|
||||
| `uv sync` | ⚠️ Maybe | Local development; can regenerate the lock |
|
||||
| `uv sync --frozen` | ❌ No | CI/CD, Docker, reproducible builds |
|
||||
|
||||
Remember: `uv.lock` only changes when you run a command that tells it to. Keep it in sync with your project and commit it whenever it changes.
|
||||
Loading…
Add table
Reference in a new issue