feat: add optional Langfuse observability integration

This contribution adds optional Langfuse support for LLM observability and tracing.
Langfuse provides a drop-in replacement for the OpenAI client that automatically
tracks all LLM interactions without requiring code changes.

Features:
- Optional Langfuse integration with graceful fallback
- Automatic LLM request/response tracing
- Token usage tracking
- Latency metrics
- Error tracking
- Zero code changes required for existing functionality

Implementation:
- Modified lightrag/llm/openai.py to conditionally use Langfuse's AsyncOpenAI
- Falls back to standard OpenAI client if Langfuse is not installed
- Logs observability status on import

Configuration:
To enable Langfuse tracing, install the observability extras and set environment variables:

```bash
pip install lightrag-hku[observability]

export LANGFUSE_PUBLIC_KEY="your_public_key"
export LANGFUSE_SECRET_KEY="your_secret_key"
export LANGFUSE_HOST="https://cloud.langfuse.com"  # or your self-hosted instance
```

If Langfuse is not installed or environment variables are not set, LightRAG
will use the standard OpenAI client without any functionality changes.

Changes:
- Modified lightrag/llm/openai.py (added optional Langfuse import)
- Updated pyproject.toml with optional 'observability' dependencies

Dependencies (optional):
- langfuse>=3.8.1

lightrag/llm/openai.py

@@ -10,8 +10,18 @@ import pipmaster as pm
 if not pm.is_installed("openai"):
     pm.install("openai")
 
+# Try to import Langfuse for LLM observability (optional)
+# Falls back to standard OpenAI client if not available
+try:
+    from langfuse.openai import AsyncOpenAI
+    LANGFUSE_ENABLED = True
+    logger.info("Langfuse observability enabled for OpenAI client")
+except ImportError:
+    from openai import AsyncOpenAI
+    LANGFUSE_ENABLED = False
+    logger.debug("Langfuse not available, using standard OpenAI client")
+
 from openai import (
-    AsyncOpenAI,
     APIConnectionError,
     RateLimitError,
     APITimeoutError,

pyproject.toml

@@ -113,6 +113,11 @@ offline = [
     "lightrag-hku[offline-docs,offline-storage,offline-llm]",
 ]
 
+observability = [
+    # LLM observability and tracing dependencies
+    "langfuse>=3.8.1",
+]
+
 [project.scripts]
 lightrag-server = "lightrag.api.lightrag_server:main"
 lightrag-gunicorn = "lightrag.api.run_with_gunicorn:main"