Commit graph

3391 commits

Author SHA1 Message Date
yangdx
29bac49fb9 Handle empty query results by returning None instead of fail responses
• Return None when no context found
• Add structured failure metadata
• Use PROMPTS["fail_response"] for content
• Keep API compatible
2025-10-15 12:04:49 +08:00
yangdx
a81c122fd9 Bump API version to 0240 2025-10-14 14:03:42 +08:00
yangdx
130b4959dc Add PREPROCESSED (multimodal_processed) status for multimodal document processing
• Add DocStatus.PREPROCESSED enum value
• Update API routes and response models
• Add preprocessed filter in web UI
• Update localization files
• Handle preprocessed status in deletion
2025-10-14 14:02:05 +08:00
yangdx
64900b5457 Add frontend source code update warning
- Add frontend freshness check logic
- Warn when rebuild needed
- Remove --production from install docs
2025-10-14 10:59:21 +08:00
yangdx
f3740d822b Bump API version to 0239 2025-10-14 04:11:37 +08:00
yangdx
a8bbce3ae7 Use frozen lockfile for consistent frontend builds 2025-10-14 03:34:55 +08:00
yangdx
c0b1552e49 Remove .gitkeep file by ensuring webui dir exists on bun build 2025-10-14 03:22:48 +08:00
yangdx
8bf411314e Standardize build commands and remove --emptyOutDir flag
• Uses exact versions from `bun.lock` without updating
• Skips devDependencies, saving time and bandwidth
• The --emptyOutDir already configured in vite.config.ts
2025-10-14 02:49:25 +08:00
yangdx
ee45ab5104 Move frontend build check from setup.py to runtime server startup
• Remove complex setup.py build checks
• Add runtime frontend validation
• Update install docs with uv/bun steps
• Simplify setup.py to minimal form
• Check webui before server starts
2025-10-14 02:21:26 +08:00
yangdx
be9e6d1612 Exclude Frontend Build Artifacts from Git Repository
• Automate frontend build in CI/CD
• Add build validation checks
• Clean git repo of build artifacts
• Comprehensive build guide docs
• Smart setup.py build validation
2025-10-13 23:43:34 +08:00
yangdx
074f0c8b23 Update docstring for adelete_by_doc_id method clarity 2025-10-12 10:12:45 +08:00
yangdx
8a0098996c Update webui assets 2025-10-12 03:13:43 +08:00
yangdx
79f623a2a7 Bump core version to 1.4.9.3 2025-10-12 02:57:13 +08:00
yangdx
e1af1c6d14 Update webui assets 2025-10-12 02:51:22 +08:00
yangdx
f402ad276c Bump API version to 0238 2025-10-12 02:05:09 +08:00
yangdx
baab992431 Update pymilvus dependency from 2.5.2 to >=2.6.2 2025-10-11 22:42:02 +08:00
yangdx
289337b2c8 Bump API version to 0237 2025-10-11 13:45:29 +08:00
yangdx
e1e4f1b02c Fix get_by_ids to return None for missing records consistently 2025-10-11 13:34:26 +08:00
yangdx
7cddd56461 Revert core version to 1.4.9..2 2025-10-11 13:06:58 +08:00
yangdx
9be22dd666 Preserve ordering in get_by_ids methods across all storage implementations
- Fix result ordering in vector stores
- Update KV storage get_by_ids methods
- Maintain order in doc status storage
- Return None for missing IDs
2025-10-11 12:37:59 +08:00
yangdx
a5c05f1b92 Add offline deployment support with cache management and layered deps
• Add tiktoken cache downloader CLI
• Add layered offline dependencies
• Add offline requirements files
• Add offline deployment guide
2025-10-11 10:28:14 +08:00
yangdx
12facac506 Enhance graph API endpoints with detailed docs and field validation
- Remove redundant README section
- Add Pydantic field validation
- Expand endpoint docstrings
- Include request/response examples
- Document merge operation benefits
2025-10-10 12:49:00 +08:00
yangdx
85d1a563b3 Merge branch 'adminunblinded/main' 2025-10-10 12:31:47 +08:00
NeelM0906
b7c77396a0 Fix entity/relation creation endpoints to properly update vector stores
- Changed create_entity to use rag.acreate_entity() instead of direct graph manipulation
  - Changed create_relation to use rag.acreate_relation() instead of direct graph manipulation
  - This ensures vector embeddings are created and entities/relations are searchable
  - Adds proper concurrency locks and metadata population
2025-10-09 17:02:17 -04:00
NeelM0906
f6d1fb98ac Fix Linting errors 2025-10-09 16:52:22 -04:00
yangdx
b3ed264707 Refactor PostgreSQL retry config to use centralized configuration
• Move retry config to ClientManager
• Remove env var parsing from PostgreSQLDB
• Add config params to test setup
2025-10-10 03:44:13 +08:00
yangdx
e758204ab2 Add PostgreSQL connection retry mechanism with comprehensive error handling
• Implement connection retry with backoff
• Add transient error detection
• Pool management with timeout guards
2025-10-10 03:06:01 +08:00
Yasiru Rangana
ae9f4ae73f fix: Remove trailing whitespace for pre-commit linting 2025-10-09 15:01:53 +11:00
NeelM0906
9f44e89de7 Add knowledge graph manipulation endpoints
Added three new REST API endpoints for direct knowledge graph manipulation:

- POST /graph/entity/create: Create new entities in the knowledge graph
- POST /graph/relation/create: Create relationships between entities
- POST /graph/entities/merge: Merge duplicate/misspelled entities while preserving relationships

The merge endpoint is particularly useful for consolidating entities discovered after document processing, fixing spelling errors, and cleaning up the knowledge graph. All relationships from source entities are transferred to the target entity, with intelligent handling of duplicate relationships.

Updated API documentation in lightrag/api/README.md with usage examples for all three endpoints.
2025-10-08 15:59:47 -04:00
Yasiru Rangana
ec40b17eea feat: Add token tracking support to openai_embed function
- Add optional token_tracker parameter to openai_embed()
- Track prompt_tokens and total_tokens for embedding API calls
- Enables monitoring of embedding token usage alongside LLM calls
- Maintains backward compatibility with existing code
2025-10-08 14:36:08 +11:00
yangdx
f1e0110716 Merge branch 'kevinnkansah/main' 2025-10-07 23:04:59 +08:00
yangdx
f2c0b41e78 Make PostgreSQL statement_cache_size configuration optional
• Remove forced int conversion
• Allow None values for cache size
• Add conditional parameter setting
2025-10-07 22:57:21 +08:00
Aleks Vujić
dd8f44e621
Fixed typo in log message when creating new graph file 2025-10-07 14:30:05 +02:00
kevinnkansah
fdcb034da0 chore: distinguish settings 2025-10-06 12:01:40 +02:00
kevinnkansah
22a7b482c5 fix: renamed PostGreSQL options env variable and allowed LRU cache to be an optional env variable 2025-10-06 11:56:09 +02:00
kevinnkansah
d8a9617c0e fix: fix: asyncpg bouncer connection pool error
Prepared statement caching is disabled by setting
`statement_cache_size=0` in the `asyncpg` connection pool parameters.
This is necessary to prevent
`asyncpg.exceptions.InvalidSQLStatementNameError` when using
transaction-level connection poolers like Supabase Supavisor or
pgbouncer, which do not support prepared statements.
2025-10-06 00:36:25 +02:00
kevinnkansah
108cdbe133 feat: add options for PostGres connection 2025-10-05 23:29:04 +02:00
yangdx
d473f635d8 Update webui assets 2025-10-05 14:02:18 +08:00
yangdx
aac787bafb Clarify chunk tracking log message in _build_llm_context 2025-10-05 13:33:55 +08:00
yangdx
457d51952e Add doc_name field to full docs storage
- Store file_path in full_docs storage
- Update PostgreSQL implementation by map file_path to doc_name
- Other storage implementation automatically handles the new field
2025-10-05 11:44:27 +08:00
yangdx
1574fec7f0 Update webui assets 2025-10-05 10:41:46 +08:00
yangdx
b5f8376756 Update webui assets 2025-10-05 09:27:38 +08:00
yangdx
dde728a32f Bump core version to 1.5.0 and API to 0236 2025-10-05 09:25:57 +08:00
Jon
cf2a024e37 feat: Add endpoint and UI to retry failed documents
Add a new `/documents/reprocess_failed` API endpoint and corresponding
UI button to retry processing of failed and pending documents. This
addresses a common recovery scenario when document processing fails due
to server crashes, network errors, or LLM service outages.

Backend changes:
- Add ReprocessResponse model with status, message, and track_id fields
- Add POST /documents/reprocess_failed endpoint that triggers background
  reprocessing of FAILED, PENDING, and interrupted PROCESSING documents
- Reuses existing apipeline_process_enqueue_documents for consistency
- Includes comprehensive docstring and logging for observability

Frontend changes:
- Add TypeScript types and API function for the new endpoint
- Add retry handler with intelligent polling (fast refresh → normal)
- Add "Retry Failed" button in Documents page toolbar
- Button disabled when pipeline is busy to prevent duplicate operations
- Complete i18n support (English and Chinese translations)

This feature provides a convenient way to recover from processing
failures without requiring a full filesystem rescan.
2025-10-04 16:46:29 -04:00
yangdx
b9c37bd937 Fix linting 2025-10-03 02:10:02 +08:00
yangdx
112349ed5b Modernize type hints and remove Python 3.8 compatibility code
• Use collections.abc.AsyncIterator only
• Remove sys.version_info checks
• Use union syntax for None types
• Simplify string emptiness checks
2025-10-02 23:15:42 +08:00
yangdx
cec784f60e Update webui assets 2025-10-02 22:02:42 +08:00
yangdx
a250d881f2 Update webui assets 2025-10-02 21:24:39 +08:00
yangdx
6bf6f43d96 Remove bold formatting from instruction headers in prompts 2025-10-02 00:58:03 +08:00
yangdx
bb6138e748 fix(prompt): Clarify reference section restrictions in prompt templates 2025-10-01 22:35:26 +08:00