gmakstutis/LightRAG
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
4611 commits 5 branches 72 tags 705 MiB
Commit graph

2971 commits

Author SHA1 Message Date
yangdx
7206c07468 Remove deprecated content field from doc status 
- Drop content column from LIGHTRAG_DOC_STATUS
- Clean up doc status handling code
- Maintain backward compatibility
 2025-07-29 23:19:36 +08:00
yangdx
1e1adcb64a Add index on track_id column in doc status table of PostgreSQL 2025-07-29 23:03:09 +08:00
yangdx
6014b9bf73 feat: add track_id support for document processing progress monitoring 
- Add get_docs_by_track_id() method to all storage backends (MongoDB, PostgreSQL, Redis, JSON)
- Implement automatic track_id generation with upload_/insert_ prefixes
- Add /track_status/{track_id} API endpoint for frontend progress queries
- Create database indexes for efficient track_id lookups
- Enable real-time document processing status tracking across all storage types
 2025-07-29 22:24:21 +08:00
yangdx
dafdf92715 Remove content fallback logic in get_docs_by_status from Redis 2025-07-29 19:13:07 +08:00
yangdx
40a4cacee0 Merge branch 'main' into remove-content-from-doc-status 2025-07-29 16:15:01 +08:00
yangdx
92bbb7a1b3 Remove content fallback and standardize doc status handling 
- Remove content_summary fallback logic
- Standardize doc status processing
- Handle missing file_path consistently
 2025-07-29 16:13:51 +08:00
yangdx
24c36d876c Remove content field from DocProcessingStatus, update MongoDB and PostgreSQL implementation 2025-07-29 14:52:45 +08:00
administrator
9c3e1505b5 fix timeout issue 2025-07-29 13:38:46 +07:00
yangdx
8274ed52d1 feat: separate document content from doc_status to improve performance 
This optimization significantly improves doc_status query/update performance by avoiding large string operations during frequent status checks.
 2025-07-29 14:20:07 +08:00
administrator
c26dfa33de Fix: corrected unterminated f-string in config.py 2025-07-29 11:21:23 +07:00
yangdx
9923821d75 refactor: Remove deprecated max_token_size from embedding configuration 
This parameter is no longer used. Its removal simplifies the API and clarifies that token length management is handled by upstream text chunking logic rather than the embedding wrapper.
 2025-07-29 10:49:35 +08:00
yangdx
f4c2dc327d Fix linting 2025-07-29 09:57:41 +08:00
yangdx
75d1b1e9f8 Update Ollama context length configuration 
- Rename OLLAMA_NUM_CTX to OLLAMA_LLM_NUM_CTX
- Increase default context window size
- Add requirement for minimum context size
- Update documentation examples
 2025-07-29 09:53:37 +08:00
yangdx
645f81f7c8 fixes a critical bug where Ollama options were not being applied correctly 
`dict.update()` modifies the dictionary in-place and returns `None`.
 2025-07-29 09:52:25 +08:00
Michele Comitini
bd94714b15 options needs to be passed to ollama client embed() method 
Fix line length

Create binding_options.py

Remove test property

Add dynamic binding options to CLI and environment config

Automatically generate command-line arguments and environment variable
support for all LLM provider bindings using BindingOptions. Add sample
.env generation and extensible framework for new providers.

Add example option definitions and fix test arg check in OllamaOptions

Add options_dict method to BindingOptions for argument parsing

Add comprehensive Ollama binding configuration options

ruff formatting Apply ruff formatting to binding_options.py

Add Ollama separate options for embedding and LLM

Refactor Ollama binding options and fix class var handling

The changes improve how class variables are handled in binding options
and better organize the Ollama-specific options into LLM and embedding
subclasses.

Fix typo in arg test.

Rename cls parameter to klass to avoid keyword shadowing

Fix Ollama embedding binding name typo

Fix ollama embedder context param name

Split Ollama options into LLM and embedding configs with mixin base

Add Ollama option configuration to LLM and embeddings in lightrag_server

Update sample .env generation and environment handling

Conditionally add env vars and cmdline options only when ollama bindings
are used. Add example env file for Ollama binding options.
 2025-07-28 12:05:40 +02:00
yangdx
ee53e43568 Update webui assets 2025-07-28 02:52:32 +08:00
yangdx
769f77ef8f Update webui assets 2025-07-28 02:26:07 +08:00
yangdx
98ac6fb3f0 Bump api version to 0192 2025-07-28 01:42:51 +08:00
yangdx
f2ffff063b feat: refactor ollama server configuration management 
- Add ollama_server_infos attribute to LightRAG class with default initialization
- Move default values to constants.py for centralized configuration
- Refactor OllamaServerInfos class with property accessors and CLI support
- Update OllamaAPI to get configuration through rag object instead of direct import
- Add command line arguments for simulated model name and tag
- Fix type imports to avoid circular dependencies
 2025-07-28 01:38:35 +08:00
yangdx
598eecd06d Refactor: Rename llm_model_max_token_size to summary_max_tokens 
This commit renames the parameter 'llm_model_max_token_size' to 'summary_max_tokens' for better clarity, as it specifically controls the token limit for entity relation summaries.
 2025-07-28 00:49:08 +08:00
yangdx
d0d57a45b6 feat: add environment variables to /health endpoint and centralize defaults 
- Add 9 environment variables to /health endpoint configuration section
- Centralize default constants in lightrag/constants.py for consistency
- Update config.py to use centralized defaults for better maintainability
 2025-07-28 00:30:56 +08:00
yangdx
9c4e98ec3b Unify entity extraction prompt between passes 
- Disallow hallucinated info in descriptions
- Align reminder steps with main extraction
 2025-07-27 23:06:55 +08:00
Daniel.y
4eef9f3778
Merge pull request #1845 from AkosLukacs/patch-2 
Better prompt for entity description extraction to avoid hallucinations
 2025-07-27 22:38:08 +08:00
yangdx
3951a44666 Revert file_path build method, built from related chunks 2025-07-27 21:56:20 +08:00
yangdx
d70c584d80 Bump api version to 0191 2025-07-27 21:24:53 +08:00
yangdx
f2d051eea5 Fix: Improve keyword extraction prompt for robust JSON output. 
*   Emphasize strict JSON output in key extration prompt
*   Clean up prompt examples in key extration prompt
*   Log raw LLM response on JSON error
 2025-07-27 21:10:47 +08:00
yangdx
3f5ade47cd Update README 2025-07-27 17:26:49 +08:00
yangdx
e09929b42e Refine rerank filtering log message for clarity 2025-07-27 16:57:38 +08:00
yangdx
f4bca7bfb2 Fix linting 2025-07-27 16:50:45 +08:00
yangdx
a9565d7379 feat: Skip rerank filtering when min_rerank_score is 0.0 2025-07-27 16:50:12 +08:00
yangdx
ebaff228aa feat: Add rerank score filtering with configurable threshold 
- Add DEFAULT_MIN_RERANK_SCORE constant (default: 0.0)
- Add MIN_RERANK_SCORE environment variable support
- Filter chunks with rerank scores below threshold in process_chunks_unified
- Add info-level logging for filtering operations
- Handle empty results gracefully after filtering
- Maintain backward compatibility with non-reranked chunks
 2025-07-27 16:37:44 +08:00
yangdx
99e3812c38 refactor: unify file_path handling across merge and rebuild functions 
- Replace simple string concatenation with build_file_path() in:
  - _merge_edges_then_upsert
  - _rebuild_single_entity
  - _rebuild_single_relationship
- Ensures consistent deduplication, length limiting, and error handling
- Aligns with existing _merge_nodes_then_upsert implementation
 2025-07-27 12:37:24 +08:00
yangdx
cf1ca39b3f Refine entity continuation prompt to avoid duplicates. 
- Clarify finding missing entities
- Instruct not to repeat extractions
 2025-07-27 10:48:29 +08:00
yangdx
0dfbce0bb4 Update the README to clarify the explanation of concurrent processes. 2025-07-27 10:39:28 +08:00
yangdx
055629d30d Reduce default max total tokens to 30k 2025-07-27 10:33:06 +08:00
yangdx
a67f93acc9 Replace hardcoded max tokens with DEFAULT_MAX_TOTAL_TOKENS constant 
- Use constant in process_chunks_unified
- Update WebUI default to match (32000)
 2025-07-26 11:23:54 +08:00
yangdx
7b915b34f6 Refactor: move build_file_path function from operate.py to utils.py 2025-07-26 10:52:59 +08:00
yangdx
c8c3545454 refactor: extract file path length limit to shared constant 
• Add DEFAULT_MAX_FILE_PATH_LENGTH constant
• Replace hardcoded 4090 in Milvus impl
 2025-07-26 10:45:03 +08:00
yangdx
8e7014d366 Merge branch 'separator_file_path' 2025-07-26 10:39:03 +08:00
yangdx
a943265257 fix: preserve file path order in build_file_path function 2025-07-26 10:21:32 +08:00
yangdx
6efa8ab263 Improve file path length warning message clarity and urgency 
• Change debug to warning level
• Simplify message wording
 2025-07-26 10:00:18 +08:00
yangdx
e7baf54ec2 Update webui assets 2025-07-26 08:43:12 +08:00
xuewei
56c3cb2dbe Improve build_file_path log 2025-07-26 08:38:02 +08:00
okxuewei
912fc0fc31
Merge branch 'HKUDS:main' into separator_file_path 2025-07-26 08:17:35 +08:00
yangdx
b3c2987006 Reduce default MAX_TOKENS from 32000 to 10000 2025-07-26 08:13:49 +08:00
xuewei
a49b7758e1 Merge branch 'main' of https://github.com/okxuewei/LightRAG into separator_file_path 2025-07-26 00:47:48 +08:00
xuewei
b4da3de7d9 Improve file_path drop policy 2025-07-26 00:46:02 +08:00
xuewei
55e2678a1e Improve file_path FieldSchema 4090 2025-07-26 00:22:25 +08:00
yangdx
6a99d7ac28 Update webui assets 2025-07-25 22:03:58 +08:00
yangdx
4ae44bb24b Bump core version to 1.4.5 and api version to 0190 2025-07-25 11:15:04 +08:00