Commit graph

123 commits

Author SHA1 Message Date
yangdx
46c13e23f0 Add confirmation dialog for pipeline cancellation
(cherry picked from commit 81e3496aa4)
2025-12-04 19:11:15 +08:00
yangdx
74d0a22020 Add pipeline cancellation feature with UI and i18n support
- Add cancelPipeline API endpoint
- Add cancel button to status dialog
- Update status response type
- Add cancellation UI translations
- Handle cancellation request states

(cherry picked from commit f89b5ab101)
2025-12-04 19:11:15 +08:00
yangdx
ec9b4862d0 Simplify pipeline status dialog by consolidating message sections
• Remove separate latest message section
• Combine into single pipeline messages area
• Add overflow-x-hidden for better display
• Change break-words to break-all
• Update translations across all locales

(cherry picked from commit 2476d6b7f8)
2025-12-04 19:09:01 +08:00
Raphael MANSUY
fe9b8ec02a
tests: stabilize integration tests + skip external services; fix multi-tenant API behavior and idempotency (#4)
* feat: Implement multi-tenant architecture with tenant and knowledge base models

- Added data models for tenants, knowledge bases, and related configurations.
- Introduced role and permission management for users in the multi-tenant system.
- Created a service layer for managing tenants and knowledge bases, including CRUD operations.
- Developed a tenant-aware instance manager for LightRAG with caching and isolation features.
- Added a migration script to transition existing workspace-based deployments to the new multi-tenant architecture.

* chore: ignore lightrag/api/webui/assets/ directory

* chore: stop tracking lightrag/api/webui/assets (ignore in .gitignore)

* feat: Initialize LightRAG Multi-Tenant Stack with PostgreSQL

- Added README.md for project overview, setup instructions, and architecture details.
- Created docker-compose.yml to define services: PostgreSQL, Redis, LightRAG API, and Web UI.
- Introduced env.example for environment variable configuration.
- Implemented init-postgres.sql for PostgreSQL schema initialization with multi-tenant support.
- Added reproduce_issue.py for testing default tenant access via API.

* feat: Enhance TenantSelector and update related components for improved multi-tenant support

* feat: Enhance testing capabilities and update documentation

- Updated Makefile to include new test commands for various modes (compatibility, isolation, multi-tenant, security, coverage, and dry-run).
- Modified API health check endpoint in Makefile to reflect new port configuration.
- Updated QUICK_START.md and README.md to reflect changes in service URLs and ports.
- Added environment variables for testing modes in env.example.
- Introduced run_all_tests.sh script to automate testing across different modes.
- Created conftest.py for pytest configuration, including database fixtures and mock services.
- Implemented database helper functions for streamlined database operations in tests.
- Added test collection hooks to skip tests based on the current MULTITENANT_MODE.

* feat: Implement multi-tenant support with demo mode enabled by default

- Added multi-tenant configuration to the environment and Docker setup.
- Created pre-configured demo tenants (acme-corp and techstart) for testing.
- Updated API endpoints to support tenant-specific data access.
- Enhanced Makefile commands for better service management and database operations.
- Introduced user-tenant membership system with role-based access control.
- Added comprehensive documentation for multi-tenant setup and usage.
- Fixed issues with document visibility in multi-tenant environments.
- Implemented necessary database migrations for user memberships and legacy support.

* feat(audit): Add final audit report for multi-tenant implementation

- Documented overall assessment, architecture overview, test results, security findings, and recommendations.
- Included detailed findings on critical security issues and architectural concerns.

fix(security): Implement security fixes based on audit findings

- Removed global RAG fallback and enforced strict tenant context.
- Configured super-admin access and required user authentication for tenant access.
- Cleared localStorage on logout and improved error handling in WebUI.

chore(logs): Create task logs for audit and security fixes implementation

- Documented actions, decisions, and next steps for both audit and security fixes.
- Summarized test results and remaining recommendations.

chore(scripts): Enhance development stack management scripts

- Added scripts for cleaning, starting, and stopping the development stack.
- Improved output messages and ensured graceful shutdown of services.

feat(starter): Initialize PostgreSQL with AGE extension support

- Created initialization scripts for PostgreSQL extensions including uuid-ossp, vector, and AGE.
- Ensured successful installation and verification of extensions.

* feat: Implement auto-select for first tenant and KB on initial load in WebUI

- Removed WEBUI_INITIAL_STATE_FIX.md as the issue is resolved.
- Added useTenantInitialization hook to automatically select the first available tenant and KB on app load.
- Integrated the new hook into the Root component of the WebUI.
- Updated RetrievalTesting component to ensure a KB is selected before allowing user interaction.
- Created end-to-end tests for multi-tenant isolation and real service interactions.
- Added scripts for starting, stopping, and cleaning the development stack.
- Enhanced API and tenant routes to support tenant-specific pipeline status initialization.
- Updated constants for backend URL to reflect the correct port.
- Improved error handling and logging in various components.

* feat: Add multi-tenant support with enhanced E2E testing scripts and client functionality

* update client

* Add integration and unit tests for multi-tenant API, models, security, and storage

- Implement integration tests for tenant and knowledge base management endpoints in `test_tenant_api_routes.py`.
- Create unit tests for tenant isolation, model validation, and role permissions in `test_tenant_models.py`.
- Add security tests to enforce role-based permissions and context validation in `test_tenant_security.py`.
- Develop tests for tenant-aware storage operations and context isolation in `test_tenant_storage_phase3.py`.

* feat(e2e): Implement OpenAI model support and database reset functionality

* Add comprehensive test suite for gpt-5-nano compatibility

- Introduced tests for parameter normalization, embeddings, and entity extraction.
- Implemented direct API testing for gpt-5-nano.
- Validated .env configuration loading and OpenAI API connectivity.
- Analyzed reasoning token overhead with various token limits.
- Documented test procedures and expected outcomes in README files.
- Ensured all tests pass for production readiness.

* kg(postgres_impl): ensure AGE extension is loaded in session and configure graph initialization

* dev: add hybrid dev helper scripts, Makefile, docker-compose.dev-db and local development docs

* feat(dev): add dev helper scripts and local development documentation for hybrid setup

* feat(multi-tenant): add detailed specifications and logs for multi-tenant improvements, including UX, backend handling, and ingestion pipeline

* feat(migration): add generated tenant/kb columns, indexes, triggers; drop unused tables; update schema and docs

* test(backward-compat): adapt tests to new StorageNameSpace/TenantService APIs (use concrete dummy storages)

* chore: multi-tenant and UX updates — docs, webui, storage, tenant service adjustments

* tests: stabilize integration tests + skip external services; fix multi-tenant API behavior and idempotency

- gpt5_nano_compatibility: add pytest-asyncio markers, skip when OPENAI key missing, prevent module-level asyncio.run collection, add conftest
- Ollama tests: add server availability check and skip markers; avoid pytest collection warnings by renaming helper classes
- Graph storage tests: rename interactive test functions to avoid pytest collection
- Document & Tenant routes: support external_ids for idempotency; ensure HTTPExceptions are re-raised
- LightRAG core: support external_ids in apipeline_enqueue_documents and idempotent logic
- Tests updated to match API changes (tenant routes & document routes)
- Add logs and scripts for inspection and audit
2025-12-04 16:04:21 +08:00
yangdx
1a0dc94f55 Update userPrompt label to clarify additional output functionality
• Rename "User Prompt" to clearer label
• Update across all language files
• Better describes prompt purpose
2025-09-26 10:40:20 +08:00
yangdx
d933b20562 Update query mode tooltips with clearer retrieval strategy descriptions 2025-09-25 00:17:59 +08:00
yangdx
a911cd1c43 Add copy functionality messages to chat locales
- Add copyEmpty message
- Add copySuccess variants
- Add copyFailed message
- Add manual copy instruction
- Support 5 languages (ar,en,fr,zh,zh_TW)
2025-09-24 23:28:49 +08:00
yangdx
188c912c9a Improve graph labels UI text clarity across all locales
• More specific "no matching nodes" text
• Change "Label" to "Search node name"
• Update placeholder text for clarity
• Apply changes to all 5 languages
• Enhance user experience consistency
2025-09-24 20:05:37 +08:00
yangdx
18968c6b6b Update webui assets 2025-09-23 23:46:17 +08:00
yangdx
2475194ab9 Update i18n translation 2025-09-23 23:41:31 +08:00
yangdx
34d91eb288 Update node type translation 2025-09-23 22:33:39 +08:00
yangdx
1d65fed692 Update graph search and label tooltips for clarity across all locales 2025-09-21 01:18:37 +08:00
yangdx
afb2df93f4 Update graph labels terminology from "labels" to "nodes" across locales 2025-09-21 00:31:17 +08:00
yangdx
446767ba50 feat: enhance graph refresh with dynamic tooltips and loading states
• Add refresh loading state with spinner
• Dynamic tooltips based on current label
• Improved refresh logic for labels
• Force dropdown re-render on refresh
• Update i18n for new tooltip messages
2025-09-20 22:46:09 +08:00
yangdx
9db8f2fce5 feat: Add popular labels and search APIs with history management
- Add popular/search label endpoints
- Implement SearchHistoryManager utility
- Replace client-side with server search
- Add graph data version tracking
- Update UI for better label discovery
2025-09-20 02:03:47 +08:00
yangdx
5311083f43 Rename "Process" entity type to "Method" across all components 2025-09-14 02:30:05 +08:00
yangdx
7060cf17f0 Add Process and Data entity types to LLM extraction system
• Add Process and Data to default types
• Update env.example configuration
• Add translations for new entities
• Support 5 languages (en/zh/fr/ar/tw)
2025-09-14 01:14:47 +08:00
yangdx
eeff0d5c67 Fix thinking state management and UI improvements for chat messages
- Reset expansion state on new thinking
- Prevent content leakage during thinking
- Update thinkingTime type to allow null
- Add memory leak prevention cleanup
- Add missing i18n translations
2025-09-08 18:47:41 +08:00
yangdx
2ba10dbb5b Add AI thinking process display with expandable UI and timing
- Add thinking fields to Message type
- Create expandable thinking section
- Track thinking time during streaming
2025-09-08 18:09:04 +08:00
yangdx
5a5d5e4a34 Add document translation key to all locale files 2025-09-03 12:50:27 +08:00
yangdx
4db43c43f3 Add product and other entity type translations
- Add "product" translations
- Add "other" translations
- Update 5 locale files
- Extend entity type coverage
2025-09-02 03:01:59 +08:00
yangdx
1f86543772 Update i18n translation and webui assets 2025-08-19 16:23:05 +08:00
yangdx
1af0803c62 fix(ui): fix selection state management in paginated views
- Replace DeselectDocumentsDialog with smart selection button
- Auto-reset selection on page/filter changes
- Remove deletion restrictions and update i18n
2025-08-17 10:38:12 +08:00
yangdx
e064534941 feat(ui): enhance ClearDocumentsDialog with loading spinner and timeout protection
- Add loading spinner animation during document clearing operation
- Implement 30-second timeout protection to prevent hanging operations
- Disable all interactive controls during clearing to prevent duplicate requests
- Add comprehensive error handling with automatic state reset
2025-08-17 01:33:39 +08:00
yangdx
a8f7e125c4 Update i18n translation 2025-07-31 01:59:26 +08:00
yangdx
93dede163d feat: move document list reset button to right side with ghost style 2025-07-30 22:37:22 +08:00
yangdx
0eac1a883a Feat: add file path sorting for document manager
- Add file_path sorting support to all database backends (JSON, Redis, PostgreSQL, MongoDB)
- Implement smart column header switching between "ID" and "File Name" based on display mode
- Add automatic sort field switching when toggling between ID and file name display
- Create composite indexes for workspace+file_path in PostgreSQL and MongoDB for better query performance
- Update frontend to maintain sort state when switching display modes
- Add internationalization support for "fileName" in English and Chinese locales

This enhancement improves user experience by providing intuitive file-based sorting
while maintaining performance through optimized database indexes.
2025-07-30 18:46:55 +08:00
yangdx
74eecc46e5 feat(pagination): Implement document list pagination backends and frontend UI
- Add pagination support to BaseDocStatusStorage interface and all implementations (PostgreSQL, MongoDB, Redis, JSON)
- Implement RESTful API endpoints for paginated document queries and status counts
- Create reusable pagination UI components with internationalization support
- Optimize performance with database-level pagination and efficient in-memory processing
- Maintain backward compatibility while adding configurable page sizes (10-200 items)
2025-07-30 17:58:32 +08:00
yangdx
84b09aa5da feat: add threshold status line to StatusCard with i18n support
- Add cosine_threshold, min_rerank_score, related_chunk_number to LightragStatus type
2025-07-28 02:51:36 +08:00
yangdx
958ed80b66 Update translation strings for document processing
- Renamed "maxParallelInsert" to be more descriptive
- Removed "maxTokens" from all translations
2025-07-28 02:24:41 +08:00
yangdx
5aceca0052 feat(webui): enhance status card with new health endpoint data
- Update StatusCard to display consolidated server info with parallel insert limits and summary settings
- Merge LLM and embedding configurations with async parameters for cleaner display
- Add new status fields to TypeScript interface (summary_language, max_parallel_insert, etc.)
2025-07-28 02:19:27 +08:00
yangdx
93e7a5b3aa Update i18n translation 2025-07-16 10:16:32 +08:00
zrguo
7c882313bb remove chunk_rerank_top_k 2025-07-15 11:52:34 +08:00
zrguo
4e425b1b59 Revert "update from main"
This reverts commit 1d0376d6a9.
2025-07-14 16:29:00 +08:00
zrguo
1d0376d6a9 update from main 2025-07-14 16:27:49 +08:00
zrguo
c9cbd2d3e0 Merge branch 'main' into rerank 2025-07-14 16:24:29 +08:00
zrguo
ef2115d437 Update token limit 2025-07-14 15:53:48 +08:00
yangdx
eb31ff0f90 Update i18n translation 2025-07-13 00:46:12 +08:00
yangdx
ab561196ff Feat: Added reranker config and lock status to status card of WebUI 2025-07-13 00:41:54 +08:00
yangdx
db22cad2c8 feat: add workspace and MAX_GRAPH_NODES to /health endpoint and webui 2025-07-07 01:39:48 +08:00
yangdx
bdcd55a871 Feat: Add delete upload file option to document deletion 2025-06-25 19:02:46 +08:00
yangdx
be41af3749 Update i18n files 2025-06-25 17:16:21 +08:00
yangdx
8ea7d7ad85 Feat: Add document deletion for WebUI 2025-06-25 17:09:10 +08:00
yangdx
fd5f61f166 Enhances property edit dialog with error handling and loading states
- Displays error messages when save operations fail
- Shows loading spinner during save attempts
- Improves UX with loading state indicators
- Updates translations for save-related strings in multiple languages
2025-05-11 12:35:51 +08:00
yangdx
12a8caa6ed Remove unsed i18n translate key 2025-05-08 04:17:25 +08:00
yangdx
d914de441e Add user_prompt to WebUI 2025-05-08 04:15:21 +08:00
yangdx
eb52878219 Update i18n translation 2025-04-23 02:26:50 +08:00
yangdx
e97e54b7a5 Feat: support query mode prefix in retrieval input 2025-04-23 01:26:34 +08:00
yangdx
3b8959051d Fix i18n translate error 2025-04-22 12:11:24 +08:00
yangdx
3b724c5207 Update i18n translation 2025-04-22 11:56:03 +08:00