Commit graph

576 commits

Author SHA1 Message Date
clssck
b6074b9a81 chore(lightrag, lightrag_webui): improve code quality and security
- Extract PostgreSQL storage check into named variable for clarity
- Move APIRouter initialization into create_table_routes function scope
- Add robust type handling for database query results
- Add input validation for table names and pagination parameters
- Add regex-based SQL injection prevention for table name sanitization
- Improve clipboard copy fallback logic and error handling
- Add memoization for JSON serialization to prevent unnecessary recalculations
- Hide meta column from table explorer UI display
- Sort table columns alphabetically for consistent ordering
- Add keyboard accessibility to status filter buttons
- Add preprocessed status filter to document manager
- Update @tanstack/react-query from 5.60.0 to 5.87.1
- Extract dev storage config into constant to reduce duplication
- Update documentation comments for clarity
2025-11-27 21:39:42 +01:00
clssck
a9edadef45 feat: add Table Explorer feature with dynamic table data fetching and schema display
- Implemented Table Explorer component to allow users to select and view database tables.
- Added API calls for fetching table list, schema, and paginated data.
- Introduced row detail modal for displaying and copying row data.
- Enhanced DataTable component to support row click events.
- Updated UI components for better user experience and accessibility.
- Added mock data for development mode to facilitate testing.
- Updated localization files to include new terms related to tables.
- Modified settings store to include storage configuration for conditional UI rendering.
- Improved styling and layout for various components to align with new design standards.
2025-11-27 18:27:14 +01:00
clssck
48c7732edc feat: add automatic entity resolution with 3-layer matching
Implement automatic entity resolution to prevent duplicate nodes in the
knowledge graph. The system uses a 3-layer approach:

1. Case-insensitive exact matching (free, instant)
2. Fuzzy string matching >85% threshold (free, instant)
3. Vector similarity + LLM verification (for acronyms/synonyms)

Key features:
- Pre-resolution phase prevents race conditions in parallel processing
- Numeric suffix detection blocks false matches (IL-4 ≠ IL-13)
- PostgreSQL alias cache for fast lookups on subsequent ingestion
- Configurable thresholds via environment variables

Bug fixes included:
- Fix fuzzy matching false positives for numbered entities
- Fix alias cache not being populated (missing db parameter)
- Skip entity_aliases table from generic id index creation

New files:
- lightrag/entity_resolution/ - Core resolution module
- tests/test_entity_resolution/ - Unit tests
- docker/postgres-age-vector/ - Custom PG image with pgvector + AGE
- docker-compose.test.yml - Integration test environment

Configuration (env.example):
- ENTITY_RESOLUTION_ENABLED=true
- ENTITY_RESOLUTION_FUZZY_THRESHOLD=0.85
- ENTITY_RESOLUTION_VECTOR_THRESHOLD=0.5
- ENTITY_RESOLUTION_MAX_CANDIDATES=3
2025-11-27 15:35:02 +01:00
yangdx
8c07c91833 Remove deprecated response_type parameter from query settings
- Bump API version to 0254
- Remove response format UI controls
- Hard-code response_type in query params
- Add migration for version 19
- Clean up settings store structure
2025-11-12 12:19:30 +08:00
yangdx
80f2e691fc Remove redundant i18n import triggered the Vite “dynamic + static import” warning 2025-11-09 02:48:11 +08:00
yangdx
678e17bb5c Revert "fix(ui): Remove dynamic import for i18n in settings store"
This reverts commit 0216325e0f.
2025-11-06 21:00:47 +08:00
yangdx
0216325e0f fix(ui): Remove dynamic import for i18n in settings store 2025-11-06 16:15:32 +08:00
yangdx
7ccc1fdd27 Add frontend rebuild warning indicator to version display
- Return bool from check_frontend_build()
- Add ⚠️ symbol to outdated versions
- Show tooltip with rebuild message
- Add translations for warning text
- Fix tailwind config filename typo
2025-10-31 06:09:46 +08:00
yangdx
58c83f9da5 Add auto-refresh of popular labels when pipeline completes
• Monitor pipeline busy->idle transitions
• Reload labels on dropdown open if needed
• Add onBeforeOpen callback to AsyncSelect
• Clear refresh flags after processing
• Improve label sync with backend state
2025-10-31 04:45:35 +08:00
yangdx
71b27ec4aa Optimize property edit dialog to use trimmed value consistently 2025-10-31 00:08:02 +08:00
yangdx
4cbd876126 feat: Update node color and legent after entity_type changed
- Move color constants to utils module
- Extract resolveNodeColor function
- Update node colors on type changes
- Simplify hook color logic
2025-10-31 00:03:55 +08:00
yangdx
79a17c3f7f Fix graph value handling for entity_id updates
• Use finalValue for entity_id changes
• Keep original value for other props
• Fix property update logic
2025-10-30 23:43:46 +08:00
yangdx
54c48dce38 Fix z-index layering for GraphViewer UI panels 2025-10-29 02:09:13 +08:00
yangdx
b32b2e8b9e Refactor merge dialog and improve search history sync
- Extract MergeDialog to separate component
- Update search history on entity rename
- Add dropdown refresh trigger mechanism
- Sync query label with entity changes
- Force graph re-render after updates
2025-10-28 01:52:49 +08:00
yangdx
5155edd8d2 feat: Improve entity merge and edit UX
- **API:** The `graph/entity/edit` endpoint now returns a detailed `operation_summary` for better client-side handling of update, rename, and merge outcomes.
- **Web UI:** Added an "auto-merge on rename" option. The UI now gracefully handles merge success, partial failures (update OK, merge fail), and other errors with specific user feedback.
2025-10-27 23:42:08 +08:00
yangdx
25f829ef48 Enable editing of entity_type field in node properties 2025-10-27 12:27:57 +08:00
yangdx
9ed19695bb Remove separate retry button and merge functionality into scan button 2025-10-25 04:12:44 +08:00
yangdx
81e3496aa4 Add confirmation dialog for pipeline cancellation 2025-10-25 03:55:00 +08:00
yangdx
2476d6b7f8 Simplify pipeline status dialog by consolidating message sections
• Remove separate latest message section
• Combine into single pipeline messages area
• Add overflow-x-hidden for better display
• Change break-words to break-all
• Update translations across all locales
2025-10-25 03:34:54 +08:00
yangdx
f89b5ab101 Add pipeline cancellation feature with UI and i18n support
- Add cancelPipeline API endpoint
- Add cancel button to status dialog
- Update status response type
- Add cancellation UI translations
- Handle cancellation request states
2025-10-24 15:30:27 +08:00
yangdx
8dc23eeff2 Fix RayAnything compatible problem
• Use "preprocessed" to indicate multimodal processing is required
• Update DocProcessingStatus to process status convertion automatically
• Remove multimodal_processed from DocStatus enum value
• Update UI filter logic
2025-10-22 20:15:29 +08:00
yangdx
162370b6e6 Add optional LLM cache deletion when deleting documents
• Add delete_llm_cache parameter to API
• Collect cache IDs from text chunks
• Delete cache after graph operations
• Update UI with new checkbox option
• Add i18n translations for cache option
2025-10-22 12:19:23 +08:00
yangdx
80668aae22 Improve file path truncation labels and UI consistency
• Standardize FIFO/KEEP truncation labels
• Update UI truncation text format
2025-10-21 15:39:31 +08:00
yangdx
019dff5248 Update truncation message format in properties tooltip 2025-10-21 04:46:07 +08:00
yangdx
637b850ec5 Add truncation indicator and update property labels in graph view
• Add truncate tooltip to source_id field
• Add visual truncation indicator (†)
• Bump API version to 0242
2025-10-20 23:03:01 +08:00
yangdx
dab1c35834 Optimize chat performance by reducing animations in inactive tabs
• Add isTabActive prop to ChatMessage
• Disable spinner in inactive tabs
• Reduce opacity for inactive content
• Hide loading indicator when inactive
• Pass tab state from RetrievalTesting
2025-10-17 21:17:01 +08:00
yangdx
4c3ab58473 Improve AsyncSelect layout and text overflow handling
- Add responsive width container
- Improve text truncation with tooltips
2025-10-17 19:10:36 +08:00
yangdx
130b4959dc Add PREPROCESSED (multimodal_processed) status for multimodal document processing
• Add DocStatus.PREPROCESSED enum value
• Update API routes and response models
• Add preprocessed filter in web UI
• Update localization files
• Handle preprocessed status in deletion
2025-10-14 14:02:05 +08:00
yangdx
50210e25e8 Add @tailwindcss/typography plugin and fix Tailwind config
• Add typography plugin dependency
• Fix plugin imports in config
• Add source paths to CSS
2025-10-14 03:14:38 +08:00
Daniel.y
5290b60e9e
Merge pull request #2196 from zl7261/main
i18n: fix mustache brackets
2025-10-12 03:11:02 +08:00
yangdx
f2fb1202a3 Move accordion keyframes from CSS to Tailwind config and add fallback 'auto' value 2025-10-12 01:55:20 +08:00
yangdx
44f51f883f Add fallback value for accordion content height CSS variable
• Improve CSS robustness
• Prevent undefined height values
2025-10-12 01:46:58 +08:00
yangdx
2d9334d35f Simplify Root component by removing async i18n initialization
• Remove useState and useEffect hooks
• Import i18n directly instead of async init
• Eliminate loading state check
2025-10-12 01:17:37 +08:00
杨广
1a4d6775b5 i18n: fix mustache brackets 2025-10-11 16:50:32 +08:00
yangdx
a31192dd5a Update i18n file for pipeline UI text across locales 2025-10-05 14:01:22 +08:00
yangdx
d550f1c58c Fix linting 2025-10-05 10:42:15 +08:00
yangdx
0aef6a16b8 Add theme-aware edge highlighting colors for graph control 2025-10-05 10:40:25 +08:00
yangdx
0c1cb7b731 Improve document tooltip display with track ID and better formatting
• Add track ID to tooltip display
• Remove JSON braces from metadata
• Reorder tooltip content layout
• Clean up metadata indentation
• Show track ID before metadata
2025-10-05 10:13:11 +08:00
yangdx
7b1f8e0f6f Update scan tooltip to clarify it also reprocesses failed documents 2025-10-05 09:23:56 +08:00
yangdx
bf6ca9dd97 Add retry failed button translations and standardize button text
- Add missing AR/FR/TW translations
- Shorten EN/ZH button text to "Retry"
2025-10-05 09:20:33 +08:00
Jon
cf2a024e37 feat: Add endpoint and UI to retry failed documents
Add a new `/documents/reprocess_failed` API endpoint and corresponding
UI button to retry processing of failed and pending documents. This
addresses a common recovery scenario when document processing fails due
to server crashes, network errors, or LLM service outages.

Backend changes:
- Add ReprocessResponse model with status, message, and track_id fields
- Add POST /documents/reprocess_failed endpoint that triggers background
  reprocessing of FAILED, PENDING, and interrupted PROCESSING documents
- Reuses existing apipeline_process_enqueue_documents for consistency
- Includes comprehensive docstring and logging for observability

Frontend changes:
- Add TypeScript types and API function for the new endpoint
- Add retry handler with intelligent polling (fast refresh → normal)
- Add "Retry Failed" button in Documents page toolbar
- Button disabled when pipeline is busy to prevent duplicate operations
- Complete i18n support (English and Chinese translations)

This feature provides a convenient way to recover from processing
failures without requiring a full filesystem rescan.
2025-10-04 16:46:29 -04:00
yangdx
181525ffc2 Merge branch 'main' into zl7261/main 2025-10-02 22:01:16 +08:00
yangdx
19a4158419 Fix linting 2025-10-02 22:01:00 +08:00
yangdx
b1a4e7d7f0 Fix linting 2025-10-02 21:21:33 +08:00
yangdx
d4abe70411 Hide dev options in production builds 2025-10-02 21:21:16 +08:00
Roman Marchuk
1bd84f0005
Merge branch 'main' into fix/dark-mode-graph-text-colors 2025-10-01 17:41:28 -04:00
Roman Marchuk
7297ca1d5c Fix dark mode graph labels for system theme and improve colors
- Fix dark mode detection to work when theme is set to 'system'
- Add real-time system theme change detection
- Update label colors from cyan to white for better readability
- Update edge colors to medium gray (#888888) for better contrast
- Add development feature: random graph generator button in settings
- Enhance random graph with edge attributes and labels
- Install missing graphology layout dependencies
2025-10-01 17:36:50 -04:00
zl7261
ffcd75a408
decalre targetNode after check sourceNode 2025-09-29 15:24:35 +08:00
zl7261
6a8de2edb2
web_ui: check node source and target 2025-09-29 15:22:24 +08:00
yangdx
86195c613e Fix linting 2025-09-29 13:10:25 +08:00