Commit graph

101 commits

Author SHA1 Message Date
clssck
663ada943a chore: add citation system and enhance RAG UI components
Add citation tracking and display system across backend and frontend components.
Backend changes include citation.py for document attribution, enhanced query routes
with citation metadata, improved prompt templates, and PostgreSQL schema updates.
Frontend includes CitationMarker component, HoverCard UI, QuerySettings refinements,
and ChatMessage enhancements for displaying document sources. Update dependencies
and docker-compose test configuration for improved development workflow.
2025-12-01 17:50:00 +01:00
clssck
77df910525 chore: add citation system and code formatting setup
Add citation.py module for document citation tracking and management.
Configure Biome and Ruff for consistent code formatting across TypeScript
and Python. Update webui with improved component organization, API client
refactoring, and enhanced user interface patterns. Add formatting configs
and dependency updates for build toolchain optimization.
2025-11-30 20:51:43 +01:00
clssck
43af31f888 feat: add db_degree visibility and orphan connection UI
Graph Connectivity Awareness:
- Add db_degree property to all KG implementations (NetworkX, Postgres, Neo4j, Mongo, Memgraph)
- Show database degree vs visual degree in node panel with amber badge
- Add visual indicator (amber border) for nodes with hidden connections
- Add "Load X hidden connection(s)" button to expand hidden neighbors
- Add configurable "Expand Depth" setting (1-5) in graph settings
- Use global maxNodes setting for node expansion consistency

Orphan Connection UI:
- Add OrphanConnectionDialog component for manual orphan entity connection
- Add OrphanConnectionControl button in graph sidebar
- Expose /graph/orphans/connect API endpoint for frontend use

Backend Improvements:
- Add get_orphan_entities() and connect_orphan_entities() to base storage
- Add orphan connection configuration parameters
- Improve entity extraction with relationship density requirements

Frontend:
- Add graphExpandDepth and graphIncludeOrphans to settings store
- Add min_degree and include_orphans graph filtering parameters
- Update translations (en.json, zh.json)
2025-11-29 21:08:07 +01:00
clssck
b6074b9a81 chore(lightrag, lightrag_webui): improve code quality and security
- Extract PostgreSQL storage check into named variable for clarity
- Move APIRouter initialization into create_table_routes function scope
- Add robust type handling for database query results
- Add input validation for table names and pagination parameters
- Add regex-based SQL injection prevention for table name sanitization
- Improve clipboard copy fallback logic and error handling
- Add memoization for JSON serialization to prevent unnecessary recalculations
- Hide meta column from table explorer UI display
- Sort table columns alphabetically for consistent ordering
- Add keyboard accessibility to status filter buttons
- Add preprocessed status filter to document manager
- Update @tanstack/react-query from 5.60.0 to 5.87.1
- Extract dev storage config into constant to reduce duplication
- Update documentation comments for clarity
2025-11-27 21:39:42 +01:00
clssck
a9edadef45 feat: add Table Explorer feature with dynamic table data fetching and schema display
- Implemented Table Explorer component to allow users to select and view database tables.
- Added API calls for fetching table list, schema, and paginated data.
- Introduced row detail modal for displaying and copying row data.
- Enhanced DataTable component to support row click events.
- Updated UI components for better user experience and accessibility.
- Added mock data for development mode to facilitate testing.
- Updated localization files to include new terms related to tables.
- Modified settings store to include storage configuration for conditional UI rendering.
- Improved styling and layout for various components to align with new design standards.
2025-11-27 18:27:14 +01:00
clssck
48c7732edc feat: add automatic entity resolution with 3-layer matching
Implement automatic entity resolution to prevent duplicate nodes in the
knowledge graph. The system uses a 3-layer approach:

1. Case-insensitive exact matching (free, instant)
2. Fuzzy string matching >85% threshold (free, instant)
3. Vector similarity + LLM verification (for acronyms/synonyms)

Key features:
- Pre-resolution phase prevents race conditions in parallel processing
- Numeric suffix detection blocks false matches (IL-4 ≠ IL-13)
- PostgreSQL alias cache for fast lookups on subsequent ingestion
- Configurable thresholds via environment variables

Bug fixes included:
- Fix fuzzy matching false positives for numbered entities
- Fix alias cache not being populated (missing db parameter)
- Skip entity_aliases table from generic id index creation

New files:
- lightrag/entity_resolution/ - Core resolution module
- tests/test_entity_resolution/ - Unit tests
- docker/postgres-age-vector/ - Custom PG image with pgvector + AGE
- docker-compose.test.yml - Integration test environment

Configuration (env.example):
- ENTITY_RESOLUTION_ENABLED=true
- ENTITY_RESOLUTION_FUZZY_THRESHOLD=0.85
- ENTITY_RESOLUTION_VECTOR_THRESHOLD=0.5
- ENTITY_RESOLUTION_MAX_CANDIDATES=3
2025-11-27 15:35:02 +01:00
yangdx
8c07c91833 Remove deprecated response_type parameter from query settings
- Bump API version to 0254
- Remove response format UI controls
- Hard-code response_type in query params
- Add migration for version 19
- Clean up settings store structure
2025-11-12 12:19:30 +08:00
yangdx
80f2e691fc Remove redundant i18n import triggered the Vite “dynamic + static import” warning 2025-11-09 02:48:11 +08:00
yangdx
678e17bb5c Revert "fix(ui): Remove dynamic import for i18n in settings store"
This reverts commit 0216325e0f.
2025-11-06 21:00:47 +08:00
yangdx
0216325e0f fix(ui): Remove dynamic import for i18n in settings store 2025-11-06 16:15:32 +08:00
yangdx
4cbd876126 feat: Update node color and legent after entity_type changed
- Move color constants to utils module
- Extract resolveNodeColor function
- Update node colors on type changes
- Simplify hook color logic
2025-10-31 00:03:55 +08:00
yangdx
79a17c3f7f Fix graph value handling for entity_id updates
• Use finalValue for entity_id changes
• Keep original value for other props
• Fix property update logic
2025-10-30 23:43:46 +08:00
yangdx
b32b2e8b9e Refactor merge dialog and improve search history sync
- Extract MergeDialog to separate component
- Update search history on entity rename
- Add dropdown refresh trigger mechanism
- Sync query label with entity changes
- Force graph re-render after updates
2025-10-28 01:52:49 +08:00
yangdx
d5fe6bb33e Increase user prompt history capacity and dropdown height
• History limit: 8 → 12 items
• Dropdown height: max-h-60 → max-h-96
• Better visibility for longer prompts
• Improved user experience
2025-09-26 10:27:03 +08:00
yangdx
0330882b12 feat: add delete functionality to user prompt history dropdown
• Add delete buttons to history items
• Decrease history limit to 8 items
• Improve dropdown styling
2025-09-25 18:55:44 +08:00
yangdx
71367c7bc2 Add user prompt history dropdown to query settings
- Create UserPromptInputWithHistory component
- Move user prompt field to top of panel
- Add history tracking to settings store
- Include keyboard navigation support
- Auto-save prompts on query execution
2025-09-25 06:16:32 +08:00
yangdx
7a66c009ed Refactor node type legend color mapping
- Reduce chaos legend by storing standard node type
- Reset lengend on graph manual refresh
- Optimize color mapping for new node type schema
2025-09-23 19:48:16 +08:00
yangdx
9db8f2fce5 feat: Add popular labels and search APIs with history management
- Add popular/search label endpoints
- Implement SearchHistoryManager utility
- Replace client-side with server search
- Add graph data version tracking
- Update UI for better label discovery
2025-09-20 02:03:47 +08:00
yangdx
d5e8f1e860 Update default query parameters for better performance
- Increase chunk_top_k from 10 to 20
- Reduce max_entity_tokens to 6000
- Reduce max_relation_tokens to 8000
- Update web UI default values
- Fix max_total_tokens to 30000
2025-08-18 19:32:11 +08:00
yangdx
755a08edec Remove History Turns config option and force value to 0
- Remove History Turns UI component from QuerySettings
- Update settings store version to 17 with migration
- Force history_turns parameter to always be 0 in queries
- Prevent future modifications to history_turns setting
2025-07-31 11:22:48 +08:00
yangdx
45f27fccc3 feat(webui): Implement intelligent polling and responsive health checks
- Relocate the health check functionality from aap.tsx to state.ts to enable initialization by other components.
- Replaced the fixed 5-second polling with a dynamic interval. The polling interval is extended to 30 seconds when the no files in pending an processing state.
- Data refresh is triggered instantly when  `pipelineBusy` state changed
- Health check is triggered  after clicking "Scan New Documents" or "Clear Documents"
2025-07-31 01:37:24 +08:00
yangdx
e60c26ea77 feat: set default page size to 10 and persist in localStorage
- Change default documents page size from 20 to 10 rows
- Add documentsPageSize setting to settings store with persistence
- Update DocumentManager to use and save page size preference
- Include migration for existing users to get new default value
2025-07-30 18:07:38 +08:00
yangdx
33d986aa2a Change default history_turns from 3 to 0 in settings store of webui 2025-07-24 16:47:54 +08:00
yangdx
8103b200db Set DEFAULT_HISTORY_TURNS to 0 2025-07-16 02:20:27 +08:00
yangdx
7735970a3b Keep old query settings untouched 2025-07-15 17:24:13 +08:00
zrguo
7c882313bb remove chunk_rerank_top_k 2025-07-15 11:52:34 +08:00
zrguo
2d7f87e936 Update settings.ts 2025-07-14 16:39:42 +08:00
zrguo
ef2115d437 Update token limit 2025-07-14 15:53:48 +08:00
zrguo
e1541caea9 Update webui setting 2025-07-09 12:10:06 +08:00
yangdx
9bf048a4b1 Sync frontend with backend MAX_GRAPH_NODES
- Auto-adjust graph nodes when limit changes
- Add refresh trigger for node count changes
- Store backend limit in settings
2025-07-08 00:21:42 +08:00
yangdx
d914de441e Add user_prompt to WebUI 2025-05-08 04:15:21 +08:00
yangdx
4e2e668def Bump webui setting version to 12 2025-04-29 02:36:17 +08:00
yangdx
cdd69845c9 Remove grapOperation.ts 2025-04-15 13:04:51 +08:00
yangdx
28d26c3a4a refactor: improve graph property update mechanism
- Move graph data and UI state update logic into store
- Ensure all property updates trigger PropertiesView refresh
- Add graphDataVersion dependency to PropertiesView
- Follow React best practices by avoiding direct UI state modification in utility functions
- Fix issue where non-entity_id property changes weren't reflected in UI
2025-04-15 12:41:51 +08:00
yangdx
f58fb43f67 Fix Neo4j node and edge edit problem 2025-04-15 12:41:41 +08:00
yangdx
b18bf10353 Merge branch 'edit-node' into add-graph-db-lock 2025-04-14 03:39:57 +08:00
iLuJack
a72809c59a docs(locales): add traditional chinese support 2025-04-12 16:46:46 +08:00
choizhang
7e3e685763 feat(graph): Add editing function for entity and relationship attributes 2025-04-12 00:48:19 +08:00
yangdx
9d6a3c281b Merge branch 'feat-node-color' into merge-node-color 2025-04-05 14:39:49 +08:00
yangdx
ae1e991724 Merge branch 'feat-edge-thickness' into merge-edge-thickness 2025-04-05 13:06:28 +08:00
yangdx
dcba3f776d Fix linting 2025-04-05 02:41:06 +08:00
yangdx
299c508f27 Fix webtitle display problem 2025-04-05 02:24:23 +08:00
yangdx
1c0b94c46f Fix web title cleaning problem 2025-04-05 00:41:49 +08:00
yangdx
4851f4e01e Fix linting 2025-04-04 22:52:50 +08:00
yangdx
29d01e1c74 Add title support for webui 2025-04-04 22:51:07 +08:00
choizhang
81355481c1 feat: Add legend components and toggle buttons 2025-04-03 22:42:13 +08:00
yangdx
1a1f934e27 Feat: replace min_degree with max_nodes in webui 2025-04-02 18:35:32 +08:00
choizhang
cd3817ce30 feat (graph settings): Add edge thickness range configuration function 2025-04-01 00:36:32 +08:00
yangdx
814f3b3308 feat: add pipeline busy status indicator with breathing effect
- Add pipeline_busy field to health check response
- Track pipeline busy state in frontend store
- Add breathing animation for pipeline status button
- Enhance dark mode visibility with stronger contrast
2025-03-26 13:11:53 +08:00
yangdx
d456269718 feat: Add file name display in WebUI
Backend:
- Add file_path field to DocStatusResponse
- Update document status response creation

Frontend:
- Remove metadata column
- Improve filename display with truncation and tooltips
- Add show/hide filename toggle with proper styling
- Update translations for all supported languages"
2025-03-25 22:42:46 +08:00