Commit Graph

44 Commits

Author SHA1 Message Date
d0e5c9300c Feature: Add full support for handling, indexing, and saving chat message content to the knowledge base. 2026-06-15 03:51:56 +02:00
c4956fa69e Introduce metadata versioning and upgrade logic for local library reading 2026-06-15 02:43:17 +02:00
bfb7f870c6 Feat: Add and integrate enrichment model support across backend and frontend 2026-06-15 02:42:18 +02:00
26ef19f4c6 Clear query locks during library purge and deletion 2026-06-15 02:18:13 +02:00
730c6be2cb Implement query locking for local RAG context retrieval 2026-06-15 02:16:07 +02:00
74958a5336 Improve metadata status tracking and failure handling in local RAG pipeline 2026-06-15 02:08:36 +02:00
d98bd71a5d Refactor RAG payload generation to use explicit metadata tracking and initialize metadata status in library functions 2026-06-15 02:04:03 +02:00
973e8ba219 Enhance RAG pipeline by implementing robust metadata persistence, extracting detailed information (summary, keywords, entities) into the enhanced corpus record. 2026-06-15 02:03:48 +02:00
4a624d2d30 Feat: Implement detailed content previews for library items
Adds functionality to display rich content previews (including truncated or extracted RAG text) when viewing a library item, updating backend indexing and frontend rendering accordingly.
2026-06-15 01:25:34 +02:00
9c84076a85 Update prepare job condition to check for indexed status in payload 2026-06-15 01:24:04 +02:00
b05562a2de Simplify job state checks for marking files ready and triggering preparation steps 2026-06-15 01:22:53 +02:00
b45117c52d Improve source URI handling in local RAG context by deduplicating sources 2026-06-15 01:22:53 +02:00
fd84748450 Improve RAG index building and local rag processing
- Implement robust chunking for oversized text in index_builder.
- Prevent job preparation in local_rag if source data contains failed sync items.
2026-06-15 01:21:19 +02:00
cbe30c6ccb Refactor list_libraries to async and ensure background preparation jobs are running before listing libraries 2026-06-15 01:19:17 +02:00
a5142e6dd0 Refactor local_rag to conditionally include item metadata (item_id, kind, title, url) only for managed or text/website content types 2026-06-15 01:17:03 +02:00
ec08af4640 Improve change detection and enhance content extraction logic 2026-06-15 01:15:42 +02:00
3e35ea96ec Add httpx dependency to backend/local_rag.py 2026-06-15 01:14:09 +02:00
c7f904ed01 Implement API endpoints and utilities for managing text and website content within the local RAG system, including web snapshot fetching and metadata updates. 2026-06-15 01:13:00 +02:00
ef1ecd463b Refactor path management and configuration utilities 2026-05-06 06:13:56 +02:00
6be438519c added embedding model selection 2026-03-20 12:00:44 +01:00
fecd101fd0 Add library purge functionality in backend and frontend 2026-03-20 10:33:58 +01:00
09589bf37f Enhance error handling in corpus build process 2026-03-19 23:02:54 +01:00
8f969d8a44 auto-git:
[change] backend/local_rag.py
2026-03-19 22:44:10 +01:00
f86989e722 Update library embedding and context handling to use pipeline metadata 2026-03-19 22:35:57 +01:00
b2e487262c Enhance prepare pipeline with embed model options 2026-03-19 22:34:46 +01:00
c462585543 Add embed_model to prepare pipeline response 2026-03-19 22:34:46 +01:00
49f6b14c24 Update embed model handling in local_rag.py 2026-03-19 22:34:46 +01:00
aff44c76b8 Add function to set pipeline embed model in local_rag.py 2026-03-19 22:34:25 +01:00
e8b9a437d5 Update default embedding model and modify request models in local_rag.py and unified_rag.py 2026-03-19 22:34:22 +01:00
9a92344387 Disable language detection in corpus building process 2026-03-19 22:30:06 +01:00
8cd785d1ac Update build_library, enrich_library, and embed_library functions in local_rag.py 2026-03-19 22:14:49 +01:00
a95b4c8226 Add file enrichment update endpoint and modify register/remove file methods 2026-03-19 22:14:43 +01:00
0fdebb9100 Rename source_signature to stage_signature in local_rag.py 2026-03-19 22:14:43 +01:00
b079d53841 Update local_rag.py to handle enrichment based on file count and improve progress tracking 2026-03-19 22:14:28 +01:00
10ff10dce6 Add functions for enrichment and shadow corpus building 2026-03-19 22:14:18 +01:00
7012998923 Fix signature check in prepare job handling 2026-03-19 22:14:18 +01:00
d68b3d284e Add enrichment signature and cleanup functions in local_rag.py 2026-03-19 22:13:51 +01:00
b7ab016f8e Add constants and update models in local_rag.py 2026-03-19 22:13:41 +01:00
31f38e2aee Add sync status to file entries in register_paths 2026-03-19 21:44:49 +01:00
b50803bba5 Add functions to mark files as ready or failed in local_rag.py 2026-03-19 21:43:44 +01:00
ee4dd3a2ac Add functions for handling file sync status in local_rag.py 2026-03-19 21:43:37 +01:00
db34d2c377 Add cleanup and pending prepare logic in local_rag.py 2026-03-19 21:35:29 +01:00
c5d1510423 Add pipeline metadata and prepare functionality to local_rag.py 2026-03-19 21:23:22 +01:00
795f4d272a auto-git:
[add] README.md
 [add] backend/libraries/punk/library.json
 [add] backend/libraries/punk/stage/19f1e5d2ceaab5fd1f1dc58ff07422388f156610d16dfdea2bdb35a5b9e70813--GeorgeJordac-TheVoiceOfHumanJustice.pdf
 [add] backend/libraries/punk/stage/85fce554ff7685f7bccb136aff5768e54b9ba8361672fe45dbce599598c4be4b--4_Strings_-_Take_Me_Away_Into_The_Night_Vocal_Radio_Mix_.mp3
 [add] backend/libraries/punk/stage/e816ca61aebd84159747d248fedd6d5ff318c471c36bcc31b1ac6bf9aebcd3c1--The_Evolution_of_Cooperation_Robert_Axelrod_liber3.pdf
 [add] backend/local_rag.py
 [add] backend/rag/__init__.py
 [add] backend/rag/corpus_builder.py
 [add] backend/rag/corpus_enricher.py
 [add] backend/rag/index_builder.py
 [add] backend/rag/unified_rag.py
 [add] dist/assets/index-Cc0DLWqA.css
 [add] dist/assets/index-DKAz6gtp.js
 [add] dist/index.html
 [add] src/LibraryManager.jsx
 [add] wheelcheck2117/pydantic-2.11.7-py3-none-any.whl
 [add] wheelcheck274/pydantic-2.7.4-py3-none-any.whl
 [change] backend/main.py
 [change] backend/requirements.txt
 [change] backend/schemas.py
 [change] electron/main.cjs
 [change] electron/preload.cjs
 [change] package.json
 [change] run.sh
 [change] src/App.jsx
 [change] src/InterfaceSettings.jsx
 [change] src/colorSchemes.js
 [change] src/main.jsx
 [change] src/styles.css
2026-03-19 21:07:22 +01:00