Commit Graph

191 Commits

Author SHA1 Message Date
14986260f0 Feature: Add full video support to library management
Integrates video processing into the local RAG system, allowing users to ingest videos via URL.

This includes:
- Updating content type handling for 'video' across item signatures and content extraction.
- Implementing video ingestion logic within the website creation endpoint.
- Adding a dedicated endpoint to refresh and update metadata for existing video items.
2026-06-15 06:13:46 +02:00
bb2e0c08e2 Feat(video): Implement full video ingestion pipeline using yt-dlp, Whisper transcription, and LLM summarization; updates Ollama client to support options. 2026-06-15 06:13:06 +02:00
88c4a5b0d4 Feat: Add automatic deep enrichment feature
Implements logic and UI for automatically queuing deep enrichment (Q&A generation) after content indexing, updating settings across backend, frontend, and desktop layers.
2026-06-15 04:41:31 +02:00
36b6b84129 Fix KMP duplicate lib error on macOS for dependencies (PyTorch/FAISS) 2026-06-15 04:07:37 +02:00
d0e5c9300c Feature: Add full support for handling, indexing, and saving chat message content to the knowledge base. 2026-06-15 03:51:56 +02:00
c4956fa69e Introduce metadata versioning and upgrade logic for local library reading 2026-06-15 02:43:17 +02:00
bfb7f870c6 Feat: Add and integrate enrichment model support across backend and frontend 2026-06-15 02:42:18 +02:00
26ef19f4c6 Clear query locks during library purge and deletion 2026-06-15 02:18:13 +02:00
730c6be2cb Implement query locking for local RAG context retrieval 2026-06-15 02:16:07 +02:00
7b42e25135 Enhance RAG corpus enrichment: Update prompt version, improve response validation, increase token limits, and refactor data normalization logic. 2026-06-15 02:11:20 +02:00
74958a5336 Improve metadata status tracking and failure handling in local RAG pipeline 2026-06-15 02:08:36 +02:00
32112ddbb6 Only mark text as short if QA enrichment is disabled 2026-06-15 02:08:36 +02:00
d98bd71a5d Refactor RAG payload generation to use explicit metadata tracking and initialize metadata status in library functions 2026-06-15 02:04:03 +02:00
973e8ba219 Enhance RAG pipeline by implementing robust metadata persistence, extracting detailed information (summary, keywords, entities) into the enhanced corpus record. 2026-06-15 02:03:48 +02:00
1758a27a31 Improve RAG prompt instructions for Q&A and refine processing levels in cache 2026-06-15 02:03:07 +02:00
1c7e5f0691 Feat(RAG): Make QA generation optional and support 'deep' source paths for enrichment 2026-06-15 02:02:53 +02:00
4a624d2d30 Feat: Implement detailed content previews for library items
Adds functionality to display rich content previews (including truncated or extracted RAG text) when viewing a library item, updating backend indexing and frontend rendering accordingly.
2026-06-15 01:25:34 +02:00
9c84076a85 Update prepare job condition to check for indexed status in payload 2026-06-15 01:24:04 +02:00
b05562a2de Simplify job state checks for marking files ready and triggering preparation steps 2026-06-15 01:22:53 +02:00
b45117c52d Improve source URI handling in local RAG context by deduplicating sources 2026-06-15 01:22:53 +02:00
fd84748450 Improve RAG index building and local rag processing
- Implement robust chunking for oversized text in index_builder.
- Prevent job preparation in local_rag if source data contains failed sync items.
2026-06-15 01:21:19 +02:00
cbe30c6ccb Refactor list_libraries to async and ensure background preparation jobs are running before listing libraries 2026-06-15 01:19:17 +02:00
a5142e6dd0 Refactor local_rag to conditionally include item metadata (item_id, kind, title, url) only for managed or text/website content types 2026-06-15 01:17:03 +02:00
ec08af4640 Improve change detection and enhance content extraction logic 2026-06-15 01:15:42 +02:00
3e35ea96ec Add httpx dependency to backend/local_rag.py 2026-06-15 01:14:09 +02:00
c7f904ed01 Implement API endpoints and utilities for managing text and website content within the local RAG system, including web snapshot fetching and metadata updates. 2026-06-15 01:13:00 +02:00
caaf7e6c79 Integrate backend sidecar process and external binaries into the Tauri build 2026-05-06 06:46:27 +02:00
ef1ecd463b Refactor path management and configuration utilities 2026-05-06 06:13:56 +02:00
4f6771e787 Update backend and frontend assets for chat session handling 2026-04-17 14:52:51 +02:00
0ea3bfc14b Add function to normalize chat attachments and raise exceptions for invalid types 2026-04-17 13:06:35 +02:00
d5a4ab8417 Add functions to build Ollama messages and update chat handling logic 2026-04-17 12:59:31 +02:00
4b3d510bcf Refactor attachment handling and add support for file attachments 2026-04-17 12:54:55 +02:00
67195af4c1 Rename ImageAttachment to ChatAttachment and add new fields 2026-04-17 12:51:50 +02:00
50827fd8cf Refactor chat title sanitization logic in backend and frontend 2026-04-17 10:33:44 +02:00
e332515585 Update model catalog fetching and loading state handling 2026-04-17 08:48:27 +02:00
e9b96812f2 Refactor model details caching and vision support checks in ollama_client.py 2026-04-17 08:46:43 +02:00
9607f2192a Update rerank function to use new model preference 2026-04-17 08:35:42 +02:00
84f0bc7a4d Update ollama_admin.py to include new model preferences and checks 2026-04-17 08:34:56 +02:00
2e3855cfa3 Update model listing to include Whisper models 2026-04-17 08:34:29 +02:00
3dbe3fa790 Add model catalog functionality to backend/ollama_client.py 2026-04-17 08:34:22 +02:00
9ee177d47c Add function to list available Whisper models 2026-04-17 08:33:53 +02:00
22d45c4537 auto-git:
[add] dist/assets/index-g7N0HjkK.js
 [change] backend/app_settings.py
 [change] dist/index.html
 [unlink] dist/assets/index-BOZc6wlT.js
2026-04-17 08:33:39 +02:00
48c35698cb auto-git:
[add] dist/assets/index-DifKT69O.js
 [change] backend/main.py
 [change] backend/schemas.py
 [change] backend/whisper_admin.py
 [change] dist/index.html
 [change] electron/main.cjs
 [change] src/App.jsx
 [change] src/GeneralSettings.jsx
 [change] src/audioInput.js
 [unlink] dist/assets/index-Dm7DZNSo.js
2026-04-17 04:43:29 +02:00
a0c2b28591 Add audio transcription route to backend/main.py 2026-04-16 22:07:50 +02:00
7225980fb9 Import asyncio in backend/main.py 2026-04-16 22:07:50 +02:00
4447749439 Add audio transcription request and response schemas 2026-04-16 22:07:08 +02:00
86936dcb96 Extend whisper_admin.py with audio processing and transcription features 2026-04-16 22:07:05 +02:00
e88ac88840 Add support for image attachments in chat messages and update model capabilities endpoint 2026-04-16 21:28:16 +02:00
7bbdd1ac6d Refactor title sanitization logic in backend/main.py 2026-03-20 16:26:09 +01:00
55974b90fd Add startup model preparation endpoint and update Whisper handling 2026-03-20 15:43:10 +01:00