Commit Graph

42 Commits

Author SHA1 Message Date
9abf7b25e6 Update corpus_enricher to auto-detect language if summary_lang is 'auto' 2026-03-19 22:15:02 +01:00
8cd785d1ac Update build_library, enrich_library, and embed_library functions in local_rag.py 2026-03-19 22:14:49 +01:00
a95b4c8226 Add file enrichment update endpoint and modify register/remove file methods 2026-03-19 22:14:43 +01:00
0fdebb9100 Rename source_signature to stage_signature in local_rag.py 2026-03-19 22:14:43 +01:00
b079d53841 Update local_rag.py to handle enrichment based on file count and improve progress tracking 2026-03-19 22:14:28 +01:00
10ff10dce6 Add functions for enrichment and shadow corpus building 2026-03-19 22:14:18 +01:00
7012998923 Fix signature check in prepare job handling 2026-03-19 22:14:18 +01:00
d68b3d284e Add enrichment signature and cleanup functions in local_rag.py 2026-03-19 22:13:51 +01:00
b7ab016f8e Add constants and update models in local_rag.py 2026-03-19 22:13:41 +01:00
cfcd0f070e Remove backend/libraries/dsa/library.json 2026-03-19 21:57:40 +01:00
e2abff960f Remove unused files and update library.json 2026-03-19 21:57:29 +01:00
c3ae71ad76 Add new library book and update library.json 2026-03-19 21:52:45 +01:00
6a8bf6f762 Add new corpus files and update library metadata 2026-03-19 21:52:35 +01:00
a3a229d958 Add corpus.jsonl and update library.json with new file details 2026-03-19 21:49:28 +01:00
d4858ad9f1 Add dsa library and remove kjhj library 2026-03-19 21:49:20 +01:00
469f76fda5 Remove corpus.jsonl and associated files from library.json 2026-03-19 21:49:16 +01:00
ade78d6456 auto-git:
[add] backend/libraries/kjhj/corpus.jsonl
 [add] backend/libraries/kjhj/stage/ff8bc28fa4444fd4bf2b3c2f99698a9c5cfc055b215d57190422e044f5e60b00--66_Seduction_Codes.pdf
 [add] dist/assets/index-T4npynoj.js
 [change] backend/libraries/kjhj/library.json
 [change] dist/index.html
 [unlink] backend/libraries/kjhj/stage/87b5e039fa580a8bbb84125fda96570462315b6b65090b43c7392edf53248c0d--assoziierungsabkommen-eu-ukraine--abl-l-161-vom-140529-neu.pdf
 [unlink] dist/assets/index-Bl2Chmww.js
2026-03-19 21:48:28 +01:00
d28445113c Add tqdm to backend requirements 2026-03-19 21:48:27 +01:00
4ccdfeb9f9 Handle optional tqdm import in index_builder.py 2026-03-19 21:48:27 +01:00
74100a5aaf Add PDF file and update library.json with sync error 2026-03-19 21:47:02 +01:00
33de143501 Add library.json for kjhj 2026-03-19 21:47:02 +01:00
a6be13fd8a Remove Punk library and associated files 2026-03-19 21:46:49 +01:00
cbc9ea42b3 Remove failed sync file and update library.json 2026-03-19 21:46:44 +01:00
fca0075730 auto-git:
[add] dist/assets/index-BLvuAcau.css
 [add] dist/assets/index-Bl2Chmww.js
 [change] backend/libraries/punk/library.json
 [change] dist/index.html
 [unlink] backend/libraries/punk/stage/19f1e5d2ceaab5fd1f1dc58ff07422388f156610d16dfdea2bdb35a5b9e70813--GeorgeJordac-TheVoiceOfHumanJustice.pdf
 [unlink] dist/assets/index-HsqF-Zwv.css
 [unlink] dist/assets/index-gF5dDH11.js
2026-03-19 21:46:36 +01:00
31f38e2aee Add sync status to file entries in register_paths 2026-03-19 21:44:49 +01:00
b50803bba5 Add functions to mark files as ready or failed in local_rag.py 2026-03-19 21:43:44 +01:00
ee4dd3a2ac Add functions for handling file sync status in local_rag.py 2026-03-19 21:43:37 +01:00
9826746705 Add Storyteller Tactics PDF to library and update pipeline signature 2026-03-19 21:41:13 +01:00
c415246a5f Add Humanity's Last Prompt Engineering Guide to library and update pipeline signature 2026-03-19 21:41:02 +01:00
3169f13540 Add new book to library and update pipeline signature 2026-03-19 21:40:56 +01:00
5be64f28f3 Add GeorgeJordac-TheVoiceOfHumanJustice.pdf to library and update pipeline 2026-03-19 21:40:35 +01:00
ced973d01b Remove unused files and update library.json 2026-03-19 21:40:22 +01:00
db34d2c377 Add cleanup and pending prepare logic in local_rag.py 2026-03-19 21:35:29 +01:00
c5d1510423 Add pipeline metadata and prepare functionality to local_rag.py 2026-03-19 21:23:22 +01:00
4c2ff3c875 Add corpus.jsonl to backend/libraries/punk 2026-03-19 21:16:09 +01:00
cf48fae376 Remove GeorgeJordac-TheVoiceOfHumanJustice.pdf and 4_Strings_-_Take_Me_Away_Into_The_Night_Vocal_Radio_Mix_.mp3 from library.json and stage directory 2026-03-19 21:12:20 +01:00
795f4d272a auto-git:
[add] README.md
 [add] backend/libraries/punk/library.json
 [add] backend/libraries/punk/stage/19f1e5d2ceaab5fd1f1dc58ff07422388f156610d16dfdea2bdb35a5b9e70813--GeorgeJordac-TheVoiceOfHumanJustice.pdf
 [add] backend/libraries/punk/stage/85fce554ff7685f7bccb136aff5768e54b9ba8361672fe45dbce599598c4be4b--4_Strings_-_Take_Me_Away_Into_The_Night_Vocal_Radio_Mix_.mp3
 [add] backend/libraries/punk/stage/e816ca61aebd84159747d248fedd6d5ff318c471c36bcc31b1ac6bf9aebcd3c1--The_Evolution_of_Cooperation_Robert_Axelrod_liber3.pdf
 [add] backend/local_rag.py
 [add] backend/rag/__init__.py
 [add] backend/rag/corpus_builder.py
 [add] backend/rag/corpus_enricher.py
 [add] backend/rag/index_builder.py
 [add] backend/rag/unified_rag.py
 [add] dist/assets/index-Cc0DLWqA.css
 [add] dist/assets/index-DKAz6gtp.js
 [add] dist/index.html
 [add] src/LibraryManager.jsx
 [add] wheelcheck2117/pydantic-2.11.7-py3-none-any.whl
 [add] wheelcheck274/pydantic-2.7.4-py3-none-any.whl
 [change] backend/main.py
 [change] backend/requirements.txt
 [change] backend/schemas.py
 [change] electron/main.cjs
 [change] electron/preload.cjs
 [change] package.json
 [change] run.sh
 [change] src/App.jsx
 [change] src/InterfaceSettings.jsx
 [change] src/colorSchemes.js
 [change] src/main.jsx
 [change] src/styles.css
2026-03-19 21:07:22 +01:00
728c7763e2 Add web search enrichment feature with source persistence and UI integration
Introduce optional web search enrichment flow for chat and regenerate requests. New /websearch endpoint calls enrich_prompt via SearXNG and returns enriched_prompt + citation sources.

DB: add sources_json column to chat_messages via ensure_sources_column migration helper.

Backend: persist sources_json for assistant replies (streaming and non-streaming); extend ChatRequest/RegenerateRequest to accept enriched_message and sources; history endpoint returns sources.

Frontend: add toggle for web search, settings for SearXNG URL + engines, and optional enrichment calls in sendMessage/regenerate. Render citation sources as rounded chips labeled with base domain under assistant replies.

Dependencies: add beautifulsoup4, httpx[http2], numpy for enrichment pipeline.
2025-08-27 04:27:18 +02:00
08d27f0007 feat: add message editing and regeneration endpoints
Add PUT /sessions/{session_id}/messages/{index} to edit a user message and prune all subsequent messages.

Add POST /sessions/{session_id}/regenerate to regenerate an assistant response from a given index (supports optional streaming).

Introduce EditMessageRequest and RegenerateRequest schemas.

Update the frontend to: • allow in‑place editing of user messages with an auto‑focusing textarea; • copy messages to clipboard; • regenerate assistant responses via a button.

Add new CSS rules for message options bars and the editing textarea.
2025-08-25 23:56:26 +02:00
d28d88d1f2 feat(ui,markdown): add collapsible 'Thoughts' for <think>, sanitize titles, improve focus UX; refactor table/blockquote render & styles
Backend: HTML-unescape and strip <think>/<thinking> from generated titles; trim and return cleaned value; add debug logs.

Electron: send 'window-focused' from main; expose onWindowFocus in preload.

Frontend: stream-safe AssistantMessageContent with collapsible Thoughts; switch streaming render to React state updates; autofocus textarea on window focus/new chat and when clicking empty chat area; sanitize session title client-side.

Markdown: support blockquotes; allow '-' or '*' bullets; simplify <think> removal to handle streaming; drop table wrapper div (emit <table class='nice'>); theme-aware code block headers/borders.

CSS: rounded 'nice' tables with light inner grid; blockquote styling; Thoughts toggle/panel styles.

Color: brighten Grayscale --accent.

Follow-ups: add IPC listener cleanup; ensure single source of truth for colorSchemes.
2025-08-25 21:13:09 +02:00
41c69abe28 feat: Add streaming chat + scroll persistence; improve markdown & links
Backend
- /chat: support streaming via StreamingResponse; save full reply after stream ends. Non-stream path unchanged.
- ChatRequest: add stream flag (default false).
- GenerateTitleRequest: add model and use it instead of hardcoded llama3.
- ollama_client.chat_stream(): new async generator parsing Ollama streaming JSON (both formats).
- Remove response_model from /chat to allow streaming; non-stream still returns { reply }.

Electron
- Open external links in system browser (setWindowOpenHandler, shell.openExternal).
- New IPC: update-settings, open-external-link.
- Set minimum window size; preload exposes updateSettings and openExternalLink.

Frontend (React)
- Streaming UI with live chunking; sticky-bottom only when user at bottom.
- Per-session scroll persistence and robust restore.
- New message tip to jump to latest reply when scrolled up.
- Disable Send while sending; spinner.
- General Settings: stream output toggle; propagate model/stream changes.
- Apply color scheme at boot; extract colorSchemes helper.
- Sidebar UX tweaks and unread badges.

Markdown/rendering
- Code blocks: language title bar and wrapper.
- Tables: GitHub-style parsing, per-cell borders, rounded wrapper, spacing, alignment.
- Headings: remove blank line after h1-h4.
- <hr>: handle after tables; strip following whitespace.
- Links: target=_blank with icon and URL tooltip.

Styles
- Add styles for code/table wrappers, new-message tip, toggle, spinner; hover/active vars; narrower sidebar.

API notes / breaking changes
- /chat accepts stream=true and returns text/plain streamed chunks.
- generate-title now requires a model.
- Non-stream /chat response shape unchanged.
2025-08-23 16:45:46 +02:00
191fe98ac0 initial commit 2025-08-22 23:42:34 +02:00