auto-git:
[change] README.md [change] concept_api.py [change] requirements.txt [change] websearch.py [unlink] concept-maker_gui.py [unlink] run.sh
This commit is contained in:
154
README.md
154
README.md
@@ -1,94 +1,96 @@
|
||||
# Concept Maker GUI
|
||||
# Concept Maker
|
||||
|
||||
A desktop tool to turn raw ideas, notes, and files into a clear, actionable project concept. Drag in files, jot notes, use automated websearch to find prior art, build a compact knowledge base, generate an editable Markdown concept using a local LLM, convert it to pdf and push your concept to a Git repo.
|
||||
A Tauri desktop app that turns raw ideas, notes, URLs, and files into a clear project concept. The UI is React/Vite, the desktop shell is Tauri, and the Python backend is exposed through `concept_api.py`.
|
||||
|
||||
## Features
|
||||
- Drag and drop files/folders into a table (falls back to add buttons).
|
||||
- Freeform notes area for thoughts and fragments.
|
||||
- Builds a JSONL knowledge base from your files (uses `corpus_builder.py` when present; includes a simple fallback).
|
||||
- Generates a polished Concept (Markdown) via a local Ollama model.
|
||||
- Prior-art search via SearXNG: generates smart queries, fetches pages, embeds + reranks, and shows a results window.
|
||||
- Edit the result in‑app, then save it into a local Git repo and optionally push to a remote.
|
||||
- Remembers sessions; re‑open and continue later.
|
||||
- Uses `icon.png` for the window/taskbar/dock icon.
|
||||
|
||||
- Add files, folders, and URLs as source material.
|
||||
- Write freeform notes and ask a local Ollama model to rephrase, extend, or generate a concept.
|
||||
- Build a JSONL knowledge base from selected files using `corpus_builder.py` when available, with a lightweight fallback in the backend.
|
||||
- Search for prior art through SearXNG and Ollama embeddings.
|
||||
- Preview concepts as Markdown/PDF and save them into a local concepts repo.
|
||||
- Generate an image prompt and optional image asset for a concept.
|
||||
- Save and reopen local sessions.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Node.js and npm
|
||||
- Rust toolchain
|
||||
- Python 3.9+
|
||||
- Ollama running locally (default `http://localhost:11434`)
|
||||
- Suggested default model: `mistral3.2-small:24b` (configurable in the UI)
|
||||
- OS packages as needed for Tk:
|
||||
- macOS: included with the official Python.org installer
|
||||
- Ubuntu/Debian: `sudo apt-get install python3-tk`
|
||||
- Windows: included with the official Python.org installer
|
||||
- Ollama running locally, defaulting to `http://localhost:11434`
|
||||
- Optional local SearXNG instance for prior-art search, defaulting to `http://localhost:8888`
|
||||
|
||||
Optional Python packages improve file ingestion:
|
||||
- `tkinterdnd2` (native drag & drop)
|
||||
- `pymupdf` (PDF text extraction)
|
||||
- `beautifulsoup4` (better HTML text extraction)
|
||||
Python packages are listed in `requirements.txt`. Some features also depend on external command-line tools:
|
||||
|
||||
Prior‑art search requires API access to a SearXNG instance:
|
||||
- Recommended: run SearXNG locally via Docker (installation/compose details are out of scope for this repo).
|
||||
- Project site: https://searxng.org
|
||||
- `pandoc` or `wkhtmltopdf` for higher-quality Markdown to PDF export
|
||||
- `tesseract` and `ocrmypdf` for OCR-heavy ingestion
|
||||
- `ffmpeg`/`ffprobe` for media handling
|
||||
|
||||
Note: `requirements.txt` also lists some optional extras and system tools. If installation fails on entries that are not Python packages (e.g., system binaries), install them via your OS package manager or comment those lines out.
|
||||
## Setup
|
||||
|
||||
## Quick Start
|
||||
- With the convenience script:
|
||||
- `bash run.sh`
|
||||
- Manual steps:
|
||||
- `python3 -m venv .venv`
|
||||
- `source .venv/bin/activate` (Windows: `.\\.venv\\Scripts\\activate`)
|
||||
- `python -m pip install --upgrade pip`
|
||||
- `pip install -r requirements.txt`
|
||||
- `python concept-maker_gui.py`
|
||||
Install JavaScript dependencies:
|
||||
|
||||
Ensure Ollama is running before generating concepts:
|
||||
- Install: https://ollama.com
|
||||
- Pull a model: `ollama pull mistral3.2-small:24b`
|
||||
- Start service (if not auto-started): `ollama serve`
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
|
||||
## Usage
|
||||
- Add files/folders via drag & drop or the Add buttons.
|
||||
- Write or paste notes in the Notes panel.
|
||||
- Choose the Ollama model (or keep your default).
|
||||
- Click to generate the Concept; edit as needed.
|
||||
- Save: the app can save to a local repo and push to a remote.
|
||||
Install Python backend dependencies:
|
||||
|
||||
### Prior‑Art Search (SearXNG)
|
||||
- In the Notes section, click "Find Prior Art".
|
||||
- Configure the SearXNG endpoint in the bottom bar (default `http://localhost:8888`).
|
||||
- The app will:
|
||||
- Use the selected Ollama model to generate exactly three search queries (based on your Notes, Knowledge Base, and asset filenames).
|
||||
- Query SearXNG, fetch promising pages, embed them with Ollama embeddings (`bge-m3:latest` by default), and rerank by relevance.
|
||||
- Open a results window showing URLs, scores, and content snippets; double‑click a row to open in your browser.
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Configuration
|
||||
Environment variables (optional):
|
||||
- `OLLAMA_HOST`: override Ollama URL (default `http://localhost:11434`).
|
||||
- `IDEA_HOLE_MODEL`: default model name shown in the UI.
|
||||
- `IDEA_HOLE_REMOTE`: default Git remote URL.
|
||||
- `SEARX_URL`: default SearXNG base URL for prior‑art search (default `http://localhost:8888`).
|
||||
Ensure Ollama is running before using model-backed actions:
|
||||
|
||||
## Data & Storage
|
||||
- Working data lives under `./.idea-hole/`:
|
||||
- `files/`: symlinks/copies of added files
|
||||
- `corpus.jsonl`: unified knowledge base
|
||||
- `sessions.jsonl`: saved sessions metadata
|
||||
|
||||
## Platform Notes
|
||||
- macOS dock icon: if you install PyObjC (`pip install pyobjc`), the app also sets the dock icon from `icon.png`.
|
||||
- Drag & drop requires `tkinterdnd2`; otherwise, the app uses the add buttons.
|
||||
- PDF extraction works best with `pymupdf` installed.
|
||||
|
||||
## Troubleshooting
|
||||
- Tk import errors on Linux: install `python3-tk` via your package manager.
|
||||
- Ollama connection errors: ensure `ollama serve` is running and the model is pulled.
|
||||
- Prior‑art search returns nothing: ensure your SearXNG instance is reachable at the configured URL and supports `/search?format=json`.
|
||||
- Pip fails on non-Python entries in `requirements.txt`: install those tools via your OS, or comment out the offending lines and rerun the install.
|
||||
```bash
|
||||
ollama serve
|
||||
```
|
||||
|
||||
## Development
|
||||
- Code entry point: `concept-maker_gui.py`
|
||||
- Launcher script: `run.sh`
|
||||
- Optional corpus builder: `corpus_builder.py` (if present) is invoked for richer ingestion; otherwise, a simple built‑in fallback is used.
|
||||
|
||||
Prior‑art module: `websearch.py`
|
||||
Run the web UI only:
|
||||
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Run the desktop app:
|
||||
|
||||
```bash
|
||||
npm run tauri dev
|
||||
```
|
||||
|
||||
Build the frontend:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
Build the desktop bundle:
|
||||
|
||||
```bash
|
||||
npm run tauri build
|
||||
```
|
||||
|
||||
## Project Layout
|
||||
|
||||
- `src/`: React frontend
|
||||
- `src-tauri/`: Tauri desktop shell and Rust commands
|
||||
- `concept_api.py`: JSON action backend invoked by Tauri
|
||||
- `corpus_builder.py`: optional richer file ingestion pipeline
|
||||
- `websearch.py`: prior-art search helpers
|
||||
- `requirements.txt`: Python backend dependencies
|
||||
|
||||
## Configuration
|
||||
|
||||
Environment variables:
|
||||
|
||||
- `OLLAMA_HOST`: override the Ollama URL
|
||||
- `IDEA_HOLE_MODEL`: default model name
|
||||
- `IDEA_HOLE_REMOTE`: default Git remote URL
|
||||
- `SEARX_URL`: default SearXNG base URL
|
||||
|
||||
Runtime data is stored in `.idea-hole/`, and generated concepts are stored under `concepts/` unless another repo folder is selected in the app.
|
||||
|
||||
3323
concept-maker_gui.py
3323
concept-maker_gui.py
File diff suppressed because it is too large
Load Diff
@@ -4,7 +4,7 @@
|
||||
"""
|
||||
Headless backend actions for the Concept Maker app.
|
||||
|
||||
This module avoids Tk dependencies and exposes JSON actions for the Tauri UI.
|
||||
This module exposes JSON actions for the Tauri UI without desktop GUI imports.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Core libraries used by the GUI (concept_gui.py) and corpus builder
|
||||
# Core libraries used by the Tauri Python backend and corpus builder
|
||||
pymupdf
|
||||
beautifulsoup4
|
||||
requests
|
||||
@@ -8,7 +8,6 @@ numpy
|
||||
tqdm
|
||||
ebooklib
|
||||
markdown
|
||||
tkinterdnd2
|
||||
pdflatex
|
||||
|
||||
# Optional: language detection and image text-likeness improvements
|
||||
|
||||
32
run.sh
32
run.sh
@@ -1,32 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Resolve to repo root (directory of this script)
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# Pick a Python executable
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
PY=python3
|
||||
else
|
||||
PY=python
|
||||
fi
|
||||
|
||||
echo "[setup] Creating virtual environment in .venv"
|
||||
"$PY" -m venv .venv
|
||||
|
||||
# Activate venv
|
||||
echo "[setup] Activating virtual environment"
|
||||
source .venv/bin/activate
|
||||
|
||||
# Upgrade pip (optional but helpful)
|
||||
python -m pip install --upgrade pip >/dev/null 2>&1 || true
|
||||
|
||||
# Install requirements
|
||||
echo "[deps] Installing requirements from requirements.txt"
|
||||
python -m pip install -r requirements.txt
|
||||
|
||||
# Launch the app
|
||||
echo "[run] Starting Concept Maker GUI"
|
||||
exec python concept-maker_gui.py
|
||||
|
||||
@@ -2,13 +2,13 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
"""
|
||||
Prior-art web search utilities for the Idea → Concept GUI.
|
||||
Prior-art web search utilities for the Concept Maker backend.
|
||||
|
||||
Design goals
|
||||
- Pure stdlib networking (urllib) to avoid new dependencies.
|
||||
- Works with a local SearXNG instance.
|
||||
- Uses Ollama for both query generation and embeddings.
|
||||
- Small, robust, and callable from the GUI with status callbacks.
|
||||
- Small, robust, and callable from the Python backend.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
Reference in New Issue
Block a user