From 72bb486238cddc395b56243050ec2e5e4e67940f Mon Sep 17 00:00:00 2001 From: Victor Giers Date: Thu, 7 May 2026 17:46:11 +0200 Subject: [PATCH] Update README with comprehensive documentation for SkymapGen tool --- README.md | 171 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 170 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 541f305..2ab64d7 100644 --- a/README.md +++ b/README.md @@ -1 +1,170 @@ -# Projekt in skymap-gen +# SkymapGen + +SkymapGen is a desktop tool for generating and previewing equirectangular 360 sky maps from text prompts. It combines a Tauri/Vite/Three.js interface with a Python Diffusers generation pipeline, so generated panoramas can be viewed immediately inside an interactive sphere. + +## Features + +- Generate equirectangular PNG sky maps from text prompts. +- Preview maps in a full-screen Three.js 360 viewer with orbit controls. +- Keep a thumbnail dock of generated maps from the local `output/` folder. +- Persist generation settings in the app, including size, steps, guidance, scheduler, model paths, upscaling, and seam inpainting. +- Cancel an active generation from the UI. +- Store the enriched prompt in PNG metadata for later display. +- Write generation logs to `logs/`. + +## Requirements + +- macOS, Linux, or Windows with Tauri prerequisites installed. +- Node.js and npm. +- Rust and Cargo for the Tauri desktop shell. +- Python 3.10 recommended. `run.sh` prefers `python3.10` when available and falls back to `python3`. +- Enough RAM/VRAM for SDXL generation. Apple Silicon MPS and CUDA are detected automatically when available. + +Python dependencies are listed in `requirements.txt`. JavaScript dependencies are listed in `package.json`. + +## Model Files + +The app is offline-first by default. It will use local files or cached Hugging Face assets unless downloads are explicitly enabled. + +Recommended local files: + +- `models/sdxl360Diffusion_v10.safetensors` for the SDXL 360 checkpoint. +- `RealESRGAN_x4plus.pth` in the project root if you enable Real-ESRGAN upscaling. + +If the SDXL 360 checkpoint is missing, the generator can load `ProGamerGov/sdxl-360-diffusion` from the Hugging Face cache, or download it when downloads are enabled. + +## Quick Start + +```bash +./run.sh +``` + +The script will: + +1. Create `.venv/` if needed. +2. Install Python requirements. +3. Install npm dependencies if `node_modules/` is missing. +4. Start the Tauri dev app with `npm run tauri dev`. + +To allow missing Hugging Face assets to download during startup or generation: + +```bash +SKYMAP_ALLOW_DOWNLOADS=1 ./run.sh +``` + +## Manual Setup + +```bash +python3.10 -m venv .venv +source .venv/bin/activate +pip install --upgrade pip +pip install -r requirements.txt +npm install +``` + +Start the desktop app: + +```bash +npm run tauri dev +``` + +Build the web assets only: + +```bash +npm run build +``` + +Build a Tauri app bundle: + +```bash +npm run tauri build +``` + +## Desktop Usage + +1. Launch the app. +2. Enter a prompt in the bottom prompt bar. +3. Open `Settings` to adjust generation parameters. +4. Click `Generate`. +5. The generated panorama is saved to `output/` and loaded into the 360 viewer. + +During generation, the `Generate` button changes to `Cancel`. Existing PNG maps in `output/` are shown in the thumbnail dock, and can be selected or deleted from the app. + +Default desktop generation settings: + +- Size: `1536x768` +- Steps: `25` +- Guidance: `6.5` +- Scheduler: `dpmsolver-sde` +- Upscale: `none` +- Seam inpainting: disabled +- Model: local `models/sdxl360Diffusion_v10.safetensors` when present, otherwise `ProGamerGov/sdxl-360-diffusion` +- Base model: `stabilityai/stable-diffusion-xl-base-1.0` +- VAE: `madebyollin/sdxl-vae-fp16-fix` + +## CLI Usage + +You can run the Python generator directly: + +```bash +source .venv/bin/activate +python generate_equirect.py \ + --prompt "wide alpine meadow at sunset" \ + --output-dir output \ + --width 1536 \ + --height 768 \ + --steps 25 \ + --guidance 6.5 \ + --scheduler dpmsolver-sde \ + --upscale none +``` + +Useful options: + +- `--output PATH`: write to a specific PNG path. +- `--output-dir DIR`: choose the output directory when `--output` is not set. +- `--model-path VALUE`: use a Hugging Face model id or local model path. +- `--base-model VALUE`: SDXL base pipeline used when the 360 model only provides a UNet. +- `--vae-model VALUE`: VAE repo/path for SDXL models. +- `--scheduler euler|euler_a|heun|ddim|dpmsolver|dpmsolver-sde`: override the scheduler. +- `--upscale none|realesrgan|topaz`: post-process with no upscaler, Real-ESRGAN, or Topaz Photo AI. +- `--seam-inpaint`: shift and inpaint the wrap seam before saving. +- `--allow-downloads`: allow missing model files to be downloaded. + +The generator automatically appends `, equirectangular 360 view` to prompts that do not already include that suffix. + +## Outputs + +- Generated PNGs are saved in `output/`. +- Log files from desktop generations are saved in `logs/`. +- PNG metadata includes a `prompt` text field containing the enriched prompt. +- Temporary generation folders are created under the project root and cleaned up by the generator. + +Generated outputs, model weights, local caches, virtual environments, and build artifacts are ignored by Git. + +## Troubleshooting + +If model loading fails in offline mode, either place the required files locally, warm the Hugging Face cache, or rerun with: + +```bash +SKYMAP_ALLOW_DOWNLOADS=1 ./run.sh +``` + +If Real-ESRGAN upscaling fails, make sure `RealESRGAN_x4plus.pth` exists in the project root and the Python requirements are installed. + +If generation is slow or memory constrained, reduce `width`, `height`, or `steps`, disable seam inpainting, and keep upscaling off. + +## Project Layout + +```text +. +├── generate_equirect.py # Diffusers generation and post-processing CLI +├── run.sh # One-command development launcher +├── src/ # Vite/Three.js frontend +├── src-tauri/ # Tauri Rust desktop backend +├── configs/ # SDXL single-file loading config +├── public/ # Static frontend assets +├── models/ # Local model weights, ignored by Git +├── output/ # Generated PNG maps, ignored by Git +└── logs/ # Desktop generation logs, ignored by Git +```