auto-git:
[add] README.md
This commit is contained in:
209
README.md
Normal file
209
README.md
Normal file
@@ -0,0 +1,209 @@
|
||||
# 3D Model Generator
|
||||
|
||||
A local desktop tool for turning short object ideas into image prompts, generated concept images, Replicate Hunyuan 3D models, and optional HDRI scene lighting.
|
||||
|
||||
The app runs as a `pywebview` desktop window. It uses Ollama for prompt generation, local Diffusers or Replicate for image generation, Replicate for image-to-3D conversion, and Blender for viewing generated `.glb` files.
|
||||
|
||||
## Features
|
||||
|
||||
- Generate Stable Diffusion-style prompts from a short object name.
|
||||
- Convert prompts into editable JSON generation settings.
|
||||
- Generate image batches with either:
|
||||
- local Diffusers using a local DreamShaper model
|
||||
- Replicate API using one of three configured image models
|
||||
- Generate 3D `.glb` models from selected images with `tencent/hunyuan-3d-3.1`.
|
||||
- Generate an equirectangular HDRI panorama and open the model in Blender.
|
||||
- Store image metadata in sidecar JSON files so prompts can be reused later.
|
||||
- Browse generated 2D images and generated 3D models in the built-in gallery.
|
||||
|
||||
## Requirements
|
||||
|
||||
- macOS
|
||||
- Python 3.11
|
||||
- Ollama running locally
|
||||
- An Ollama model named `mistral:latest`
|
||||
- Optional but recommended: a Replicate API token
|
||||
- Optional for local image generation: local Diffusers model files
|
||||
- Optional for HDRI: Topaz Photo AI CLI and local HDRI Diffusers model files
|
||||
- Optional for 3D viewing: Blender installed at `/Applications/Blender.app`
|
||||
|
||||
The launcher creates a local virtual environment in `.venv/` and installs Python dependencies from `requirements.txt`.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
./run.sh
|
||||
```
|
||||
|
||||
The script will:
|
||||
|
||||
1. Create `.venv/` if it does not exist.
|
||||
2. Activate the virtual environment.
|
||||
3. Install dependencies.
|
||||
4. Run `python main.py`.
|
||||
|
||||
If you want to force a specific Python executable:
|
||||
|
||||
```bash
|
||||
PYTHON=/path/to/python3.11 ./run.sh
|
||||
```
|
||||
|
||||
## Ollama Setup
|
||||
|
||||
The app expects Ollama at:
|
||||
|
||||
```text
|
||||
http://localhost:11434/api/generate
|
||||
```
|
||||
|
||||
and uses:
|
||||
|
||||
```text
|
||||
mistral:latest
|
||||
```
|
||||
|
||||
Install and run the model with:
|
||||
|
||||
```bash
|
||||
ollama pull mistral
|
||||
ollama serve
|
||||
```
|
||||
|
||||
## Replicate Setup
|
||||
|
||||
Open Settings in the app and paste your Replicate API token, or set it in your shell:
|
||||
|
||||
```bash
|
||||
export REPLICATE_API_TOKEN="r8_..."
|
||||
./run.sh
|
||||
```
|
||||
|
||||
The token is saved outside the repository at:
|
||||
|
||||
```text
|
||||
~/Library/Application Support/3d-model-generator/settings.json
|
||||
```
|
||||
|
||||
Do not commit this file.
|
||||
|
||||
## Image Generation Backends
|
||||
|
||||
Settings lets you choose between:
|
||||
|
||||
- `Local (Diffusers)`
|
||||
- `Replicate API`
|
||||
|
||||
The generated prompt JSON stays the same for both modes. Replicate mode maps the same positive prompt, negative prompt, dimensions/aspect ratio, steps where supported, and batch count to each model's API.
|
||||
|
||||
Configured Replicate image models:
|
||||
|
||||
| Model | Use | Listed price |
|
||||
| --- | --- | --- |
|
||||
| `black-forest-labs/flux-schnell` | Cheap drafts | $3 / 1000 output images |
|
||||
| `google/imagen-4-fast` | Balanced default | $0.02 / output image |
|
||||
| `bytedance/seedream-4` | Higher quality | $0.03 / output image |
|
||||
|
||||
Prices are listed by Replicate and may change.
|
||||
|
||||
## 3D Generation
|
||||
|
||||
Right-click a generated image and choose:
|
||||
|
||||
- `Generate 3D Model`
|
||||
- `Generate 3D Model + HDRI`
|
||||
|
||||
3D conversion uses:
|
||||
|
||||
```text
|
||||
tencent/hunyuan-3d-3.1
|
||||
```
|
||||
|
||||
Current listed Replicate price in the app:
|
||||
|
||||
```text
|
||||
$0.16 / unit
|
||||
```
|
||||
|
||||
The script waits up to 10 minutes for Replicate predictions before timing out.
|
||||
|
||||
## Local Model Paths
|
||||
|
||||
Some local paths are currently hard-coded and should be edited for your machine.
|
||||
|
||||
In `main.py`:
|
||||
|
||||
```python
|
||||
MODEL_PATH = "/Volumes/SD/ML-Models/diffusers/dreamshaper_8_diffusers"
|
||||
```
|
||||
|
||||
In `generate_equirect.py`:
|
||||
|
||||
```python
|
||||
model_path = "/Volumes/SD/ML-Models/diffusers/hdri-panorama-v1-diffusers"
|
||||
topaz_cli = "/Applications/Topaz Photo AI.app/Contents/MacOS/Topaz Photo AI"
|
||||
```
|
||||
|
||||
In `main.py`, external app paths are also hard-coded:
|
||||
|
||||
```python
|
||||
"/Applications/Blender.app/Contents/MacOS/Blender"
|
||||
"Adobe Photoshop 2024"
|
||||
```
|
||||
|
||||
If you only use Replicate image generation and 3D generation, the local DreamShaper path is not needed for image generation.
|
||||
|
||||
## Output Files
|
||||
|
||||
Generated files are written under:
|
||||
|
||||
```text
|
||||
web/output/
|
||||
```
|
||||
|
||||
Typical outputs:
|
||||
|
||||
- `.png` generated image
|
||||
- `.png.json` prompt metadata sidecar
|
||||
- `.glb` generated 3D model
|
||||
- `_hdri_seamless.png` generated HDRI image
|
||||
|
||||
`web/output/` is ignored by Git.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```text
|
||||
main.py Desktop app, UI API, prompt flow, image generation
|
||||
image_to_3d.py Replicate Hunyuan 3D conversion
|
||||
generate_equirect.py HDRI generation and seam fixing
|
||||
scene_setup.py Blender scene setup for generated models
|
||||
web/index.html App UI
|
||||
requirements.txt Python dependencies
|
||||
run.sh Virtualenv setup and app launcher
|
||||
```
|
||||
|
||||
## Git And Secrets
|
||||
|
||||
The repository ignores local output, model weights, virtual environments, Hugging Face cache files, and local secret/config files.
|
||||
|
||||
Before publishing, it is still worth running:
|
||||
|
||||
```bash
|
||||
git status --short --ignored
|
||||
git ls-files -co --exclude-standard
|
||||
```
|
||||
|
||||
Do not commit:
|
||||
|
||||
- `.env`
|
||||
- `settings.json`
|
||||
- Replicate API tokens
|
||||
- model weights
|
||||
- generated media
|
||||
- `web/output/`
|
||||
|
||||
## Notes
|
||||
|
||||
- `numpy<2` is pinned because some installed ML/runtime dependencies may still be incompatible with NumPy 2.x.
|
||||
- The app currently targets macOS paths and behavior.
|
||||
- HDRI generation depends on local models and Topaz Photo AI CLI.
|
||||
- Replicate image generation and Hunyuan 3D generation spend Replicate credits.
|
||||
Reference in New Issue
Block a user