Stable Diffusion Setup: Self-Hosted Image Generation

Stable Diffusion is an open-source AI model that generates images from text. By running it on your own server, you can produce unlimited images, train custom models, and work with complete privacy. This guide covers AUTOMATIC1111 Web UI and ComfyUI installation, model management, LoRA and ControlNet usage in detail.

What is Stable Diffusion?

Stable Diffusion is a latent diffusion model developed by Stability AI. Key features:

Text-to-Image (txt2img): Generate images from text descriptions
Image-to-Image (img2img): Generate new images using existing images as reference
Inpainting: Edit specific areas of an image
ControlNet: Controlled generation with pose, edge and depth maps
LoRA: Low-resource model fine-tuning
Upscaling: Enlarge low-resolution images

System Requirements

Component	Minimum	Recommended
GPU	NVIDIA 6 GB VRAM	NVIDIA 12 GB+ VRAM
RAM	16 GB	32 GB+
Disk	30 GB	100 GB+ (for models)
OS	Ubuntu 22.04 LTS	Ubuntu 22.04/24.04 LTS
CUDA	11.8+	12.1+
Python	3.10+	3.10.x

AUTOMATIC1111 Web UI Installation

AUTOMATIC1111 is the most popular web interface for Stable Diffusion.

Manual Installation

hljs bash

# Dependencies
sudo apt-get update
sudo apt-get install -y python3-venv python3-pip git wget

# Clone repository
git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui

# Download model (Stable Diffusion XL)
wget -O models/Stable-diffusion/sd_xl_base_1.0.safetensors \
  "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors"

# Start
./webui.sh --listen --port 7860 --xformers --enable-insecure-extension-access

Docker with AUTOMATIC1111

hljs yaml

version: '3.8'
services:
  stable-diffusion:
    image: ghcr.io/abetlen/stable-diffusion-webui-docker:latest
    container_name: sd-webui
    ports:
      - "7860:7860"
    volumes:
      - sd_models:/app/models
      - sd_outputs:/app/outputs
      - sd_extensions:/app/extensions
    environment:
      - COMMANDLINE_ARGS=--listen --xformers --enable-insecure-extension-access --no-half-vae
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped

volumes:
  sd_models:
  sd_outputs:
  sd_extensions:

Systemd Service

hljs bash

cat > /etc/systemd/system/sd-webui.service << 'EOF'
[Unit]
Description=Stable Diffusion Web UI
After=network.target

[Service]
Type=simple
User=sduser
WorkingDirectory=/opt/stable-diffusion-webui
ExecStart=/opt/stable-diffusion-webui/webui.sh --listen --port 7860 --xformers
Restart=on-failure
RestartSec=10
Environment=HOME=/opt/stable-diffusion-webui

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable --now sd-webui

ComfyUI Installation

ComfyUI is an advanced node-based Stable Diffusion interface. Ideal for creating complex workflows.

Manual Installation

hljs bash

# Clone repository
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

# Python virtual environment
python3 -m venv venv
source venv/bin/activate

# Dependencies
pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

# Copy model to models directory
cp /path/to/sd_xl_base_1.0.safetensors models/checkpoints/

# Start
python main.py --listen 0.0.0.0 --port 8188

Docker with ComfyUI

hljs yaml

version: '3.8'
services:
  comfyui:
    image: ghcr.io/ai-dock/comfyui:latest
    container_name: comfyui
    ports:
      - "8188:8188"
    volumes:
      - comfy_models:/opt/ComfyUI/models
      - comfy_output:/opt/ComfyUI/output
      - comfy_custom:/opt/ComfyUI/custom_nodes
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    restart: unless-stopped

volumes:
  comfy_models:
  comfy_output:
  comfy_custom:

Model Download and Management

Popular Models

hljs bash

# Stable Diffusion XL Base
wget -O models/Stable-diffusion/sd_xl_base_1.0.safetensors \
  "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors"

# SDXL Refiner
wget -O models/Stable-diffusion/sd_xl_refiner_1.0.safetensors \
  "https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/resolve/main/sd_xl_refiner_1.0.safetensors"

# Stable Diffusion 1.5 (for low VRAM)
wget -O models/Stable-diffusion/v1-5-pruned-emaonly.safetensors \
  "https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors"

LoRA Usage

LoRA (Low-Rank Adaptation) allows you to fine-tune large models with low resources.

LoRA Model Download

hljs bash

# Place LoRA models in models/Lora directory
mkdir -p models/Lora

# Download LoRA from Civitai or HuggingFace
wget -O models/Lora/detail-enhancer.safetensors \
  "https://huggingface.co/example/detail-enhancer/resolve/main/model.safetensors"

LoRA Prompt Usage

To use LoRA in AUTOMATIC1111, add it to your prompt:

a beautiful landscape, mountains, sunset <lora:detail-enhancer:0.7>

ControlNet Setup

ControlNet allows you to control image generation with pose, edge maps and depth information.

hljs bash

# ControlNet extension installation (AUTOMATIC1111)
cd extensions
git clone https://github.com/Mikubill/sd-webui-controlnet.git

# ControlNet models
mkdir -p models/ControlNet
wget -O models/ControlNet/control_v11p_sd15_canny.pth \
  "https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_canny.pth"

wget -O models/ControlNet/control_v11p_sd15_openpose.pth \
  "https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_openpose.pth"

API Image Generation

Programmatic image generation via AUTOMATIC1111 API:

hljs python

import requests
import base64
from pathlib import Path

def generate_image(prompt, negative_prompt="", steps=30, width=1024, height=1024):
    url = "http://localhost:7860/sdapi/v1/txt2img"
    payload = {
        "prompt": prompt,
        "negative_prompt": negative_prompt,
        "steps": steps,
        "width": width,
        "height": height,
        "sampler_name": "DPM++ 2M Karras",
        "cfg_scale": 7,
        "seed": -1
    }
    
    response = requests.post(url, json=payload)
    result = response.json()
    
    # Save image
    image_data = base64.b64decode(result["images"][0])
    Path("output.png").write_bytes(image_data)
    print("Image saved: output.png")

# Usage
generate_image(
    prompt="a futuristic data center, neon lights, cyberpunk style, 8k",
    negative_prompt="blurry, low quality, distorted"
)

Performance Optimization

hljs bash

# Memory optimization with xformers
pip install xformers

# Launch parameters
./webui.sh --listen \
  --xformers \
  --opt-sdp-attention \
  --no-half-vae \
  --medvram \
  --port 7860

Memory Saving Parameters

Parameter	Description	VRAM Savings
--xformers	xformers attention	Medium
--medvram	Medium VRAM mode	High
--lowvram	Low VRAM mode	Very high
--opt-sdp-attention	SDP attention	Medium
--no-half-vae	VAE float32	Quality improvement

Generate unlimited images with Stable Diffusion on REXE GPU servers. SDXL models run at high speed with NVIDIA A100 GPUs.

Stable Diffusion Setup: Self-Hosted Image Generation

Table of Contents