The Public Cloud is throttling your creativity.

If you are building an application on top of OpenAI's DALL-E or Midjourney right now, you are building on rented land. Expensive, throttled, censored land. I watched a client last week try to scale a marketing generation tool using public APIs; their latency spiked to 45 seconds per image during US peak hours. That is unacceptable.

The solution isn't magic. It's bare-metal control. By March 2023, the open-source community has given us tools that outperform paid services if—and only if—your infrastructure can handle the heat. Stable Diffusion 1.5, combined with the new ControlNet adapters, is the current gold standard for controlled image generation. But it eats I/O for breakfast.

I'm going to show you how to set up a dedicated generation node. We are focusing on the Automatic1111 web UI in API mode, secured behind Nginx, running on a high-availability environment like CoolVDS.

The Bottleneck isn't always the GPU

Everyone obsesses over VRAM. Yes, you need it. But here is the war story: We had a setup with massive A100s that was crawling. Why? Model switching.

Stable Diffusion models (checkpoints) are often 4GB to 7GB. When you switch from v1-5-pruned-emaonly.ckpt to a custom fine-tune like f222.ckpt or load a specific LoRA, that data has to move from disk to RAM to VRAM. If you are on a budget VPS with standard SSDs (or worse, spinning rust), your application hangs for 10 seconds just loading the file.

Pro Tip: Always convert your models to .safetensors format. It avoids the security risk of Python pickling inherent in .ckpt files and maps faster to memory.

This is where CoolVDS NVMe instances become the only logical choice for this workload. We are talking about disk read speeds that don't choke when you start swapping 6GB files ten times a minute.

Phase 1: The Environment (Ubuntu 22.04 LTS)

We are using Python 3.10. Do not try 3.11 yet; the torch ecosystem isn't ready for it as of this month. We need a clean slate.

sudo apt update && sudo apt upgrade -y
sudo apt install wget git python3 python3-venv python3-pip libgl1 libglib2.0-0 -y

Don't install dependencies globally. It’s sloppy. We create a dedicated user for the AI service to isolate processes—critical if you are complying with Norwegian security standards.

sudo useradd -m -s /bin/bash sduser
sudo su - sduser
mkdir ~/stable-diffusion
cd ~/stable-diffusion

Phase 2: The Core Installation & xformers

Automatic1111 is the repository we want. It has the best API support. Clone it:

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git .
python3 -m venv venv
source venv/bin/activate

Now, the secret sauce: xformers. This library from Meta dramatically speeds up attention mechanisms and reduces VRAM usage. Without it, you are wasting money.

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117
pip install xformers==0.0.16rc425

Launch the instance specifically in API mode. We also add --listen to bind to the network, but we will firewall this later.

./webui.sh --xformers --api --nowebui --listen --port 7860

If you see Model loaded in 1.2s, your I/O is healthy. If it says 15.4s, your hosting provider is stealing your CPU cycles.

Phase 3: Productionizing with Systemd and Nginx

Don't run this in a tmux session like an amateur. Create a systemd service.

/etc/systemd/system/sd-api.service

[Unit]
Description=Stable Diffusion API
After=network.target

[Service]
User=sduser
WorkingDirectory=/home/sduser/stable-diffusion-webui
ExecStart=/home/sduser/stable-diffusion-webui/webui.sh --xformers --api --nowebui --port 7860
Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Enable it:

sudo systemctl enable sd-api
sudo systemctl start sd-api

The Nginx Reverse Proxy

Direct exposure of port 7860 is a security nightmare. We need Nginx to handle SSL and rate limiting. This is crucial for GDPR compliance; you cannot transmit generated user data over plain HTTP.

server {
    listen 80;
    server_name ai.your-domain.no;

    location / {
        proxy_pass http://127.0.0.1:7860;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        
        # Increase timeout for long generation tasks
        proxy_read_timeout 300s;
    }
}

Note the proxy_read_timeout 300s;. High-step generated images (e.g., 150 steps) can take time. If Nginx cuts the connection at 60 seconds (default), your backend finishes the job but the client gets a 504 error. Wasted compute.

The Data Sovereignty Advantage

Here is why CoolVDS matters for the Norwegian market. When you use Midjourney, that data processes in the US. Under Schrems II, this is a legal grey area for enterprise data. By hosting on a CoolVDS node in Oslo, you guarantee data residency. The image is generated on Norwegian soil, stored on Norwegian NVMe, and delivered via Norwegian transit.

Furthermore, latency to the Oslo Internet Exchange (NIX) is negligible. For real-time applications (like an AI drawing tool), that 20ms savings creates a "snappy" feeling that US-hosted APIs cannot match.

Final Optimization Checks

Building your own AI infrastructure is not just about cost savings—though dropping a $3,000/month API bill to a fixed VDS cost is nice. It is about reliability. When the hype train crashes the public servers, your private node keeps rendering.

Ready to deploy? Don't let slow I/O kill your generation time. Spin up a CoolVDS High-Performance instance today and get your inference times down to where they belong.