Renting an A100 on RunPod from Claude Code to fine-tune Stable Diffusion

A walk through one real rental: an A100 80GB pod on RunPod, created and driven from Claude Code to fine-tune Stable Diffusion 1.5 on 6,413 kanji. It explains what the pod is made of, sizes the GPU from the memory the job needs, measures the speed before committing to the long run, and records the four things that went wrong, from a CPU quota PyTorch could not see to a warning that the physical machine was failing.

The job was to fine-tune Stable Diffusion 1.5 on 6,413 kanji, the second arm of a project whose first arm trained a small diffusion model on a laptop CPU. The laptop could not do this one: Stable Diffusion's denoiser has 860 million parameters, and training it at 512 pixels needs a large GPU for a few hours. Buying one makes no sense for a few hours, so the GPU was rented, from RunPod, and the whole rental was driven from Claude Code: choosing the card, creating the machine, logging into it, running the training and turning it off.

What follows is that rental, step by step, with what the machine is underneath at each step. The piece is meant to teach, and that is a choice more than a format. Almost none of the clicking had to be done by hand: an agent picked the card, created the pod, logged in and ran the job, and the provider itself hands over the instructions for the agent. What an agent cannot do for the person paying is know whether 24 GB will be enough, why a job is crawling on a card that cost two dollars an hour, or what a warning about the physical machine puts at risk. The more of the work agents take over, the more the useful part of knowing a tool becomes knowing what it is made of. The aim is that a reader could rent one, size it for a different job and predict what will go wrong.

Before RunPod26 · oct 21Read noteI used to rent GPUs through Hugging Face, by attaching paid hardware to a Space. This time I signed up on RunPod, got $15 in credit from a referral, and the first thing the site gave me was a prompt to paste into my agent so that it would set everything up. I did not create the pod or type a single command on it: Claude Code did, and I watched.

Before the pod: machines inside machines

Renting compute means renting a slice of someone else's computer, and there are three ways to cut the slice, each with a different wall between tenants.

Bare metalVirtual machinesContainersskip

Bare metal

A whole physical server for one customer, with nothing between their software and the hardware: they see every core and every GPU and install the operating system themselves (bare metal). It is the fastest and the most isolated option, and the least flexible, because the unit of sale is a whole machine.

Virtual machines

A hypervisor runs on the physical server and presents each tenant with a virtual machine: virtual processors, virtual memory and virtual devices, on which the tenant boots a complete operating system with its own kernel. The wall is strong, since tenants share nothing but the hypervisor, and it costs a little: each machine carries a whole operating system, takes tens of seconds to boot, and reaches devices through a layer of emulation unless a device is handed to it directly.

Containers

A container is a group of ordinary processes on the host's own kernel, fenced off by two kernel features. The first, namespaces, gives the processes their own view of the system (their own filesystem, process list, network interfaces and hostname), and the second, cgroups, caps what they may consume (CPU time, memory). There is no second kernel and nothing to boot: a container starts in about a second, and a GPU can be handed to it by mounting the device and the driver's libraries into it, which NVIDIA's container toolkit does. The wall is thinner than a virtual machine's, because every tenant shares one kernel, and the view is fenced more thoroughly than the resources, which shows below.

Three ways to cut a server. What each tenant shares is everything below its own box.

GPU clouds mostly rent containers. A GPU is expensive and comes in whole units, a container hands one over at full speed with no layer in between, and it starts in seconds from an image that already holds the drivers' libraries and the framework, which is what makes billing by the second worth having.

RunPod is one of the companies that sell this, of the kind now called neoclouds: clouds built around GPUs rather than general computing. Zhen Lu and Pardeep Singh, two developers at Comcast, started it in late 2021 with the Ethereum mining rigs in their New Jersey basements, once it was clear that mining was ending, and found their first users by offering free GPU time on Reddit in exchange for feedback; by January 2026 it reported $120 million in annual revenue and half a million developers1. For a rental like this one, the practical advice is short: pick it when the job needs one big GPU for hours rather than a managed service, keep anything valuable on storage that outlives the machine, and read the price as per second, because that is how it is billed.

1TechCrunch, AI cloud startup Runpod hits $120M in ARR, and it started with a Reddit post, January 2026, which also lists a 2024 seed round co-led by Intel Capital and Dell Technologies Capital.

What a pod is

A RunPod pod is a container running on a physical machine in one of RunPod's data centers. The machine (a host) has several GPUs, often eight, dozens of CPU cores and hundreds of gigabytes of memory, and it is shared: each pod on it gets a slice. Four things define the slice, and each one behaves differently from a computer of your own.

GPUThe GPU is passed through whole: inside the container, nvidia-smi shows NVIDIA A100-SXM4-80GB, 81920 MiB, the full card, used by nobody else while the pod runs. What the container brings is the CUDA runtime (the libraries inside the image); what the host brings is the driver, and a driver runs runtimes up to its own version. This pod's image ships CUDA 12.8; the host's driver reported 13.2, so it can run it.

CPU AND MEMORYThe pod gets 16 vCPU and 250 GB of memory out of a host with 128 cores and about 2 TB. The limit is enforced by the kernel's cgroups, which cap how much CPU time the container's processes may use, and that has a consequence met below: the container can still see all 128 cores.

DISKThere are three kinds, with different lifetimes. The container disk (here 40 GB) holds the image's filesystem and anything written outside /workspace, and it is wiped when the pod stops or restarts. The volume disk (50 GB, mounted at /workspace) survives a stop but lives on that one host, so it dies with the host or when the pod is terminated. A network volume lives in the data center, survives anything short of being deleted, and can be attached to another pod.

NETWORKThe container's ports are mapped to public ones. Port 22 inside became port 16519 on the host's public address, so SSH was ssh [email protected] -p 16519; an HTTP port would get a proxy URL of the form https://<pod id>-<port>.proxy.runpod.net.

The rented slice of the host, what survives what, and what is billed.

Billing follows the same parts, and it is per second: compute while the pod runs, plus each disk by the gigabyte1. For this pod the console showed $1.79 an hour of compute, $0.006 for the 40 GB container disk and $0.007 for the 50 GB volume, $1.80 in all. A stopped pod stops paying for compute and the container disk and keeps paying for its volume disk, at a higher rate; only terminating it ends every charge.

1RunPod, Pod pricing: compute and container and volume disk are billed per second, deducted from prepaid credits.

The pod's detail page: the slice of the host it gets, and the hourly price split into compute and two disks. The yellow banner is the subject of the last part of this piece.

RunPod sells this in two tiers. Machines in Secure Cloud sit in data centers RunPod runs; machines in Community Cloud belong to third parties renting their hardware through RunPod, at lower prices and with fewer guarantees. For a job of several hours, losing the machine halfway costs more than the difference, so this rental used Secure Cloud.

Sizing the GPU

The card is chosen from the memory the job needs, and for a full fine-tune most of that memory is not the model. Training with the AdamW optimiser keeps, for every parameter, the weight, its gradient and two optimiser statistics, all in 32-bit floats, and this script also keeps a moving average of the weights for sampling:

Memory per parameter, fp32 weights4 bytes gradients4 bytes AdamW moments8 bytestwo statistics EMA copy4 bytes For Stable Diffusion 1.5's U-Net parameters860M total860M × 20 bytes ≈ 17 GBbefore any activation

On top of that sit the activations of a batch of 32 images at 512 pixels (latents of 64×64), which gradient checkpointing trades for compute by recomputing them during the backward pass instead of storing them all. Measured on the running job, the total was 33.5 GB. That rules out the 24 GB cards (the RTX 4090, the RTX 3090) for this batch size without cutting the batch and adding gradient accumulation, makes a 40 GB A100 a squeeze, and leaves an 80 GB card with room.

The catalog read at 08:31 UTC, Secure Cloud, gave these candidates:

GPU Memory $ per hour Stock

A100 SXM 80 GB 1.79 medium, four data centers

A100 PCIe 80 GB 1.79 none

H100 SXM 80 GB 3.99 medium

RTX PRO 6000 Blackwell 96 GB 2.49 medium

RTX 4090 24 GB 0.89 high

At these prices an H100 would have to train this model more than 2.2 times faster than an A100 to cost less per finished run, and the speed-up reported for diffusion training is usually smaller than that; it pays when waiting matters more than money. The A100 SXM won.

Driving it from Claude Code

RunPod publishes a plugin for Claude Code1 that exposes its API as tools (listing GPUs and their stock, creating and stopping pods, reading logs) through the Model Context Protocol. Installing it is three commands, and signing in is an OAuth login in the browser, so no API key is ever pasted anywhere.

1RunPod, Agent setup.

BASH Copy

claude plugin marketplace add runpod/runpod-plugins-official claude plugin install runpod@runpod # then, inside Claude Code: /reload-plugins, and /mcp > runpod > Sign in with Runpod

The tools manage infrastructure; they do not open a shell inside the pod. For that the pod needs SSH, and SSH needs a public key registered on the account before the pod is created, because RunPod writes the account's keys into the new container as an environment variable (PUBLIC_KEY) that its start script copies into authorized_keys. The account had none, so a key was made for this purpose alone and registered:

BASH Copy

ssh-keygen -t ed25519 -N "" -C "runpod-kanji" -f ~/.ssh/runpod_ed25519

Who does what. The API manages machines and never opens a shell, so everything inside the pod goes through SSH.

The pod itself was one request. It starts from RunPod's official PyTorch template, opens only the SSH port, and pins a floor on the host's CUDA version so the image cannot land on a driver too old for it:

JSON Copy

{ "name": "kanji-diffusion-sd15", "templateId": "runpod-torch-v280", "cloud": "SECURE", "gpu": { "id": "NVIDIA A100-SXM4-80GB", "count": 1, "minCudaVersion": "12.8" }, "dataCenterIds": ["EU-RO-1", "US-KS-2", "US-MD-1", "US-WA-1"], "ports": ["22/tcp"], "startSsh": true, "startJupyter": false, "disk": 40 }

The pod was running two seconds after the request, because the host already had the image cached, and its SSH address appeared about seventeen seconds later.

The pod in the console's list, with its hourly price, GPU and data center.

Four things that went wrong

None of the four was RunPod's fault in the sense of a bug, and each one cost minutes of a billed GPU. Three came from treating the pod like an ordinary Linux machine; the fourth came from the machine underneath.

Pip refused to install

pip install -e . failed on the pod with externally-managed-environment. Ubuntu 24.04, which the image is built on, marks its system Python as managed by the operating system (PEP 668) so that pip cannot break packages apt installed. Inside a disposable container that protection guards nothing, so the fix is to say so: pip install --break-system-packages -e ., or a virtual environment if the image will live longer.

PyTorch saw 128 cores and had 16

Preparing the data renders each kanji and then encodes its captions with a small text model on the CPU. The rendering finished in a minute; the encoding sat for over five minutes with the process at 1,590% CPU and no end in sight. nproc inside the pod answered 128.

The cause is the cgroups quota from the first section. It limits how much CPU time the container's processes get, which is what makes the pod have 16 vCPU, but it does not change what the kernel reports as the number of processors, so os.cpu_count() and PyTorch's default thread pool see the host's 128 cores. PyTorch then starts 128 threads for every matrix operation, the quota lets about 16 of them run at a time, and the rest spend their time waiting and being switched in and out. The fix is to tell the libraries the real number:

BASH Copy

export OMP_NUM_THREADS=16 MKL_NUM_THREADS=16

With it, the whole preparation took about two minutes. Every one of the lost minutes was billed for an A100 that was doing nothing.

What the 128-thread default does inside a 16-vCPU quota, and what setting the thread count fixes.

On any container with a CPU quota, set OMP_NUM_THREADS (and MKL_NUM_THREADS) to the vCPU count you were given before running anything numerical on the CPU. The count is on the pod's detail page.

A remote pkill killed its own shell

Restarting the stuck job with ssh pod 'pkill -f kanji_diffusion.prepare; ...' came back with exit code 255 and did nothing else. pkill -f matches against full command lines, and the remote shell running that command had the pattern in its own command line, so it killed itself before reaching the rest. Long jobs were then launched from a script on the pod, detached from the SSH session:

BASH Copy

setsid nohup bash /workspace/run_train.sh > /dev/null 2>&#x26;1 &#x3C; /dev/null &#x26;

setsid puts the job in a new session, so closing the SSH connection does not send it a hang-up signal, and the redirections leave SSH nothing to wait for.

The host reported a critical error

Six minutes into the rental, the pod's page in the console showed a yellow banner: RunPod had detected a critical error on this machine which may affect some pods and recommended backing up and moving to a new pod. The training was healthy at that moment (GPU at 100%, 62 °C, loss falling), so the decision was about risk. Two things were exposed. The training run kept its progress only in memory, because the script saved sample sheets along the way but no checkpoints, so a failure would lose every step done. And the dataset sat on the volume disk, which lives on that very host.

Moving at once would have cost about ten minutes (a new pod, the data rendered again) to protect against a failure that might not come; staying put at risk at most the training done so far, about an hour and a half of A100 time, or $3. The run stayed. The lesson is in how the job was set up rather than in the decision: anything that takes hours should write checkpoints to a network volume, which outlives the host, and then a warning like this one costs nothing to act on.

TODO(final): outcome of the run on that host.

Measure before you commit

Before the long run, a forty-step run of the same script measured the real speed: forty steps and a sheet of samples took about 48 seconds, which means each step took 0.95 seconds at batch 32. The estimate made beforehand, from published throughput figures, was 2.5 to 3.5 seconds. On that estimate the plan was 6,000 steps in four to six hours, $9 to $14; on the measurement it became under two hours, about $3 for the training itself.

The test also checked that the whole pipeline worked end to end (loading the model, encoding the data, training, sampling and saving the result), so the long run could not fail in its last minute on a typo. At forty steps the samples were still photographs, a bird, a mountain, a pizza, already in black and white: the model had learned the colour of kanji before their shape.

TODO(final): figure with the forty-step samples next to the final ones.

Cost and shutdown

TODO(final): total time from creation to termination, total cost from the billing read, and the billing screenshot.

A pod bills until it is stopped or terminated, so the last step is not optional. Stopping keeps the volume disk, and with it whatever is on /workspace, at a storage rate; terminating deletes it and ends every charge. The results were copied to the laptop with scp first, and then the pod was terminated.