hardware//GPU//eGPU

2026-10-28

An eGPU is a desktop graphics card in an external enclosure connected to a laptop through a Thunderbolt or USB4 port, and it is used to give a laptop a dedicated GPU (for machine learning, usually an NVIDIA card for CUDA) without replacing the laptop. The port carries a PCIe link of about four lanes, a few gigabytes per second, against about 32 GB/s for the x16 slot of a desktop.

Whether that narrow link matters depends on where the work happens. For inference, the weights cross the cable once, when the model is loaded; after that every token reads them from the card's own VRAM at hundreds of gigabytes per second, and the port is idle.

The port only costs time at load, as long as the model and its KV cache fit in the card's memory. When they do not, part of the layers stays in system RAM and runs on the CPU, and that slow memory, not the cable, sets the speed.

A card with 24 GB of VRAM (an RTX 3090 or 4090) holds a 35B mixture of experts quantised to about 4.5 bits, with little room left for context.

Training moves more data over the link than inference (batches in, sometimes gradients or offloaded optimiser states), so the narrow port shows up there more often.