ML//transfer learning

Transfer learning is a training technique in which the features a model learned on a big task are reused for a smaller one, and it is the reason not every problem needs millions of labeled samples. ImageNet features transfer to medical imaging, satellite imagery and art classification. Pre-training is transfer learning at industrial scale: general language or vision is learned once and transferred to any downstream task.

Transfer learning is a training technique in which the features a model learned on a big task are reused for a smaller one, and it is the reason not every problem needs millions of labeled samples. ImageNet features transfer to medical imaging, satellite imagery and art classification. Pre-training is transfer learning at industrial scale: general language or vision is learned once and transferred to any downstream task.

In a factory it is how visual inspection gets built. Nobody trains a CNN from scratch to find defects in welds, paint or moulded parts: a network pretrained on millions of generic images is fine-tuned on the plant's own photos, and then a few hundred to a few thousand labelled images are usually enough. The early layers, which detect edges, textures and blobs, carry over almost unchanged; only the later layers learn what a porous weld looks like.

The transfer is only as good as the resemblance between the two tasks. Natural photographs transfer well to inspection photos; they transfer worse to spectrograms, where shifting a pattern along the frequency axis changes its meaning, and to signals that are not images at all.

Fine-tuning on few images makes shortcut learning easy: if every defective part was photographed on the same day under different lighting, the network learns the lighting. The station's illumination matters as much as the network, and a split by day or by batch is what exposes the shortcut (train-validation-test split).

An available pretrained model is one of the few good reasons to reach for a network on a small industrial problem at all; without one, a classical model over hand-built features is usually cheaper to train and to maintain (applied ML).