mathematics//optimization//evolutionary algorithm

An evolutionary algorithm is a derivative-free optimization method that keeps a population of candidate solutions and improves it by selection, recombination and random variation, borrowing the logic of natural selection. It is used when the thing being optimized has no useful gradient: a discrete design, a schedule, the gains of a controller tuned on a simulator, the wording of a prompt. All it needs is a way to score any candidate, the fitness function.

An evolutionary algorithm is a derivative-free optimization method that keeps a population of candidate solutions and improves it by selection, recombination and random variation, borrowing the logic of natural selection. It is used when the thing being optimized has no useful gradient: a discrete design, a schedule, the gains of a controller tuned on a simulator, the wording of a prompt. All it needs is a way to score any candidate, the fitness function.

One generation runs a short cycle, repeated until the scores stop improving or the budget ends:

1Score every candidate2Select the fitter ones3Recombine pairs (crossover)4Mutate a little5New population

The genetic algorithm is the classic form: each candidate is encoded as a string of genes (bits, numbers, choices), crossover splices two parents into a child, and mutation flips or nudges a few genes so the population keeps exploring. Selection pressure decides the balance: keeping only the very best converges fast and often prematurely on a local optimum; keeping weaker candidates in play explores more and costs more evaluations. Other members change the variation step: evolution strategies such as CMA-ES perturb real-valued vectors and adapt the spread of the perturbation as they go, and genetic programming evolves programs or formulas.

It trades evaluations for generality.

An evolutionary search will optimize almost anything that can be scored, including non-smooth, noisy and discrete objectives, and it pays for it with thousands of evaluations where a gradient method would need dozens of steps.

Choose it when each evaluation is cheap and the landscape is rough: tuning a drone's PID gains in simulation, laying out a factory floor, searching prompt wordings (automatic prompt engineering). When an evaluation is a bench test or an hour of compute, Bayesian optimization spends trials far more sparingly; when the problem is smooth with many parameters, gradient descent wins by orders of magnitude.

It parallelizes trivially: every candidate of a generation can be scored at once on its own machine.

It gives no certificate. The best candidate found is good and carries no proof of optimality, which matters for a combinatorial problem where an exact solver could prove the optimum for small instances.