mathematics//numerical methods//stiffness

Stiffness is the property of a differential equation whose dynamics mix time scales that are far apart, so that an explicit integrator is forced to take steps sized for the fastest mode long after that mode has died out, and recognising it is what keeps a simulation of hours from costing millions of steps. An electric motor is the everyday case: its electrical dynamics settle in milliseconds while its casing heats up over tens of minutes. A thermal study of that motor only cares about the slow mode, yet an explicit method must keep its step below about twice the electrical time constant, or the long-dead fast mode comes back to life as a numerical explosion (numerical stability).

Stiffness is the property of a differential equation whose dynamics mix time scales that are far apart, so that an explicit integrator is forced to take steps sized for the fastest mode long after that mode has died out, and recognising it is what keeps a simulation of hours from costing millions of steps. An electric motor is the everyday case: its electrical dynamics settle in milliseconds while its casing heats up over tens of minutes. A thermal study of that motor only cares about the slow mode, yet an explicit method must keep its step below about twice the electrical time constant, or the long-dead fast mode comes back to life as a numerical explosion (numerical stability).

Implicit methods remove the ceiling by evaluating the dynamics at the point they arrive at. Implicit Euler on x˙=λx\dot x=\lambda xx˙=λx gives xk+1=xk/(1−λΔt)x_{k+1}=x_k/(1-\lambda\Delta t)xk+1​=xk​/(1−λΔt), which decays for any step; the price is solving an equation at every step, a linear system for a linear model and a few Newton iterations for a nonlinear one. Radau and BDF are the serious versions, and LSODA switches by itself between an explicit and an implicit method as the problem demands. A mode of 1 ms coupled to one of 10 s, solved over 100 s to the same tolerance with SciPy's solve_ivp, shows the difference: the explicit RK45 needs about 190,000 evaluations of the dynamics, Radau about 5,400 and LSODA about 1,500, for the same answer. If solve_ivp takes forever on a model that looks simple, suspect stiffness before the computer.

It is a property of the model and the question together. The same motor model is stiff for a thermal study over an hour and harmless for a current-loop study over 50 ms, where the fast mode is the subject and a small step is needed anyway. A ratio between the fastest and slowest time constants of a thousand or more, with interest in the slow one, is the usual sign.

Many plants are stiff by construction: chemical kinetics with fast and slow reactions, power electronics inside a slow thermal envelope, a battery model with fast polarization and slow capacity fade, a mechanical structure with a stiff mount. Remedies other than an implicit solver are removing the fast mode from the model by treating it as instantaneous (an algebraic equation instead of a differential one), or simulating the two time scales separately (model reduction, time-scale separation).

The same shape appears in optimization. An ill-conditioned loss, steep in one direction and nearly flat in another, is a stiff system for gradient descent: the steep direction caps the learning rate and the flat one sets how long training takes (condition number).

In a real-time setting implicit methods are rarer, because the equation solved at each step has a cost that varies; an embedded estimator usually keeps the fast mode out of its model instead (numerical integration).