ML//RL//policy

A policy is the function by which an agent chooses what to do, mapping an observed state to an action or, more generally, to a probability distribution over actions, written as follows.

A policy is the function by which an agent chooses what to do, mapping an observed state to an action or, more generally, to a probability distribution over actions, written as follows.

π(a∣s)\pi(a \mid s)π(a∣s)

It reads as the probability of taking action aaa given state sss: for a quadruped robot, sss is the joint angles, the body's tilt and the terrain seen by its cameras, and aaa the torque commands for the next few milliseconds. A policy is trained to maximize the sum of a reward function over time.

In control the same object has another name: the control policy is the decision rule, in the end a piece of code, that turns past measurements into the next command, and a PID loop or an LQR gain matrix is a policy as much as a trained network is. In reinforcement learning it is usually a neural network that takes the state and returns the action, and in control the policy is the controller. What changes between fields is how it is obtained: written from a model, computed from an optimization, or learned from reward.

In a Markov decision process the optimal policy needs no separate search: once the Bellman equation is solved, it picks in each state the action that reaches the maximum. Fixing a policy also removes every choice from the process, which leaves a Markov chain whose long-run behaviour can be analysed.

In reinforcement learning, the policy decides what the agent does; a value function estimates how good a state or action is. They may share one neural network, but they answer different questions.

A deterministic policy always chooses the same action for a state. A stochastic policy preserves exploration (exploration-exploitation trade-off) and can represent uncertainty or a mixed strategy, which pays when an opponent is watching.

The agent rarely observes the full environment state. Its policy usually maps partial observations to actions, so memory and beliefs about hidden state can matter as much as the policy network; deciding on beliefs is the POMDP.

An LLM resembles a policy over tokens: context is the observation and the next token is the action. An embodied agent extends that loop until actions also alter the external environment.