Definition (Optimal)

An optimal Policy is a strategy that maximizes the expected cumulative reward (or minimizes cumulative cost) over time. In other words, no other policy yields a higher expected return from any given state or belief state.

Definition ($\epsilon$-optimal)

An ε-optimal policy is a Policy whose performance is within $\epsilon$ of the policy, where $\epsilon \geq 0$  is a small number representing the acceptable margin of sub-optimality. i.e. A policy $\gamma^{*}$ is ε-optimal if, $\forall \gamma$: $$J(\gamma) \geq J(\gamma^{*}) - \epsilon$$

Definition (Near-optimal)

A near-optimal policy is a strategy that performs close to the Policy but without a precise quantification of how close. The term is often used informally to describe policies that achieve high expected rewards, though not necessarily within a specified $\epsilon$  of the optimal.