We have that the Value Function is optimal over any arbitrary time step: $$V(t,\mathbf{X})=\inf_{u_{{[t,t+\Delta t]}}}\left\{ \int\limits _{t}^{t+\Delta t}L(t,x(\tau),u(\tau)) \, d\tau +V(t+\Delta t,x(t+\Delta t)) \right\}$$where $x(\cdot)$ is the state trajectory corresponding to $u_{[t,t+\Delta t]}$ and $x(t)=\mathbf{X}$.