We study active simultaneous localization and mapping (SLAM) as an optimal stochastic control problem with decisions under partial information. We formulate active SLAM as a (nonstandard) partially observed Markov decision process (POMDP) involving the joint state of the robot and map, propose a novel exploration cost capturing the geometry of the state space, and derive rigorous approximation guarantees. Our formulation applies under general conditions suitable for a variety of robotics problems. Numerical studies using standard learning algorithms demonstrate the ability to identify near-optimal policies.