AnyCalib: On-Manifold Learning for Model-Agnostic Single-View Camera Calibration