In this work, a novel combined trajectory planner and tracking controller is developed for autonomous vehicles operating on off-road deformable terrains. Common approaches to trajectory planning and tracking often rely on model-dependent schemes, which utilize a simplified model to predict the impact of control inputs to future vehicle response. However, in an off-road context and especially on deformable terrains, accurately modeling the vehicle response for predictive purposes can be challenging due to the complexity of the tire-terrain interaction and limitations of state-of-the-art terramechanics models in terms of operating conditions, computation time, and continuous differentiability. To address this challenge and improve vehicle safety and performance through more accurate prediction of the plant response, in this paper, a nonlinear model predictive control framework is presented that accounts for terrain deformability explicitly using a neural network terramechanics model for deformable terrains. The utility of the proposed scheme is demonstrated on high fidelity simulations for a notional lightweight military vehicle on soft soil. It is shown that the neural network based controller can outperform a baseline Pacejka model based scheme by improving on performance metrics associated with the cost function. In more severe maneuvers, the neural network based controller can achieve sufficient fidelity as compared to the plant to complete maneuvers that lead to failure for the Pacejka based controller. Finally, it is demonstrated that the proposed framework is conducive to real-time implementability.