We show that locally diffeomorphic exponential maps can be defined for any second-order differential equation, and give a (possibly nonlinear) covariant derivative for any (possibly nonlinear) connection. We introduce vertically homogeneous connections as the natural correspondents of homogeneous second-order differential equations. We provide significant support for the prospect of studying nonlinear connections via certain, closely associated secondorder differential equations. One of the most important is our generalized Ambrose-Palais-Singer correspondence.