We consider the linear stability of Hill's vortex with respect to axisymmetric perturbations. Given that Hill's vortex is a solution of a free-boundary problem, this stability analysis is performed by applying methods of shape differentiation to the contour dynamics formulation of the problem in a three-dimensional axisymmetric geometry. This approach allows us to systematically account for the effect of boundary deformations on the linearized evolution of the vortex under the constraint of constant circulation. The resulting singular integro-differential operator defined on the vortex boundary is discretized with a highly accurate spectral approach. This operator has two unstable and two stable eigenvalues complemented by a continuous spectrum of neutrally stable eigenvalues. By considering a family of suitably regularized (smoothed) eigenvalue problems solved with a range of numerical resolutions, we demonstrate that the corresponding eigenfunctions are in fact singular objects in the form of infinitely sharp peaks localized at the front and rear stagnation points. These findings thus refine the results of the classical analysis by Moffatt & Moore.