Down Syndrome is a common disorder which causes intellectual disability among other symptoms. To date, no treatment exists for the learning difficulties associated with Down Syndrome. However, the pharmaceutical drug memantine has been shown to improve learning ability in a Down Syndrome model of mice (Ts65Dn) exposed to Context Fear Conditioning (CFC), an existing technique used in determining the extent of learning capability of mice. While the effect of memantine on learning capability in Ts65Dn mice is significant, the biological mechanism responsible for restoration of learning capability by memantine is poorly understood. One possible way to characterize this mechanism is by analyzing the neural protein profile data of normal and Down Syndrome mice with and without memantine treatment. In this work, we use a series of linear support vector machines to model the differential expression of 77 proteins obtained from the nuclear cortex of normal and Ts65Dn mice, with and without memantine treatment and with and without CFC stimulation. We use feature selection by weight threshold to select those proteins which play a significant role in characterizing each model. Per our findings, these subsets of proteins can be used to build more accurate classification models of the data than those subsets chosen using unsupervised learning or statistical analyses in previous studies. We recommend that the subsets of proteins selected using our proposed method be utilized in further biological study aiming to understand the effects of memantine on learning restoration.