In image classification applications for medical diagnosis, sensitivity and specificity are important performance metrics that are often inversely related. Both high sensitivity and high specificity are not always achievable for a given neural network; the trade-off and balance between them are not easily controllable. This paper proposes "ThreshNet", a novel method to address this dilemma. ThreshNet is composed of an ensemble of different neural networks. Many well-known networks were leveraged through transfer learning. With custom-designed dense layers, network parameters were tuned to optimize performance and enhance the diversity of members in the ThreshNet networks ensemble. To yield the ThreshNet system's decision, a threshold-based algorithm is proposed. Demonstrated with a brain tumor MRI dataset, ThreshNet systems consistently outperform individual networks. Specific sensitivity-specificity trade-offs and optimization goals can conveniently be achieved by adjusting the threshold parameter. Performance variance among ThreshNet systems is smaller than those among individual networks. To locate tumor(s) predicted by ThreshNet, a ResUNet-based image segmentation model was developed, achieving a Tversky index of 90.49% in predicting pixel-wise masks to mark tumor locations.