One of the most salient features of smart city is to utilize big data to make our lives more convenient and more intelligent. This is usually achieved through solving a series of large-scale common and fundamental problems such as linear systems of equations, linear programs, etc. However, it is a very challenging task for resource-limited clients and small companies to solve such problems as the data volume keeps increasing. With cloud computing, an alternative is to solve complex problems by outsourcing them to the cloud. Nonetheless, data privacy is one of the main concerns. Many previous works on privacy-preserving outsourcing are based on cryptographic techniques like homomorphic encryption and have very high computational complexity, which may not be practical for big data applications. In this paper, we design an efficient privacy-preserving outsourcing algorithm based on arithmetic operations only for large-scale convex separable programming problems. Specifically, we first develop an efficient transformation scheme to preserve data privacy. Then we linearize the convex functions with arbitrary accuracy and solve the problem by outsourcing it to the cloud. The client can efficiently verify the correctness of the returned results to prevent any malicious behavior of the cloud. Implementations on Amazon Elastic Compute Cloud (EC2) platform show that the proposed scheme provides significant time savings.