Voice over IP is fast emerging as a strong contender to the traditional circuit-switched PSTN networks. Unlike the PSTN network, which requires dedicated lines, VoIP can share the network that is laid out to carry data traffic as well as other traffic categories. Securing VoIP and other real-time traffic is necessary considering the easy ways of hacking communication over internet. Most of the existing security solutions for VoIP such as IPSec, Secure Real Time Protocol (SRTP) and ZRTP use the standard symmetric encryption algorithms for encrypting voice traffic. In this paper, the authors propose an alternate encryption scheme that uses PKI architecture for the initial authentication and key exchange, and encrypts the real-time traffic with a symmetric algorithm using a unique key for each packet. The proposed algorithm expected to be less complex compared to the traditional encryption schemes in addition to enhancing the security of the communication. Initial analysis carried out by the authors indicates that the proposed scheme helps in improving the voice quality to a certain extent while maintaining the security of the communication