Detection of malicious DoH traffic using autoencoders
Date
2022-04-15Author
Srinivasan, Sriram
Advisor
Monroy, Sergio A.SalinasMetadata
Show full item recordCitation
Srinivasan, Sriram. 2022.
Detection of malicious DoH traffic using autoencoders -- In Proceedings: 21st Annual Undergraduate Research and Creative Activity Forum. Wichita, KS: Wichita State University, p. 28
Abstract
Domain Name System (DNS) is the phonebook of the Internet, translating
domain names to IP addresses of Internet servers hosting information user wants to find
online. DNS is unencrypted, revealing user’s browsing habits to Internet Service
Providers. To protect users’ privacy, most major browsers replaced DNS with DNS-over-
HTTPS (DoH), an encrypted form of DNS. Unfortunately, DoH can be exploited by
botnets to communicate with Command-and-Control center. The reason is that they can
use DoH to bypass traditional detection techniques that rely on the unencrypted DNS
traffic. Therefore, this research aims to design a method to detect botnet activity that uses
DoH, while protecting users’ privacy.
To detect DoH traffic from botnet malware. we propose to use autoencoders, a form of
deep neural network. The main idea of our detection approach is to use the autoencoder
recreation error to find the malicious DoH traffic. Specifically, an autoencoder aims to
recreate the input network traffic on its output. By training the autoencoder to recreate
both benign and malicious network traffic, we can obtain a 3D visualization, called
embeddings. . We then use K-means clustering to find the clusters of benign and
malicious traffic in the 3D-space. We observe that the classification precision is 91.38 %,
accuracy is 89.31 %, recall is 87.10 %, and f-score is 89.19 %.
In summary, the autoencoder provides a 3D representation using known labels for packet
flows. K-means performs the detection of whether the traffic is legitimate or malicious.
This yielded highly accurate and reliable classification results.
Description
Presented to the 21st Undergraduate Research and Creative Activity Forum (URCAF) held at the Rhatigan Student Center, Wichita State University, April 15, 2022.