My next class:

Is it Possible to Identify DNS over HTTPs Without Decrypting TLS?

Published: 2019-12-17. Last Updated: 2019-12-17 03:47:52 UTC
by Johannes Ullrich (Version: 1)
3 comment(s)

Whenever I talk about DNS over HTTPS (DoH), the question comes up if it is possible to fingerprint DoH traffic without decrypting it. The idea is that something about DoH packets is different enough to identify them.

This evening after recording my podcast, I experimented a bit with this idea to see what could be used to identify DNS over HTTPS traffic. To run this experiment, I used Firefox. I used Firefox for  a couple of different reasons:

  • I consider the Mozilla DoH implementation mature. Mozilla was one of the trailblazers for DoH and has made it very easy to enable DoH. I find Chrome to be a bit more "tricky" in that it is more careful in its use of DoH.
  • Firefox, just like Chrome, allows me to collect TLS master keys via the SSLKEYLOGFILE environment variable. This allowed me to decrypt and separate the DoH from other HTTPS traffic

At this point, I would call the experiment a "proof of concept." It is not a conclusive experiment. I only collected a few minutes of traffic and went maybe to a dozen different sites. All tests were performed on a Mac using Firefox 71 and Cloudflare as a resolver. I may get around to do more testing during the day and will update this post accordingly.

I started by running tcpdump (otherwise I forget it and realize that I need to start it after I started Firefox)

% tcpdump -i en8 -w /tmp/ssl.pcap

Next, in a different terminal, I set the SSLKEYLOGFILE environment variable

% export SSLKEYLOGFILE=/tmp/sslkeylogfile

Finally, I started Firefox from the console in the same terminal, where I set the environment variable (so it sees the environment variable). Make sure Firefox isn't already running.

% open /Application/Firefox.app

Next, I went to a few random websites (Google, CNN, isc.sans.edu, sans.edu... ). After I ran out of sites to visit, I closed Firefox and exited tcpdump.

I loaded the packet capture file and the SSL Key Logfile in Wireshark. I used version 3.1.0, which fully supports DoH and HTTP2 (Firefox uses HTTP2 for DoH). I identified the DoH traffic using the simple display filter "dns and tls." The entire DoH traffic was confined to a single connection between my host and mozilla.cloudflare-dns.com (2606:4700::6810:f8f9). Could I have just identified the traffic using this hostname? Sure. In this specific case. But you can run your own DoH server and evade simple blocklists like this.

I filtered all traffic to and from that Cloudflare host. Next, I filtered all port 443 traffic that did not involve this IP to a second file and did some simple statistics. Aside from the session length, I found that the payload length for DoH is somewhat telling. DNS queries and responses are usually a couple of hundred bytes long. HTTPS connections, on the other hand, tend to "fill" the MTU. So there is a graph of the payload size-frequency for DoH and HTTPS:

TLS Without DoH DoH Only

The 3 (4?) spikes in the DoH traffic could be due to the limited sample. But these are typical sizes for DNS payloads. Note how the DoH payload size "clusters" below 5-600 bytes, the legacy DNS reply limit. For the non-DOH traffic, the payload sizes peak close to the MTU (the MTU was 1500 Bytes here).

In short: if you see long-lasting TLS connections, with payloads that rarely exceed a kByte, you probably got a DoH connection. But I need to run more tests to verify that. Feel free to do your own experiments and see what you find. Of course, some of these artifacts may be implementation-specific. The RFC somewhat suggests the extended session length. But in other implementations (earlier Firefox versions?), I seem to remember shorter TLS sessions for DoH.

So please let me know what you find, and I will likely update this some time tomorrow if I find time to look at more traffic.

---
Johannes B. Ullrich, Ph.D., Dean of Research, SANS Technology Institute
Twitter|

Keywords:
3 comment(s)
My next class:

Comments

EDNS(0) Padding should make it more difficult to do this kind of analysis *if* the padding size is carefully choosen, https://tools.ietf.org/html/rfc7830

Mozilla Firefox does not seem to support EDNS(0) padding yet, https://bugzilla.mozilla.org/show_bug.cgi?id=1543811

What is also interesting is that the DoH endpoint you tested (Mozilla Cloudflare, https://mozilla.cloudflare-dns.com/dns-query) responds with EDNS(0) padding even if the client does not support EDNS(0). This is a bug, see RFC6891 section 7. Transport Considerations, https://tools.ietf.org/html/rfc6891#section-7:

Lack of presence of an OPT record in a request MUST be taken as an
indication that the requestor does not implement any part of this
specification and that the responder MUST NOT include an OPT record
in its response.

One can test this with sdig from powerdns authoritative nameserver v4.2

# DoH query w/o EDNS

sdig https://mozilla.cloudflare-dns.com/dns-query 443 switch.ch A recurse
Reply to question for qname='switch.ch.', qtype=A
Rcode: 0 (No Error), RD: 1, QR: 1, TC: 0, AA: 0, opcode: 0
0 switch.ch. IN A 72 130.59.31.80
2 . IN OPT 0 AAwARgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
EDNS Padding size: 70

The EDNS(0) aka OPT RR with the EDNS(0) Padding Option is shown base64 encoded but also printed. The response is padded by Cloudflare with 70 bytes.

One can also decode the EDNS(0) Padding option manually:

echo AAwARgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | base64 -D | xxd
00000000: 000c 0046 0000 0000 0000 0000 0000 0000 ...F............
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 ........

c = 12 (Option Code)
46 = 70 (Option Lenth)
0000... Padding
See also https://tools.ietf.org/html/rfc7830#section-3.

I'm not sure if Mozilla Firefox supports EDNS(0) in requests. RFC7830 allows to respond with a EDNS(0) padded response in the presence of EDNS(0) support in the request (https://tools.ietf.org/html/rfc7830#section-4). However, as the test above shows, this is irrelevant as Mozilla Cloudflare seems to respond with EDNS(0) padding even w/o EDNS(0) support.

Daniel
Thanks for the response. Just working on a followup and this is helpful. Interestingly, I don't get any response from Cloudflare if I add padding (but I do get the same padding that you observed if I send a query without padding). Odd.
Previous post didn't display. Forgive me if this is a double post.

A couple of us recently made some edits to a fork of suricata so that the code that initializes the outgoing TCP handshake and the DNS parsing code are more tightly coupled. I call it the "careful synner" This is brand new. It doesn't block anything, just detects by printing to stdout. By coupling the TCP SYN with the DNS parsing code any traffic that is DNS-less is simple to detect / block.

https://www.reddit.com/r/sysadmin/comments/emc4z0/i_made_a_thing_free_for_additional_pihole_network/
The github link can be found in the reddit post.

Diary Archives