Recently I wrote a post around the UK IP Bill and speculated how ISPs may implement the most basic requirement of the bill, to keep a list of every site each subscriber had visited. The simplest and most complete method I speculated around for doing this was inspecting DNS traffic passing over the ISP’s routers on port 53, DNS is a very old protocol and is plain text so snooping on which domains each user had visited would be as easy as running a mass tcpdump on port 53 with meta data extracting magic. Anyway, this post covers a simple proxy you can run at home to stop your DNS traffic going out over port 53 as plain text and for it to travel over HTTP with SSL encryption.
As you well know if you run tcpdump on port 53 today on your machine and you make any DNS lookup you are in for a treat, the full conversation in plain text in front of you. Check out what loading my blog looks like in terms of DNS traffic, its a treasure trove of hostnames.:
MRF28PG8WN:https_dns_proxy robe8437$ sudo tcpdump -i en0 port 53 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes 21:52:36.278484 IP mrf28pg8wn.connect.64494 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 17578+ A? robertputt.co.uk. (45) 21:52:36.323561 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.64494: 17578 1/0/0 A 184.108.40.206 (61) 21:52:40.908298 IP mrf28pg8wn.connect.60815 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 52589+ A? www-google-analytics.l.google.com. (51) 21:52:40.913541 IP mrf28pg8wn.connect.59876 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 49807+ A? pagead2.googlesyndication.com. (47) 21:52:41.091339 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.60815: 52589 1/0/0 A 220.127.116.11 (67) 21:52:41.465906 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.59876: 49807 2/0/0 CNAME pagead46.l.doubleclick.net., A 18.104.22.168 (103) 21:52:42.537092 IP mrf28pg8wn.connect.49631 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 21419+ A? www.google.com. (32) 21:52:42.537195 IP mrf28pg8wn.connect.54188 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 48472+ A? pagead.l.doubleclick.net. (42) 21:52:42.789511 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.54188: 48472 1/0/0 A 22.214.171.124 (58) 21:52:43.485905 IP mrf28pg8wn.connect.53642 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 36701+ A? pagead-googlehosted.l.google.com. (50) 21:52:43.611953 IP mrf28pg8wn.connect.49631 > fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain: 21419+ A? www.google.com. (32) 21:52:43.643998 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.53642: 36701 1/0/0 A 126.96.36.199 (66) 21:52:43.769184 IP fwdr-8.fwdr-8.fwdr-8.fwdr-8.domain > mrf28pg8wn.connect.49631: 21419 1/0/0 A 188.8.131.52 (48)
From this we can clearly see my I visited the domain robertputt.co.uk plus a load of ad traffic and Google Analytics. This would easily allow ISPs to complete one of their responsibilites of the IP Bill, to record every website which a user visits, note this says website not webpage so domain / subdomain / hostname is sufficient here.
So you are probably thinking; this is DNS it’s a core bit of how the internet works, you can’t do anything to change that. Well you may be suprised, Google has launched a new variation of it’s public DNS product called DNS-over-HTTPS, you can check out the docs for it here. This service essentially allows you to do DNS lookups over a HTTPS session which as wel all know is encrypted and not suseptable to the tcpdump MITM seen above, however there is a big issue, you cannot configure your machine to do DNS over HTTPS, most machines network configuration only allows talking to a traditional DNS server on port 53 in plain text using the standard protocol.
Do not dispair, there is a way you can use DNS over HTTPS today, although its a little ugly to get setup. The technique involves running a proxy locally which takes requests like a normal DNS server using the standard protocol on port 53 in plain text, it then reaches out to Google DNS-over-HTTPS gets the result and responds to the client in the traditional manor. I wrote a small proxy for this purpose in Python using dnslib and requests, you can fetch it here – Py-DNS-over-HTTPS-Proxy. It’s not a very nice script but it does the job, feel free to raise a PR if you have improvements :-).
So how do I get this thing working, first create a virtualenv, install the requirements and checkout the script...:
MRF28PG8WN:envs robe8437$ virtualenv dns_proxy Using base prefix '/Library/Frameworks/Python.framework/Versions/3.5' New python executable in /Users/robe8437/Python/envs/dns_proxy/bin/python3.5 Also creating executable in /Users/robe8437/Python/envs/dns_proxy/bin/python Installing setuptools, pip, wheel...done. (exequor_api) MRF28PG8WN:envs robe8437$ cd dns_proxy/ (exequor_api) MRF28PG8WN:dns_proxy robe8437$ source bin/activate (dns_proxy) MRF28PG8WN:dns_proxy robe8437$ pip install dnslib requests Collecting dnslib Using cached dnslib-0.9.6.tar.gz Collecting requests Using cached requests-2.12.4-py2.py3-none-any.whl Building wheels for collected packages: dnslib Running setup.py bdist_wheel for dnslib ... done Stored in directory: /Users/robe8437/Library/Caches/pip/wheels/f4/3d/d1/b941767759a29d9a8df99b00c6f4204aeb6e5f12429f9e2e4e Successfully built dnslib Installing collected packages: dnslib, requests Successfully installed dnslib-0.9.6 requests-2.12.4 (dns_proxy) MRF28PG8WN:dns_proxy robe8437$ git clone https://github.com/robputt796/Py-DNS-over-HTTPS-Proxy.git Cloning into 'Py-DNS-over-HTTPS-Proxy'... remote: Counting objects: 32, done. remote: Compressing objects: 100% (18/18), done. remote: Total 32 (delta 13), reused 23 (delta 7), pack-reused 0 Unpacking objects: 100% (32/32), done. Checking connectivity... done. (dns_proxy) MRF28PG8WN:dns_proxy robe8437$
Now lets run the proxy and test it out… by default it runs as a non-privileged user on port 8053. First I start the proxy Python script...:
(dns_proxy) MRF28PG8WN:dns_proxy robe8437$ python Py-DNS-over-HTTPS-Proxy/https_dns_proxy/__init__.py
and in a new tab run tcpdump against port 53 on my network device...:
sudo tcpdump -i en0 port 53
and then in a third tab I run my DNS query to the proxy listening on the loopback device...
MRF28PG8WN:~ robe8437$ dig @localhost -p8053 A robertputt.co.uk
; @localhost -p8053 A robertputt.co.uk ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; opcode: QUERY, status: NOERROR, id: 65000 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION: ;robertputt.co.uk. IN A
;; ANSWER SECTION: robertputt.co.uk. 14399 IN A 184.108.40.206
;; Query time: 184 msec ;; SERVER: 127.0.0.1#8053(127.0.0.1) ;; WHEN: Fri Jan 6 22:11:23 2017 ;; MSG SIZE rcvd: 50
This time we see no traffic for the DNS query in the tcpdump as the request has been sent via the proxy over HTTPS.
So how can we use the proxy to actually serve DNS requests for the system? To test the proxy as the system’s resolver I quit the instance of the proxy I was running as my own user, escalated to root and edited the script to change the listening port from 8053 to 53, I then executed the proxy as root… Now obviously in the real world you would never do this, I simply did this to test the theory in reality you should use authbind or something similar to run the process under a standard user account. Next I tested the proxy using the dig command in a seperate tab...:
MRF28PG8WN:~ robe8437$ dig @localhost google.com ; @localhost google.com ; (2 servers found) ;; global options: +cmd ;; Got answer: ;; opcode: QUERY, status: NOERROR, id: 1391 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;google.com. IN A ;; ANSWER SECTION: google.com. 299 IN A 220.127.116.11 ;; Query time: 184 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Fri Jan 6 22:41:03 2017 ;; MSG SIZE rcvd: 44 MRF28PG8WN:~ robe8437$
As it appears to be working I edited my host’s DNS configuration to use the loopback address (127.0.0.1) as the resolver.
Now time to see if I can browse the web using the DNS-over-HTTPS proxy. Wohoo it seemed to load the BBC website just fine and we can see the DNS requests for loading the page in the logging in the stdout of the proxy script...
The proof is in the pudding...
Now, to test that this actually silences all DNS traffic on port 53 and that all our DNS requests now leave our machine over encrypted HTTPS, whilst the proxy is up and running and your system is pointed at localhost for DNS resolution run tcpdump again listening on port 53 and try surfing a few websites...:
MRF28PG8WN:Py-DNS-over-HTTPS-Proxy robe8437$ sudo tcpdump -i en0 port 53 Password: tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
Silence! As you can see there is no traffic in plain text over port 53 anymore and all your DNS queries are being made securely against the Google DNS-over-HTTPS webservice. Of course this is in no way configured in a manor which you could use day to day, it is running as root which is uber bad practice, the script is very thrown together and is not very robust, and if the webserver serving the requests goes down there is no failover or roundrobin to another Google DNS-over-HTTPS server, however as a simple experiment to hide your DNS traffic from your ISP it works very well and proves it can be done.
Please feel free to comment your thoughts and as always PRs are welcome to build the script out further if you’d like to add features. Thanks for reading!