eero and Split-Horizon DNS

Update: As of Septmber 14, 2017 eero has updated all eero networks to enable hairpinning. That means the config I laid out isn't necessary, but maybe this is still useful for some folks.

At this point, I'd guess most folks reading this are familiar with eero and similar distributed mesh home wifi products. For those that aren't, the tl;dr is that in many environments, a single router with a set of radios won't cover a whole home very well, and a distributed system with multiple radios working together is a better solution (read the linked Sweethome overview - it covers this better than I will). I'm a happy eero customer, and I'm not super interested in getting into why eero was a better fit for me than its competitors at the time I made the decision. As I said, overall I've been very happy.

However, I did have to get a little creative to solve a specific problem:

NAT/Egress Hairpinning

I have long run a few internal services on various web-servers running inside and outside my network.

Unfortunately, eero doesn't currently support a feature called Hairpinning, which allows clients inside the network to egress the network and get a response from the public IP of the same network. As a result, I had to access these services by one name externally (eg. [server].alvani.me), and another name internally ([server-hostname].local). This is frustrating since it breaks scripts and alerts that rely on consistent DNS names for services, and it means I need a second set of bookmarks to access services inside versus outside my network, plus the congnitive load of remmebering if I should use this name or that name to access this service. The whole point of DNS is to make this easy on us simple humans.

Split Horizon DNS to the Rescue

Split Horizon is a pretty common implementation in the enterprise world. In general, it's overkill for home networks, since there aren't a lot of DNS needs for most home networks. I really didn't want to, but eventually it became clear that for the short term, it'd be a good idea to run my own internal DNS server which handled request for the *.jehanalvani.com and *.alvani.me domains, while forwarding all other requests to upstream DNS providers. I could run in this configuration until eero eventually adds support for the hairpinning.

I'm writing this to document how I configured split-horizon on a Synology with Synology's DNS Server software. There's no reason this couldn't be adapted fairly easily to BIND or whatever other DNS Server you might want to run.

I started by installing DNS Server on my DS414Play.

Then, in DNS Server, I configured my forwarders.

Forwarders

Oh, if you leave the "Limit Source IP service" box checked, make sure you add your internal network CIDR to the list of permitted IPs. Otheriwise you'll get "Forbidden" on your DNS requests.

I added the zones I care about

Zones

A Quick Aside About DNS Zones

If you're familar with DNS, this is not a section you'll need to read, but I got some feedback that the concept of DNS Zones are confusing. For our purposes, DNS Zones are any domain(s) or subdomain(s) you want your nameserver to respond for. Of course, zones are much more powerful, and can be used in a lot of ways. The bottom line is that Zones are a method of drawing a demarcation for the responsibilities of different nameservers.

Moving on:

Finally, inside the zones, I configured the subdomains and services I care about

Zone config

This matches the public DNS configuration of the same services, just replacing the IP address with the internal IP address of the server running the service. In my case, they're all the same, because I'm using a reverse proxy to interpret the requests and send them to the appropriate servers.

Now, any client that sends requests to this name server should recieve the internal record as an answer. I can test with a couple simple NSLOOKUPs from inside the network.

First, query a common public nameserver:

nslookup syno.jehanalvani.com 4.2.2.2

Which returns

>jehan.alvani@MBP $ nslookup syno.jehanalvani.com 4.2.2.2
Server:        4.2.2.2
Address:    4.2.2.2#53

Non-authoritative answer:
syno.jehanalvani.com    canonical name = dynamic.jehanalvani.com.
dynamic.jehanalvani.com    canonical name = starman.synology.me.
Name:    starman.synology.me
Address: 50.35.122.71

Where 4.2.2.2 is the nameserver that was queried, and the results of the non-autoritative answer are exactly what I've configured for my pubilc subdomain. (syno.jehanalvani.com is a CNAME to dynamic.jehanalvani.com, which is itself a CNAME to synology's dynamic DNS service). Ineffecient? Maybe a little. But I don't have to rememebr anything other than "dynamic.jehanalvani.com" after initial setup.

And a query against my Synology's private IP returns the following

`nslookup syno.jehanalvani.com 10.0.1.20`

>jehan.alvani@MBP $ nslookup syno.jehanalvani.com 10.0.1.20
Server:        10.0.1.20
Address:    10.0.1.20#53

syno.jehanalvani.com    canonical name = dynamic.jehanalvani.com.
Name:    dynamic.jehanalvani.com
Address: 10.0.1.20

Boom. That's what I want to see.

DHCP configuration

The next step is to tell the clients inside the network to use this nameserver. Clients inside the network needs to know to send requests to my nameserver. If you're configuring for your network, you might prefer not to set DHCP name servers to the your internal server, instead electing to give manual DNS server to specific clients. I wanted to be able to use any device on the network to reach these URIs, so I opted to configure eero's DHCP server to direct all clients to my internal nameserver.

DHCP Config

Caveat

On a reddit thread, someone asked if this configuration works with eero Plus. The short answer is: yes! The long answer is: yes, but with some qualifications.

eero hasn't published much about how eero Plus works. However, it seems to hijack outbound DNS requests, sending the request to a nameserver eero controls. It's important to note that this only affects outbound DNS requests, so requests that say inside your network still work, as long as there is a responder to accept the requests. In the configuration above, that means that any records in the defined zones will be successful, but anything that needs to be forwarded to an upstream server will be hijacked by eero Plus.

This breaks services like Unlocator and Unblock-Us which rely on their own DNS servers receiving all requests, and interpreting specific requests. I had to disable eero Plus for the time being, since I make use of these services (as you can tell if you're familiar with the nameservers I configured as forwarders).

Is it worth it?

Obviously, that depends on your needs and your level of comfort. I have been happy with this setup for the year+ that I've been using it. However, as soon as eero adds support for NAT hairpinning, I'll happily pull it out. This does introduce a bit of fragility I don't care for, mostly in that it forces my Synology to be powered on for anyone on my network to be able to reach any internet resources.

The Sad State of MLB's Blackout Policy

Thus, as long as I am standing in Carlisle, PA, I can’t watch Phillies broadcasts. However, if I go a half hour to the west in Chambersburg, PA, Phillies games are not shown on television, but also are not blacked out in the packages. If I go a half hour to the east in Harrisburg, PA, Phillies games are blacked out in the packages, but shown on local television. Therefore, this tiny strip of land in Central PA is the only spot in the entire country that one can not legally watch Phillies games. It is purgatory for a Phillies fan.

It's completely ridiculous that these restrictions are in place. If the MLB was trying to proctect the local affiliates, they would show the local affiliate stream with the ads intact, and give revenue back to the affiliates. The current state just hurts customers.

If only there was some way to get around MLB's ridiculous blackout policy.

Selective use of Unblock-Us.com with my Very Own DNS Server

I’m using a service called Unblock-Us to specific domains for me. The service is really excellent; a DNS-based service that (I assume) works by accepting DNS requests on their service, they proxying the request and all responses through their network. I say “I assume” because when I emailed Unblock-Us for confirmation, they wouldn’t confirm or deny my assumptions. I guess they didn’t want to give up the recipe to their secret sauce. Can’t blame them.

Now, while Unblock-Us is DNS-based, I’m not too cool with the idea of sending all of my DNS requests across the internet. I cooked up a little modification to my caching DNS server that sends the domains I specify to Unblock-Us, forwarding other requests to public DNS servers the first time, then just serving them up locally. Here’s how I did that.

  1. First things first, I signed up for Unblock-Us[1], and I activated it.

  2. I created a fresh SD card for my Raspberry Pi. You could run this on any Mac or pretty much any Linux distro. I’m sure you could make it work on Windows, though I have no idea how. There are plenty of reasons to use something more powerful than a Raspberry Pi, but I don’t care about them for the time being. The Pi is fine for me.

  3. I got the Raspberry Pi online and gave it a static IP on my network.

  4. Installed BIND 9, a great and really widely-used DNS server.
    sudo apt-get install bind9 on Debian (or Raspbian or Ubuntu) systems.

  5. Modified my configuration files, by adding the following lines to the listed files:

    1. /etc/bind/named.conf.options
      This specifies the DNS servers that my BIND server will forward requests to when it doesn’t already know how to handle them. It’ll take all answers from these guys and cache them until the TTL expires, so it can handle future requests without going out to the internet.

       forwarders {
              4.2.2.2;
              8.8.8.8;
        };
      
    2. /etc/bind/named.conf.local
      This defines the zones for specific domains that will just be forwarded to Unblock-Us’s DNS servers.

      ######
       # Conditional Forwarding Zones: These zones forward their DNS requests as specified
       ######
      
       Zone "unblock-us.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
       Zone "domain1.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };}; 
       Zone "domain2.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
       ⋯
       Zone "domainN.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
      

      The first line, above, sends all requests for unblock-us.com to the Unblock-Us DNS servers (primary, secondary, and tertiary in order). The other lines, I populate with any other domains I’d like to send to Unblock-Us, just by replacing “domain1.com”, “domain2.com” … “domainN.com”. For example, if I wanted to send DNS requests for Google, Netflix, and Apple to Unblock-Us, my file would contain the following lines:

      ######
       # Conditional Forwarding Zones: These zones forward their DNS requests as specified
       ######
      
       Zone "unblock-us.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
       Zone "google.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };}; 
       Zone "netflix.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
       Zone "apple.com" { type forward; forward only; forwarders { 208.122.23.22; 208.122.23.23; 184.106.242.193; };};
      

      It’s worth noting that Unblock-Us doesn’t support Google or Apple, so while they will properly handle the DNS request, they will not provide any additional benefit. I was just providing them as a configuration example. Netflix is a supported site, and a full list of supported sites can be found here.

  6. Finally, I updated DHCP settings on my router[2] to point to my BIND server as the primary DNS server, and public DNS[3] as the secondary DNS server. As my devices DHCP leases came to expire, they’d check in with the router, and the router would hand them a new lease with the updated DNS settings.

I’m sticking this here because I thought some of you might find it helpful. This isn’t a solution for those who are less than technically inclined. To be honest, I don’t know enough about BIND to really troubleshoot it, but there’s tons of helpful documentation online. If I learn anything significant, though, I’ll post more about it.

Update August 13, 2014: I've been meaning to update this post for a while. Toward the end of last baseball season, this configuration stopped working. I reached out to Unblock Us about it, and they weren't able to give me much direction, except that the domains that MLB.tv, etc that need to be redirected to Unblock Us change frequently. They'd only support a configuration where all DNS traffic was hitting their DNS servers. So, I reverted to using Unblock-Us DNS on my Airport Extreme, and being done with it.

I suppose it would probably be possible to sniff the outbound DNS request being made by your computer when accessing the services, and redirecting those domains, but my fear is that it will be tedious to maintain as content providers switch CDNs, etc.

Update October 24, 2014: An interesting comment from Nick shed a little more light on capturing Netflix traffic. Worth reading if this setup is still something you'd like to do. Please do read.


  1. An affiliate link. If you sign up with this, I’ll get a little kickback. If you don’t want to use my affiliate link, here’s a non-affiliate link.  ↩

  2. Or DHCP server, if you run a seperate DHCP server. I’m running it all from my AirPort Extreme, though.  ↩

  3. I’m using 4.2.2.2, just like the forwarder on my BIND server.  ↩