Updating rsync on OS X

I recently needed to move a few directorys of tens- or hundreds-of-thousands of files to my Synology. Perfect use for rsync!

Unfortunately, rsync on OS X is stuck at 2.6.9, and I wanted to take advantage of some of the new features of 3.1.0. Specifically better handling of OS X metadata, and progress indication.

Fortunately, this walks you through a quick build and installation of rsync 3.1.0 in your /usr/local/bin folder. The benefit is that you can install your new version alongside the OS X included version (which is installed in /usr/bin/). Then you can add a couple aliases to your .bash_profile to treat them appropriately.

My aliases are as follows:

alias oldrsync="/usr/bin/rsync"
 alias rsync="/usr/local/bin/rsync"
 alias nrsync="/usr/local/bin/rsync -a  --info=progress2"

The first makes the system-installed 2.6.9 version refrencable by using the command oldrsync. The second makes the version in /usr/local/bin/ (which is 3.1.0, in my case) the one that runs when I type rsync. Finally, the third references the new version of rsync with a couple flags I almost always use.

Source: http://selfsuperinit.com/2014/01/04/an-upd...

Unblock-Us and Netflix Update

Nick wrote in with a good note regarding my Unblock-Us + BIND setup:

I noticed after setting up the netflix.com zone that unblock.us resolved most of the Netflix addresses to a CNAME, e.g.:

 secure.netflix.com.   86400   IN  CNAME   secure-1848156627.us-west-9.elb.amazonaws.com.

My ISP’s DNS server did not know about the address secure-1848156627.us-west-9.elb.amazonaws.com, but the unblock.us DNS server resolved it successfully. So I just added another zone for amazonaws.com, and forwarded those requests to unblock.us. That seems to have resolved it - Netflix now works. Not ideal, since the rule is a bit general, but I’m happy to have it working.

Good investigation, and little things like this may resolve some of the issues I was seeing with this setup, last year. I don't have the patience to keep up with it, but I'm certain some of you are more patient people than I am.

Selective use of Unblock-Us.com with my Very Own DNS Server

I’m using a service called Unblock-Us to specific domains for me. The service is really excellent; a DNS-based service that (I assume) works by accepting DNS requests on their service, they proxying the request and all responses through their network. I say “I assume” because when I emailed Unblock-Us for confirmation, they wouldn’t confirm or deny my assumptions. I guess they didn’t want to give up the recipe to their secret sauce. Can’t blame them.

Now, while Unblock-Us is DNS-based, I’m not too cool with the idea of sending all of my DNS requests across the internet. I cooked up a little modification to my caching DNS server that sends the domains I specify to Unblock-Us, forwarding other requests to public DNS servers the first time, then just serving them up locally. Here’s how I did that.

  1. First things first, I signed up for Unblock-Us[1], and I activated it.

  2. I created a fresh SD card for my Raspberry Pi. You could run this on any Mac or pretty much any Linux distro. I’m sure you could make it work on Windows, though I have no idea how. There are plenty of reasons to use something more powerful than a Raspberry Pi, but I don’t care about them for the time being. The Pi is fine for me.

  3. I got the Raspberry Pi online and gave it a static IP on my network.

  4. Installed BIND 9, a great and really widely-used DNS server.
    sudo apt-get install bind9 on Debian (or Raspbian or Ubuntu) systems.

  5. Modified my configuration files, by adding the following lines to the listed files:

    1. /etc/bind/named.conf.options
      This specifies the DNS servers that my BIND server will forward requests to when it doesn’t already know how to handle them. It’ll take all answers from these guys and cache them until the TTL expires, so it can handle future requests without going out to the internet.

       forwarders {
    2. /etc/bind/named.conf.local
      This defines the zones for specific domains that will just be forwarded to Unblock-Us’s DNS servers.

       # Conditional Forwarding Zones: These zones forward their DNS requests as specified
       Zone "unblock-us.com" { type forward; forward only; forwarders {;;; };};
       Zone "domain1.com" { type forward; forward only; forwarders {;;; };}; 
       Zone "domain2.com" { type forward; forward only; forwarders {;;; };};
       Zone "domainN.com" { type forward; forward only; forwarders {;;; };};

      The first line, above, sends all requests for unblock-us.com to the Unblock-Us DNS servers (primary, secondary, and tertiary in order). The other lines, I populate with any other domains I’d like to send to Unblock-Us, just by replacing “domain1.com”, “domain2.com” … “domainN.com”. For example, if I wanted to send DNS requests for Google, Netflix, and Apple to Unblock-Us, my file would contain the following lines:

       # Conditional Forwarding Zones: These zones forward their DNS requests as specified
       Zone "unblock-us.com" { type forward; forward only; forwarders {;;; };};
       Zone "google.com" { type forward; forward only; forwarders {;;; };}; 
       Zone "netflix.com" { type forward; forward only; forwarders {;;; };};
       Zone "apple.com" { type forward; forward only; forwarders {;;; };};

      It’s worth noting that Unblock-Us doesn’t support Google or Apple, so while they will properly handle the DNS request, they will not provide any additional benefit. I was just providing them as a configuration example. Netflix is a supported site, and a full list of supported sites can be found here.

  6. Finally, I updated DHCP settings on my router[2] to point to my BIND server as the primary DNS server, and public DNS[3] as the secondary DNS server. As my devices DHCP leases came to expire, they’d check in with the router, and the router would hand them a new lease with the updated DNS settings.

I’m sticking this here because I thought some of you might find it helpful. This isn’t a solution for those who are less than technically inclined. To be honest, I don’t know enough about BIND to really troubleshoot it, but there’s tons of helpful documentation online. If I learn anything significant, though, I’ll post more about it.

Update August 13, 2014: I've been meaning to update this post for a while. Toward the end of last baseball season, this configuration stopped working. I reached out to Unblock Us about it, and they weren't able to give me much direction, except that the domains that MLB.tv, etc that need to be redirected to Unblock Us change frequently. They'd only support a configuration where all DNS traffic was hitting their DNS servers. So, I reverted to using Unblock-Us DNS on my Airport Extreme, and being done with it.

I suppose it would probably be possible to sniff the outbound DNS request being made by your computer when accessing the services, and redirecting those domains, but my fear is that it will be tedious to maintain as content providers switch CDNs, etc.

Update October 24, 2014: An interesting comment from Nick shed a little more light on capturing Netflix traffic. Worth reading if this setup is still something you'd like to do. Please do read.

  1. An affiliate link. If you sign up with this, I’ll get a little kickback. If you don’t want to use my affiliate link, here’s a non-affiliate link.  ↩

  2. Or DHCP server, if you run a seperate DHCP server. I’m running it all from my AirPort Extreme, though.  ↩

  3. I’m using, just like the forwarder on my BIND server.  ↩