Category: How To

Pi-Hole – a Black Hole for Advertisements

Pi-hole was first released back in 2015, I’m not certain when I became aware of it but given my interest in the Raspberry Pi I’m pretty sure I heard about it fairly soon afterwards. I did find this tweet from 2016

Now while I was aware of the project, I didn’t start running it for a while. It was only at some point during my containerization of my server that I started to run pi-hole in a container (Oct 2018 give or take a bit)

Running it as a container isn’t too hard – but you’ll probably have to turn off the DNS server than is running already to avoid the port conflict.

Here is the Makefile I was using to manage my pi-hole deployment.

Unfortunately – something happened to my configuration / state – such that I could not update my container without it hanging. Fortunately having the rollback target let me quickly restore the previous version. I’ve tested the makefile on another temporary machine and it appeared to work, so it should be a reasonable base if you wanted to go the container route.

One of the problems of running in a container is the networking in general. I struggled with the mapping of the web UI access as the same machine is also running my public facing web server. While I could map the DNS port (53) and access it over IPv6 – all of the IPv6 traffic appeared as if it were coming from the docker network vs. from the source machines.

This takes away from one of the great values of running pi-hole – the additional insight it gives you to what your various devices are doing on the network. With the docker networking mess, I was missing all of the IPv6 traffic effectively (because I couldn’t tell the devices apart).

After stalling on the decision, and some explorations on how I could use macvlan support in docker to give a container a unique (from the host) IP address – I just bought some nice hardware to solve the problem. Setting that hardware up is covered in the previous post.

Now we can install pi-hole. I would encourage you to read the script before just piping it into bash, however in the big picture we’re going to trust the folks that wrote this code to also provide updates – and those updates could be evil too.

The script is interactive, you’ll need to answer some questions to perform the install. I found it interesting that the setup script doesn’t ask for IPv6 DNS severs, but does allow you to specify customer IPv4 servers. During the setup it looks like it is changing my network setup to be a static IP address. Post install script I know I’m going to have to tweak things.

Since the default web password is generated, you probably want to set one.

Visiting the web interface under “Settings->DNS” I added my upstream IPv6 DNS servers. I’m using the CIRA DNS and if you’re a Canadian I would encourage you to do the same.

On the same settings page I enabled conditional forwarding and specified my local lan range and main router which is running my DHCP server. It was pointed out to me that additional configuration is required for IPv6 conditional forwarding, I haven’t done this yet.

My OpenWRT router provides multiple IPv6 addresses and the setup script detected IPv6 address isn’t the right one.  Poking around, it appears /etc/pihole/setupVars.conf contains the information and I just need to tweak it. Generally you should not change that file by hand, but I did for this one thing and it fixed the problem.

As I feared, the setup script changed my /etc/dhcpcd.conf to reflect a static IP address. I may later change this but I had already effectively tweaked the DHCP server to answer the same static address.

At this point – I have a working pi-hole, I just need to configure some clients to point there.

As mentioned above, I run OpenWRT as my router.  There are 2 places we need to configure to point all DNS queries to the pi-hole. This can be done by modifying how it responds to DHCP requests – as it will provide the DNS server as part of that transaction.

An alternative approach to this would be to set your upstream DNS server to be the pi-hole. I didn’t take this approach because I was concerned about DNS loops and networking was a lot more complicated when things were in a container, the approach I’ll cover is what worked with the container version as well.

Changing the DNS entry that is provided by the DHCP exchange is easy to find in the config file /etc/config/dhcp file – there are two lines in a section that looks like:

Finding the place in the LuCI UI to add these always causes me to stumble around for a while. The two options list dhcp_option and list dns are in slightly different places.

The IPv4 setting can be found under Network->Interfaces, edit your Lan interface. Then pick the Advanced tab. We need to add a dhcp option 6,149.112.121.30.

Then select the IPv6 Settings tab. Here we add to the Announced DNS Servers section 2620:10A:80BB::30.

Once you’ve done this your pi-hole will start getting traffic from devices that get an address on your network. You may have to wait for the devices to update their connections.

I noticed that IPv6 addresses were not reverse mapping – but specifically asking my router for the bad addresses seems to indicate that it also can’t reverse map, so maybe there is an OpenWRT problem here. Also – it seemed to get better after a while, and more address->name mappings were discovered. I asked in the pi-hole forum about this behaviour.

It turns out that this is an ordering problem. Pi-hole won’t look up a failed address again, but it does build the network table and bind things together by MAC address. The work around is to modify your /etc/pihole/pihole-FTL.conf to have REFRESH_HOSTNAMES=ALL. There is a slight downside to this that every hour there will be a storm of reverse DNS lookups as all hosts are refreshed.

A few final observations.

  • The magic DNS name pi.hole now works on my network. This brings you directly to the pi-hole dashboard.
  • Tools->Network shows lots more useful information. In docker you didn’t get MAC addresses and generally things were more chaotic.
  • Pi-hole is blocking more than 1/3 of the DNS lookups. Sure some of this is because the ad-software is probably failing and trying again, but still that’s a lot of DNS queries.
  • I discovered the Group Management feature, and this seems to be a way to allow clients to opt out of ad blocking. This is super useful as previously I was just changing the DNS on the clients.

 

Raspberry Pi Ubuntu Server

Ever since the launch of the Raspberry Pi I’ve been a fan. I’ve bought and been gifted many of them over time. It reminds me a little of the NSLU2 (slug), but builds on the amazing hardware advancements driven by smartphones.

I recently bought my first RPi4 – the base model 2Gb version. BuyAPi.ca is local, but I had it shipped. Great prices, and my order was prepped and shipped within 3hrs of submitting it. Even via regular mail, it showed up quickly (days).

What I bought:

I have plenty of micro-sd cards around so there was no need to get another.

The Pi ships in a very cute little box.

The first thing I did was to install the heatsinks. I had to visit the web page to figure out where to put them on the board. Peeling the plastic off the adhesive tape was a little tricky, but a sharp knife blade helped me get under the edge.

There are several choices for the OS to run, and for many of my previous projects I’ve stuck with Raspbian (now known as the Raspberry Pi OS). Since my intended use for this Pi is to run it as a server hosting Pi-hole, I opted to go with Ubuntu server.

Downloading and flashing the image file to the sd-card was straight forward. I connected the Rpi4 to wired ethernet and power, and booted right after the card was flashed. By visiting my main router I could see the DHCP address that the Pi had been given.

By default – the device shows up as hostname ‘ubuntu’ and the default user is ‘ubuntu’. You can ssh directly to the machine, and on first login you are forced to change the password. The password policy requires non-trivial passwords, so it’s not a bad solution to getting going.

At this point I no longer need to access the device physically (or at least very often). I drilled some holes in a bit of wood to accept the brass stand-offs, then mounted the Pi to the stand offs. This let me mount the whole thing to the wall along side my other infrastructure bits (router, modem, voip box, switch).

Based on my server configuration post – I want the new Rpi4 server to be similarly set up.

Stuff I want to do:

  1. Change the username
  2. Change the hostname
  3. Configure automatic updates
  4. Forward email
  5. Fix timezone
  6. Redirect logs
  7. Install Prometheus monitoring

The rest of this post is the details on those steps.

Continue reading “Raspberry Pi Ubuntu Server”

Correct Playlist – Segment Map for Bluray

Before streaming was the primary way to get new content, I collected a lot of movies on DVD / bluray. I still have a large collection, but have been slowly converting it to be hosted on my Plex server. This gives me a Netflix like viewing experience, but for my own personal movie collection.

Handbrake is great for dealing with DVDs. MakeMKV is how I rip bluray disks, then feed the resulting rip into Handbrake to compress it down. I do all of this on my Ubuntu system.

Generally this works really well. Handbrake will automatically select the longest video, and that’s almost always the actual movie itself. With MakeMKV that selection is more manual, but picking 1 title from a list of 10 isn’t all that hard.

When I got to the Hunger Games series, things got a bit more interesting. Starting with Catching Fire the bluray shows you 100’s of feature length titles – all the same duration, but with different segment maps. It seems that all but one of these titles has things shuffled in the wrong order.

Initially I naively picked the 1st and used that one, but upon watching the movie it was obvious there was 1 scene out of place, and thus two weird jump cuts to the flow of the movie. Enough to be annoying.

If you got digging around, you can find advice on which of the many to pick from the list. It turns out that there are multiple versions of the movie: rental, US release, Canadian release, etc. It’s much better to figure it out for the disk you have. The MakeMKV forum has a post about using PowerDVD and Process monitor to figure this out. I struck out here as I didn’t have a Windows machine with the right software combination, and it seems I wasn’t able to get Ubuntu to natively play back a bluray either.

Recently I came across a way to use MakeMKV to do the full process, again thanks to a MakeMKV forum post.

    1. Use MakeMKV to back up the full disk.

    2. Use MakeMKVcon to dump info to a text file

    3. Isolate the segment lists from /tmp/xx.txt. It turns out that “,26,” is unique enough to grab all of the segment lists. For Mockingjay Part 1 there are 550 segment lists on the disk, 519 of these are the length of the movie.

    4. Observations:
      a) All the movie length lists start with the same segment: 519
      b) It seems they all end with 520
      c) There are only 20 chunks in each segment list, and we already know 2 of them. Only 18 to sort into order correctly
      d) The MakeMVK backup has all of the chunks in backup/<disk>/BDMV/STREAM/
    5. Now we just need to play a copy of the movie, I have the DVD as well so VLC can play that back for me. Start at the first chunk, verify it is the start, watch the end to determine the scene break. Then figure out what the next chunk is.
      By building an incrementally specific grep, I can figure out the next chunk options. It is fewer choices than you might imagine. Each one had 2-4 possibilities.
      Hint: as you identify chunks, record the duration – this helps figure out where on the DVD playback you need to review to find the scene break.
    6. Once we identify the correct chunk order – we can go back to MakeMKV and rip the correct stream. A web search can also help verify which one is the right one, as I did for Mockingjay Part 1.

It took me about 35mins to get through step 5, much shorter than watching the whole movie. During the course of the chunk identification, I came across 3 where I had no choice, the only next chunk was the same one. After walking through 12 chunks, I hit a point where there was only 1 segment list left. I quickly verified the segment end/start matches and then double checked against the web search.