Author: Roo

Adventures in 4K – Ripping 4K UHD Blu-Ray

For my birthday a few months ago, I got a copy of The Matrix in 4k. Previously I had only the original DVD that I bought when it first came out. I popped the 4k blu-ray into my blu-ray drive and started up MakeMKV only to discover that my system was unable to read a UHD disc.

Thankfully the 4k blu-ray comes with a normal blu-ray that contains a 1080p copy of the movie and I was able to rip that to my personal collection. While I do have a 4k capable TV, my primary projection setup is still only 1080p so having more bits available isn’t actually a better setup. Still, owning a 4k disc and not being able to use it bugged me.

It turns out the MakeMKV folks run a forum, and there are recommendations there for the right drives to buy in order to rip the 4k discs. There is a thread Ultimate UHD Drives Flashing Guide Updated 2024 which is required reading if you want to get started. I also checked out CanadaComputers which is my local go-to computer store, often having better prices than you can find online.

My pick was the LG WH16NS40 which was both low cost, and appeared to be well supported by the MakeMKV forum. Of course, it isn’t as simple as buy the drive and rip 4k media, you need to modify the firmware. The fact that I had to modify the drive to get it to do what I wanted made this a must have item so it went on my Christmas list. Thankfully I was on the good list and when it was time to unwrap gifts I had my hands on a new drive.

Installing the drive into my Linux machine was pretty straight forward. I ended up replacing another older DVD drive I had in there. On the label of my new drive, I could see the model number (WH6NS40) and manufacture date (June 2024). There was also an indication of the ROM version (1.05).

I run MakeMKV in a container, for me this is a great way to encapsulate the right setup and make it easy to repeat. The new drive showed up just fine to MakeMKV – but I didn’t expect it to support 4k UHD discs just yet.

I will summarize things further down, so you can skip to the summary if you want. However, the bulk of this post will be my discovery process on re-flashing the drive.

Time to head off to the guide and read it carefully.

The first thing I took note of was the correct firmware I wanted based on my drive. This was in the “Recommandation” section near the top.

WH16NS40 on any Firmware directly to > WH16NS60 1.02MK

So I want the 1.02MK version, and it seems I can get there with a single flash vs. needing to do multiple steps.

A bit further down in the same guide, I came across

LG 1.04+ / BU40N 1.03 / Asus 3.10+ and similar
The newer OEM firmwares cannot be flashed easily due to the additional downgrade checks implemented by the drive/firmware manufacturer.

Oh oh. So I may have problems? I am pretty sure I have the 1.05 firmware.

As I read on, it seems the recommended flashing tool is for Windows, and while I have a few Windows systems the drive is installed in a box that only has Linux on it. I spent some time reading through various forum posts and searching for other related material.

At this point I have more confidence that yes, my drive is supported – but it’s a question about how exactly to fix this drive (under Linux) to make it go. Bonus points if I manage to do this all inside of a container.

I did find an older thread that discusses flashing things under Linux. It pointed at a stand alone flashing tool on github, but it was reading through this thread when I discovered that MakeMKV itself contains the sdftool and supports the flashing process. This means I already have the tool inside the MakeMKV container.

Here is how I run the container

For your system you will want to adjust the volume mappings and device mappings to match what is on your host system. This works great for me, and I can access both of the blu-ray drives on my system and write newly ripped files to my host filesystem.

Looking at the browser view of the MakeMKV container I can see that the new drive is recognized, and in the right side panel it even calls out the details for LibreDrive support.

Shelling into the docker container, I can see that sdftool is a link to makemkvcon.

I had read about the possibility of dumping the original firmware as a backup plan in case things go very badly, but it seems this is actually not possible. It seems the manufacturers have made this more complicated in the name of security or something.

I grabbed the “all you need firmware pack” from the guide. This is a very small set of alternative firmwares, only one matches my LG ‘desktop’ sized drive so it was easy to identify the one I wanted to use.

I also needed the SDF.bin file that is hosted on the makemkv site.

In theory, I have all the bits I need. The sdftool, the SDF.bin, and the modified firmware.

At this point, I’m back following the guide. The Mac/Linux portion which walks you through things. I can dump information about the current firmware from my drive

Now I know the existing firmware version, it does not appear to be an exact match to the ones in the list from the guide under “Newer OEM Firmwares and encrypted”. However, the following is a pretty close match:

This drive was made in June 2024 and most probably has a firmware from after 2020 – so a very close match to the list above, and the date of manufacture makes it very likely that my drive has ‘encrypted’ firmware.

Ok – to recap what the plan looks like.

  1. Grab the sdf.bin file
  2. Download the modified firmware(s)
  3. Dump existing firmware versions – determine if you are encrypted or not (likely you are)
  4. Flash the drive

Easy right?

From outside the container we can copy in the firmware we need

And inside the container we can pull down the SDF.bin file.

Then we just need to do the very scary flash part.

There is a very long (minutes) pause where the flashing is taking place.. longer than I can hold my breath.. uh.. did I just make a brick?? fuuuuuu….

I can see from another terminal session that it is eating CPU, pegged at 100%.

After 10+ mins of hanging.. I hesitantly CTRL-C’d the thing..

Thankfully, everything seems ok – I’m exactly where I started. Whew.

I found that adding the verbose (-v) flag was probably a good idea, and a forum thread that indicated that there should be more output from the command. Maybe it’s getting stuck starting up?

I had a few thoughts. Maybe I need to run the container with less restrictions? (docker –privileged) No, that didn’t change anything.

Then I found someone having the same problem recently. It seems the solution they used was to just use Windows. I did ponder how I might setup a temporary Windows install to do this. Then I found this thread that discusses MakeMKV hanging after loading the SDF.bin file, this feels like it may be the same problem. In that case the issue is with the most recent version of MakeMKV (1.17.8).

I started looking for an older version of the container I’ve been using, one that has MakeMKV (1.17.7). It turns out that jlesage/makemkv:v24.07.1 is a few tags back, but has that version. Let’s see if using this version will work better.

This seems to be much better, I’m now getting an error message instead of a 100% CPU hang. Also, apparently I need to remove the disc from the drive which is something I can do.

I only very briefly held my breath as I typed in ‘yes’ and let it continue to do the work. It only took a minute or so to flash the drive and report success.

I needed to re-start the makemkv container. Then it was showing me my drive was good to go

As you can see the LibreDrive information now shows

And I can now read 4k UHD blu-ray discs without problem. I was able to rip the 4k version of the Matrix (53Gb) without issue. My setup was only showing [4x] speed, but I suspect this is more a limitation of my overall system vs. the drive which I suspect can go faster. I’m still very pleased to be able to pull the bits.

Summary – the TL;DR version

Recent versions of the LG WH16NS40 can be modified to read 4k UHD blu-ray discs. This can be accomplished under Linux, using the MakeMKV container.

There is a bug in MakeMKV version 1.17.8 which causes it to hang with 100% CPU. Using version 1.17.7 still works as of the date of this post.

Absolutely read the guide.

Start up the MakeMKV container.

I downloaded the firmware bundle, and picked the matching one for my drive. I then copied it from my host filesystem into the container

Then shell into the container and download the SDF.bin file.

Now we issue the flash command

That’s it. We have a modified firmware installed.  Time to enjoy 4k goodness.

OpenBeken with Globe Electric Smart Plugs

I needed some more home automation friendly outlets, and my preferred Sonoff S31 outlets are hard to get at the moment. It also seems that the S40 has been released as a replacement, but moves away from the ESP micro-controller. This got me looking for what I could get locally.

The Home Depot carries the Defiant series, which when I looked up the details on the FCC website it seemed like a great candidate having an ESP32 inside. However, it seems that the eFuse has been flipped to prevent any further firmware updates. I then moved on and found a sale price of $22.99 for a 3 pack of the Globe Electric Smart Plugs.

Now these plugs do not have an ESP device in them, but a WB2S module. Some folks take advantage of the fact that this is pin compatible with an ESP device and will just swap in a whole new logic board. However, you can also use OpenBeken which I’ve done before. Since these were pretty cheap I picked up 3 boxes (9 plugs).

These plugs are fairly compact and you can fit two in an outlet (top and bottom) if you wanted.

The first thing we need to do is open these. This is fairly easily done using a pocket knife and a little bit of patience / prying. After doing a few of these I got pretty quick, a minute or two to open them. I captured a short video I put up on YouTube showing this process.

Now that it’s open, you can remove the exterior case entirely and you have access to the internals. The micro-controller was easy to spot, it is on it’s own tiny circuit board.

You can see why some folks are simply removing the entire controller board and swapping in a compatible one with a more friendly chip to program. However, with OpenBeken we have the option of reprogramming this chip with some firmware to give us local only control (via Home Assistant).

At the top you can see this is a WB2S board, and the chip is a BK7231TQN32. For the most part this chip is referred to as a BK7231T.

We’re going to need to hook up 4 wires to this to reprogram it, 3.3V, GND, Tx and Rx. Let’s figure out where those pins are.

This is the other side of the board, and if you look carefully you can see at the bottom edge we have starting from the left side BAT, GND, IRX, ITX. These are the 4 pins we need to connect with. Thankfully we can access each of these from the bottom side of the board as the controller board slots into the main circuit board on it’s edge.

Not very pretty, but it gets the job done. The wires are from some stripped ethernet cable and I think they are 26AWG – fairly small. While this was convenient for me to connect some wires to, upon inspection it seems the only reason there is any solder here is to support the board-to-board connection, I’m pretty sure all of the actual interfacing to the circuits is on the other side.

Someone on the Elektroda forum seems to have done the first work on one of these plugs. A lot of people using OpenBeken have used the CloudCutter project to reprogram things without wiring anything up, but this seemed more complicated to me and not guaranteed to work with all devices.

My first attempt I used the same software as I had previously. The only change was to download a different firmware because this is a different module. Unfortunately I goofed somewhere and when flashing the device it failed, then it was no longer responsive to my attempts to connect via serial at all. It may be a brick now, boo.

For my second attempt, I did more reading about the ESPHome support for BK72xx chips. There is quite detailed information about flashing the chips, and it seems to strongly recommend you do not use the old tool I had previously used. There is also a section there on un-bricking things which I’ll have to try later.

Thus I picked the ltchiptool – and installed it under Windows. The easy way to do this on Windows was use the Microsoft Store to install “Python 3.10” – then run pip install ltchiptool[gui]

Once installed, you can launch the GUI with python -m ltchiptool gui

While I may eventually end up using ESPHome, I decided to stick with OpenBeken and grabbed the latest OpenBK7231T_UA_1.17.800.bin from their download page. I first used the ltchiptool to download the existing flash image to confirm my wiring was working. Then I held my breath and uploaded the firmware.. and it worked.

Then it was a simple matter of repeating things. Crack open the case. Solder some wires. Hook it up to ltchiptool, download to confirm things are working, upload new firmware. De-solder the wires. Re-assemble. Now I had 8 outlets reprogrammed.

A dab of crazy glue helped re-seal the exterior cases. So far this seems just fine after a few insert / removals of the plugs.

Next we need to do some configuration. Upon first boot the device will offer up a captive portal. Connect your computer to the “OpenBK76231N_XXXXX” WiFi and visit http://192.168.4.1 in a browser. From here we can pick “Config” then “Configure Wifi & Web” to setup the Wifi connection.

Once the device is on our WiFi network, it can then see the internet which allows us to use the “Web Application” to further configure the device. You can launch the Web Application from the main landing page. I did find that it was important to access the device via IP address (ie: http://192.168.1.56) vs. by name.

Once we launch the Web Application, we can easily configure the device by using the “Config” tab, and searching for the right template from the web. I have to say this is a pretty clever way to do things.

You can see I’ve searched up the “Globe Globe smart plug” and have used “Copy Device Settings” to populate the “Pin Settings”. At the bottom of the page there is a “Save” button to persist the choice. Now if we revisit the Home page we will see the device now offers up control of the power. The button on the side of the plug also now works to toggle power.

The last thing we’ll do is connect this up to Home Assistant. The easy path here is to setup MQTT (Config->Configure MQTT) and get connect to your broker. I was left scratching my head why Home Assistant wouldn’t see these new devices – and after a lot of probing around my MQTT setup and verifying that data was flowing, I finally watched the video the explains it.

Short story was – after you’ve got MQTT setup, and the main page will show you the MQTT State is connected – you need to do one more thing to have it emit a discovery payload to Home Assistant. (Config->Home Assistant Configuration) and hit “Start Home Assistant Discovery”. Then the devices just appeared with their short name in Home Assistant.

Hoarder – a self hosted link collection and web archive

I found out about Hoarder via the self-hosted podcast. While I don’t always agree with the opinions of the hosts, they’ve helped me discover useful things a few times. I’d certainly recommend checking the podcast out.

The Bookmark Everything App

Quickly save links, notes, and images and hoarder will automatically tag them for you using AI for faster retrieval. Built for the data hoarders out there!

The install is docker friendly and based on compose. It’s a very simple 3 steps to get a test instance setup.

  1. Download the compose yaml.
  2. Create a .env file with a few values
  3. Then docker compose up

Seems like it supports “sign up” – if you host this visibly externally you may have some spammy sign-ups.. this may be something you can disable.. (yes, you can disable this as I find out below)

After you have created a user – you are greeted with this blank canvas

I currently run Wallabag – which I landed on after trying a few other choices. It was the best choice for my needs at the time, but also super basic. Wallabag has a mobile app which I find useful – as it makes sharing links I find on mobile easy to my Wallbag install.

Wallabag often struggles to capture a page – but it at least keeps the link. One example is this website – which has some sort of scraper blocker. You get a page that indicates it is protected by this.

Ok – so how does Hoarder do with a link https://www.thekitchn.com/instant-pot-bo-kho-recipe-23242169?

For comparison – this is what wallabag got..

The capture in Hoarder take a bit of time – not long, but it renders sort of a blank-ish card immediately and then the image fills in.


Let’s take a closer look at the tile that it created for me

The top of the tile is a picture and link to the original URL (1). The link (1) is also the same destination.
The date (2) and expansion arrows (2) both take you to a larger locally hosted view.
(3) is a menu of options

Let’s dive deeper into the expanded (locally hosted view)

The overall capture/rendering of the page from the local version is pretty good. Links in the text haven’t been re-written, but that’s both expected and generally useful.

This view also offers the option to view a screenshot – which is as you expect.

Since I didn’t provide an OpenAI key nor did I configure Ollama the fancy “Summarize with AI” button just gives me an error.

Looking – it seems this setup 3 unique containers

  • ghcr.io/hoarder-app/hoarder:release
  • getmeili/meilisearch:v1.11.1
  • gcr.io/zenika-hub/alpine-chrome:123

but.. I’m not seeing any storage on the host – this is probably bad, because that means at least one of these containers is stateful (and looking at the compose — there are two data volumes)

I have a preference of storing my data on the host filesystem as a volume mapping… maybe I’ll warm up to the whole docker volume thing, but it always feels like a big hack. (Read on and you’ll find that there is a way to avoid the storage concerns that I have here).

The search appears limited to the title only (boo) – tags are supported in search too.. but no deep searching within the text of the articles.

Looking more at the doc – persistence is something you can configure

DATA_DIR – “The path for the persistent data directory. This is where the db and the uploaded assets live.”

and it does appear you can stop signups from happening

DISABLE_SIGNUPS – “If enabled, no new signups will be allowed and the signup button will be disabled in the UI”

Interesting options for the crawler (disabled by default)

CRAWLER_FULL_PAGE_SCREENSHOT – “Whether to store a screenshot of the full page or not. Disabled by default, as it can lead to much higher disk usage. If disabled, the screenshot will only include the visible part of the page”

CRAWLER_FULL_PAGE_ARCHIVE – “Whether to store a full local copy of the page or not. Disabled by default, as it can lead to much higher disk usage. If disabled, only the readable text of the page is archived.”

CRAWLER_VIDEO_DOWNLOAD – “Whether to download videos from the page or not (using yt-dlp)”

Overall – I’m pretty impressed. I’m not sure I’m quite ready to dump wallabag, but this might become a project I tackle during the holiday break. That stew recipe is pretty amazing, absolutely worth trying.