Computing – Page 15

Server Upgrade: Part 1 – the build

new-server-parts

I’ve been on and off for nearly 2 years talking about getting new hardware for the server that runs lowtek.ca. The current server is a modest Atom based board with only 2GB of RAM, I’m pretty sure my phone has more compute power. My reasons for upgrading were mostly so I could move over to using a Docker based deployment, having done a bunch of Docker things for work – I’ve really gotten to like it for managing software stacks. The Atom chip doesn’t support some of the virtualization that is needed, heck – it’s also a 32bit system only. Stalling on buying new hardware wasn’t all that hard, there was always other priorities. Then I discovered the current server was put together 7 years ago! Worse, the IDE drive that is the boot volume is older than that.. Clearly I have a ticking time bomb for a server.

I wanted to get a skylake processor, but was happy to stick with an entry level Pentium – it will let me run all the software I care about and while it’s not the fastest, it will be much much faster than the Atom. I wanted 4 RAM slots on the motherboard, I figure 16GB now – with the potential to double that later. I also needed 6 SATA ports (Raid 5 + boot drive), and if I need to get more I’ll buy a PCIe SATA extension port. I went ASUS based on their durability reputation. I picked up a new power supply (because who wants to trust one that is already 7 years old), and a cheap SSD drive which will be much more reliable than any spinning platter.

ASUS B150M-A/M.2 Motherboard – it was a great price, had an 1151 socket, 4 RAM slots and 6 sata ports.
G.SKILL Aegis 16GB (2x8GB) DDR4 – the cheap RAM of the day
Corsair Builder Series CX430 V2 430 – I picked a well priced brand name power supply
Intel Pentium G4400 – Skylake Pentium, base model
Silicon Power Slim S60 2.5″ 60GB – sweet deal on a SSD

If you follow the links, you’ll notice that the first 3 are CanadaComputers and the last two are newegg.ca. I usually buy from CanadaComputers, but while they list the G4400 online you can’t actually buy it at the store. If I’m buying something online, well – newegg.ca is as good as anywhere, and they not only had the G4400 cheaper, they also had a crazy clear out price on the SSD ($37.99).

I'm such a geek. Just sliced my finger open on a CPU heatsink. #ouch

— Roo (@andrew_low) November 5, 2016

The build went smoothly, I was able to re-use an old Dell PC case to house the new hardware temporarily. I had a 3.5″ adapter for the SSD kicking around, and the motherboard came with a couple of SATA cables.

Getting an OS installed proved to be a bit trickier. Something funky is wrong with my desktop Ubuntu, downloads with Chrome are often corrupted (md5sum fails). The 16.10 Ubuntu install even when it is a correct download, will not self check it’s files correctly AND while I could boot it from USB, I couldn’t get it to install on my new hardware.

Hmm. New hardware, and Ubuntu 16.10 won't install. Dropping back to 16.04.1 and all is good. Bleeding edge indeed.

— Roo (@andrew_low) November 6, 2016

I believe the problem is something to do with the newer kernel level, but given that 16.04 is the LTS version and what I’ll be running on the server anyways, it’s not worth banging my head on this problem further.

All in, I spent a bit more than $400 CAD and have a couple of rebates in the mail. Not bad as I’ll probably run this hardware for at least 5 years, and it’ll be a much needed upgrade.

Next up – I need to install the server version and start building out the software stack that will take over the current servers functionality. Once that is mostly ready, it’ll be time to do a hardware swap into the real server case that has all of the drives in it.

JavaScript and the Single Threaded Lie

JavaScript is single threaded

Yes, but.. it is also heavily asynchronous. Since my experience porting Node.js to new platforms, it’s turned into one of my go to languages for hacking web apps. Unfortunately I still fall into the async trap.

Let’s look at some code:

function myFunction(value, callback) {
  setTimeout(function () {
    console.log('Processing value: ' + value);
    callback();
  }, 2000);
}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {
  var item = array[i];
  myFunction(item, function () {
    console.log('Done processing value: ' + item);
  });
}

function myFunction(value, callback) {

setTimeout(function () {

console.log('Processing value: ' + value);

callback();

}, 2000);

}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {

var item = array[i];

myFunction(item, function () {

console.log('Done processing value: ' + item);

});

}

And here is the output of the code if we run it:

$ node bad.js 
Processing value: 1
Done processing value: 9
Processing value: 2
Done processing value: 9
Processing value: 3
Done processing value: 9
Processing value: 4
Done processing value: 9
Processing value: 5
Done processing value: 9
Processing value: 6
Done processing value: 9
Processing value: 7
Done processing value: 9
Processing value: 8
Done processing value: 9
Processing value: 9
Done processing value: 9

$ node bad.js

Processing value: 1

Done processing value: 9

Processing value: 2

Done processing value: 9

Processing value: 3

Done processing value: 9

Processing value: 4

Done processing value: 9

Processing value: 5

Done processing value: 9

Processing value: 6

Done processing value: 9

Processing value: 7

Done processing value: 9

Processing value: 8

Done processing value: 9

Processing value: 9

Done processing value: 9

Now, in this example – it’s pretty clear what is causing the issue. The setTimeout() is doing a delay of 2 seconds. This is an async call, so the for loop has completed it’s entire cycle before any of the callbacks get a chance to run. Timing issues also make things more confusing if things sometimes work since you can’t guarantee the order of asynchronous functions.

The trap for me, is when the code is more complex and it isn’t obvious to see that there is an async call in the way. Also, my brain keeps telling me that for each iteration through the loop, I (think I’m) creating a new variable (item) and that should provide correct isolation for the state.

There are two simple solutions to this problem.

Solution 1 – use the call stack:

function myFunction(value, callback) {
  setTimeout(function () {
    console.log('Processing value: ' + value);
    callback();
  }, 2000);
}

function helper(item) {
  myFunction(item, function () {
    console.log('Done processing value: ' + item);
  });
}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {
  var item = array[i];
  helper(item);
}

function myFunction(value, callback) {

setTimeout(function () {

console.log('Processing value: ' + value);

callback();

}, 2000);

}

function helper(item) {

myFunction(item, function () {

console.log('Done processing value: ' + item);

});

}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {

var item = array[i];

helper(item);

}

Move the async call myFunction into a helper and pass the value to it. This moves it from being a local variable to one on the stack (as a parameter to helper).

Solution 2 – have the callback give the value back:

function myFunction(value, callback) {
  setTimeout(function () {
    console.log('Processing value: ' + value);
    callback(value);
  }, 2000);
}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {
  var item = array[i];
  myFunction(item, function (value) {
    console.log('Done processing value: ' + value);
  });
}

function myFunction(value, callback) {

setTimeout(function () {

console.log('Processing value: ' + value);

callback(value);

}, 2000);

}

var array = [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ];

for (var i = 0; i < array.length; i++) {

var item = array[i];

myFunction(item, function (value) {

console.log('Done processing value: ' + value);

});

}

To be honest, solution 2 is just another call stack approach – but it’s a different enough pattern to be a second solution.

The output of both good solutions looks like:

$ node good.js 
Processing value: 1
Done processing value: 1
Processing value: 2
Done processing value: 2
Processing value: 3
Done processing value: 3
Processing value: 4
Done processing value: 4
Processing value: 5
Done processing value: 5
Processing value: 6
Done processing value: 6
Processing value: 7
Done processing value: 7
Processing value: 8
Done processing value: 8
Processing value: 9
Done processing value: 9

$ node good.js

Processing value: 1

Done processing value: 1

Processing value: 2

Done processing value: 2

Processing value: 3

Done processing value: 3

Processing value: 4

Done processing value: 4

Processing value: 5

Done processing value: 5

Processing value: 6

Done processing value: 6

Processing value: 7

Done processing value: 7

Processing value: 8

Done processing value: 8

Processing value: 9

Done processing value: 9

SSL for everybody

letsencrypt

SSL certificates are a great way to ensure that the website you’re connected to is really the one you think you’re connected to, and it also keeps the traffic between your client and the server secure. The HTTPS protocol uses SSL certificates. The main problem with the SSL infrastructure was that you needed to get one that was signed by one of the central trusted authorities – and generally if you wanted one of these you had to pay for one. There were a few places that would give you a free certificate for personal use, the other alternative was to use a self signed certificate but there were usability issues because it isn’t signed by a trusted authority.

This all changed recently with Let’s Encrypt – you can now get a free certificate with very little effort. If you maintain a website or host an app, you should check letsencrypt.org out. The remainder of this post is a cleaned up set of notes on what I did.

I started out here https://letsencrypt.org/getting-started/ – which seemed to be a good starting point. Then I figured I’d make sure my server met the criteria they had for support, the documentation had some details covering this. I was happy to see Ubuntu 12.04+ and Apache 2.x support, so this made me fairly confident my server was supported.

But.. my Ubuntu doesn’t seem to have a letsencrypt package

$ sudo apt-get install letsencrypt
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package letsencrypt

$ sudo apt-get install letsencrypt

Reading package lists... Done

Building dependency tree

Reading state information... Done

E: Unable to locate package letsencrypt

No problem, we’ll just follow along with https://letsencrypt.org/getting-started/

$ git clone https://github.com/letsencrypt/letsencrypt
$ cd letsencrypt
$ ./letsencrypt-auto --help

$ git clone https://github.com/letsencrypt/letsencrypt

$ cd letsencrypt

$ ./letsencrypt-auto --help

The last command, while asking for help – will do some bootstrapping of let’s encrypt. So don’t skip it. The scripts include calls to sudo, so you don’t have to be root to run them but it will ask for root access.

[Security note – it is always a little bit scary running random scripts, always worth looking at them. There is a growing trend of having “wget -O – http://randomscript.com | bash” be normal, but you should be afraid]

Some exciting updates to my server from doing just the bootstrap. My /etc/ca-certificates got updated (it was probably way overdue), it also dragged me up to date for libssl. It took a while to finish, but we finally got the help screen.

At this point, I have the let’s encrypt tools installed on my server, so time to try them out.

Hopefully the following command is going to register us and get a new certificate for my old expired one.

$ ./letsencrypt-auto --apache

1	$ ./letsencrypt-auto --apache

Well that didn’t work, it picked up some ‘other’ domains I host — but not my main lowtek.ca one. Weird, but probably due to my non-standard configuration of Apache due to years of hacking it. It was easy to bail out so no harm done. Let’s try this then:

$ ./letsencrypt-auto --apache --domains lowtek.ca

1	$ ./letsencrypt-auto --apache --domains lowtek.ca

Ok – much better, email sign up and an agreement (which yes, I took the time to read – it was only 6 pages). It seems as I don’t have a virtual host setup for lowtek.ca and needed to manually pick the apache config file (not a big deal), this was why the first try didn’t work.

Visiting https://lowtek.ca/ shows no more certificate error (woot!) and all looks good. It was really this easy.

The end of the script even suggests you visit: https://www.ssllabs.com/ssltest/analyze.html?d=lowtek.ca to check for issues. These SSL Labs tests, show that my certificate from letsencrypt will expire in just under 3 months, so I’ll want to add a cron job to do a renew. They also gave me a B rating, with lots of gorpy details on why.

To renew, I just need to run this command from time to time

$ ./letsencrypt-auto renew

1	$ ./letsencrypt-auto renew

That’s really easy to do with cron – so I added an entry to my root user crontab to run this once a month.