In my last post I wrote about my goals of achieving digital sovereignty by self hosting services and their infrastructure. The first step in this process is figuring out where they should be hosted. For previous personal projects I had used AWS's LightSail for relatively easy and not too expensive VPS hosting in the AU Sydney region. But Jeff Bezos can and should go and do rude things to himself, so I wanted to find some Australian owned hosting with a Melbourne presence.
Hosting
I did briefly consider running an additional server from my own cupboard in my study, as I do have fibre to the premise NBN with a nice 1000/50 plan that provides more than enough outbound bandwidth for my personal needs. However, as reliable as my internet generally is, I did feel like I wanted a bit more resilience than home internet and power generally provides. I also wasn't so interested in managing more physical hardware, so I ruled out home based hosting for this endeavour. But it is worth pointing out that as long as you have a static IP configured on your home internet, there's nothing stopping you from physically hosting this stuff in your home.[1]
So to get a good recommendation on some Australian owned and based hosting, I turned to my favourite AU tech nerd community in The Sizzle's Slack group, where Binary Lane was presented as a strong recommendation. For me, a simple VPS in the Melbourne region fit the bill, and I could start for a very reasonable AU$15/month with 2 VCPUs, 4GB ram, 60GB storage and 3000GB/month of data transfer [2] running a Linux OS.
Add a few additional bucks for automated daily backups (including off-site backups) and we have basic disaster recovery incase something goes horribly wrong with the VPS. More on that later.
Managing Server Software
Its well and good to have a place to run software, but you also have to keep it up to date, and that includes all the other software libraries that each service is dependant on. The old school way is to run software directly on the server, but the modern way is to use Software Containers, which effectively bundle a whole OS together along with the software in question. This allows you to not worry about what version of libraries and dependencies are installed on the host OS. Docker is the main standard used here, with most open source software services offering a docker image, providing a super easy way to get things running. Docker on its own is great, but you still need to put a web server in-front of it, and manage configuring and deploying it. So there are many tools that help you to manage these containers.
Now I have friends who swear by Proxmox as the best way to manage all your containerised services. In a universe where I had more ample time I probably would have explored this and gone down this route; however the tools you know are often the best tools to value one's time (assuming they are fit for purpose); so for me I chose Dokku.
Dokku is an open source "Platform as a Service" tool, that mimics the behaviour of Heroku which is the same but a proprietary paid service (owned by Salesforce). Both allow you to quickly deploying code straight up to them - building a docker image for you based on your code requirements. It can also work directly with docker images though, and to this end, it makes it super easy to manage the additional requirements such as databases and related service, protect secrets such as API keys or email credentials, and provide persistent storage[[3]].
Dokku includes the web server nginx and manages its configuration for routing incoming traffic to the right app based on the URL's domain or path. It hides a lot of the configuration and complexity, whilst still allowing you customise details (such as setting maximum file upload sizes etc) when required.
So with Dokku, all I really need to install on the host OS itself is Dokku and some of its plugins that apps might need, such as PostgreSQL, MySQL, Redis and most importantly; Lets Encrypt. The later of course enabling us to issue SSL Certificates for free for our app domains (and importantly keep them renewed and up to date automatically).
Monitoring
With any service, we expect a level of reliability; there's nothing worse than going to use something only to find out it’s broken. This is the biggest downside to self hosting - you are now responsible for keeping things working (and up to date with security patches)! The best outcome is to be notified of any issues with a service before you need it, so we need a way to monitor the things we've set up. Maybe the server's just run out of disk space, or maybe you've pissed off somebody on the internet and they're now targeting you with a denial of service attack.
Binary Line itself offers basic monitoring of CPU, Memory, Disk Storage, Data Inbound and Outbound etc, allowing you to configure email notifications when these hit a configurable threshold. This is great, providing a clue that something might be amiss and that you should investigate.
But it doesn't tell you when a specific service is down, or even if all of them are down. To do that, a specific independent monitoring service should be used. This is still on my list to look into; there are self hosting options here too of course; but ideally you'd want this running on seperate infrastructure (otherwise if the key issue is that your VPS itself has crashed and not started back up, or the data center has lost power, your self hosted monitoring solution will also be offline). But since we're just running some non critical personal things here for now this is fine, enterprise grade monitoring adds costs that I'd deem unnecessary at the moment.
Backups
Related to reliability, the other super important thing to consider is how you will keep things backed up. The great thing about Dokku and Container Images in general is that they provide much of the work needed to bring services back up and running on a new server if something goes astray (or even to upgrade, or rollback if the upgrade went badly). The data that we need to be concerned about backing up and being able to restore are:
- The Dokku configuration for the app
- User data in local file paths mounted into to the container
- User data in any database connected to the app[[4]]
Binary Lane's backup offering gives a full disk image of your VPS. This allows a complete recovery of the server from a backup quickly, in event of catastrophic issue, for example an OS upgrade gone bad. It also allows mounting a backup image as a file volume on the existing server, allowing you to go in and cherry pick the files you may want. At additional cost you can get 'off-site' storage of backups too so that if the data center explodes, the backups don't explode with it.
However these backups still all rely on Binary Lane being accessible, so ideally I'd prefer a local backup too in my home. Luckily another nice thing about Binary Lane is their public API allowing you to programatically do anything they offer in their console. This includes scripting to automatically perform a backup and download it locally, preserving as many snapshots as you may desire. And amazingly such a script was already made, so I didn't even need to do that myself. Thanks Dean! I have this running daily at 3am on my home linux server, storing the last 30 days of backups. [5]
What's next?
So far Binary Lane has been a solid choice, great value and offering all the tools I need [6]. Dokku too has been solid (this is no shock to me, as I've been using it in my professional career for development servers for nearly a decade). Since starting this journey I've bumped my single VPS up to a 4vCPU, 8GB ram VPS with 100GB storage - for a super reasonable AU$35/m (including a daily backup snapshot) running eight seperate services now in Dokku. Having all my services on one VPS is definitely a weakness, but this is always a tradeoff with additional monetary and time costs of multiple VPSs.
Next I'll look at what I did to get Ghost up and running for this blog, with some basic self hosted analytics (importantly; one that is privacy focused and respectful of user's tracking desires).
Top Photo by Taylor Vick / Unsplash
Technically you could also utilise a dynamic IP with a dynamic DNS service, but really if you're going to all this effort, just pay your ISP the extra couple of bucks per month to get a static IP. ↩︎
These specs are virtualised; you are not getting direct access to physical hardware (which is of course also an option for a much higher monthly fee). Think of a virtual machine in the cloud, which doesn't actually care what else is running on it. VPSs are much cheaper but you do share the physical resources with other people and this includes outbound IP addresses. The downside to this is that other people could potentially impact your services. You wouldn't want to run a commercial service that earns you money on a VPS, but for less critical personal projects, they're great value. ↩︎
The best way to think of container images, are that they are read only. Any time you want to upgrade the image, you effectively throw away the old one, and replace it with the new one. This includes all files within the container, so if any of those have changed, they don't survive this process. So to persist data, it needs to be kept elsewhere, accessible to the app in the container, but not in the container itself (via externally linked databases, file paths or 'block storage' like AWS's S3 or compatible services.) ↩︎
Database data and mounted file folders are actually the same thing, as any database container image needs to store its database files in a mounted file path to persist the data too. But I find it good to conceptually treat these differently, as there can be more ways to deal with database data than there are with plain files, that allow more flexibility in data recovery. ↩︎
Home Linux server? Didn’t I just say I didn’t want a home server? I did, although that was really about not wanting another one. What I keep on my home server are things generally specific to my home, such as large media file storage and running HomeBridge for integrating non-apple HomeKit compatible devices with HomeKit. ↩︎
I promise that this is no hidden sponsorship or anything, I've been genuinely super happy and impressed with Binary Lane, hence my praise - it was a great recommendation from The Sizzle community. It doesn't have the bells and whistles and millions of options like the big international cloud services I've used professionally such as AWS, Azure, GCP, Digital Ocean etc, but what it does, is great. ↩︎