Encrypted Backups and Storage on the Cloud

published: 2022-09-01
tags: cloud encryption linux

I've been on a bit of an adventure the last few days trying cloud backups and general access storage, this started when I ran out of space on my "media server," which is just a Raspberry Pi 4 with an external SSD attached. This little Pi holds all my media, which admittedly isn't a whole lot. It's currently sitting at 382.1GB with the rest of the space being taken up by the various other services running it, like my Gotify server and the Jellyfin server used to access the media. I also use Navidrome for music and a local file-browser server to facilitate quick uploads from my phone and some other small scripts I keep running there instead of on my main PC.

I started looking at hard drives. I thought if I'm getting a new one, I might as well get a big one but (without sales at least) big (ish) drives at around 6 TB and larger are still fairly expensive. A combination of recent topics with friends and on some sites that I frequent prompted me to have a look at cloud storage.

To start, I do actually have a larger 4 TB external drive, however I use primarily for backups, it's plugged once a week-or-so; a backup is done, which comprised of copying over files manually, it was then put away. It's also quite old at this point, I'd guess 6 years or more. It was at the point where I don't really trust it anymore. Another thing is, I, and many others of course, have been looking at reducing power usage because of the insane costs as of late, so I don't really want to go adding any spinning disk hard drive(s) that are probably doing to be running for a long time every day. Finally, I'm just on a bit of a general "I don't want to really deal having things around" lately. I've cleaned up a lot and even replaced my mouse and keyboard with wireless ones (though not just for the sake of it, they needed replacing,) my desk is fucking clean and minimal!

Using cloud storage will ultimately end up costing more than just buying a physical drive. I realise this, of course, but it has some advantages I think outweigh that:

It's a small cost I can tackle on a month-by-month basis
I can expand it if needed (this will vary depending on exactly how you're doing it, of course)
I don't have to deal with managing physical drives
I can use this storage from anywhere and from multiple devices, relatively easily

So, off I went, looking for a cloud provider. I knew of some from friends who use them primarily for backups, things like Backblaze, which is purportedly very good. Services like these always seem to come with a cost for downloading things after you've uploaded, which didn't really suit my needs.

I could pay for something like Google Drive or OneDrive but really, I just don't want to associate with these companies, and they tend to be on the expensive side.

Slight tangent, but about a month or so ago I moved my VPS from Digital Ocean to Hetzner because as it turned out, I got far higher specs for the same price in comparison. In my search I noticed none other than Hetzner pop-up! I'd looked at their storage volumes already, which are chunks of storage you attach to your VPS, which work out at about half of Digital Oceans equivalent but still quite expensive, at the time of this post: €54.12 for 1 TB. Now, as it turns out, I missed another option they have, unrelated to their VPS's; storage boxes. These are much cheaper and independent of their other services. I bought the base 1 TB box for testing, which comes in at €3.94, but after using it for a bit, I'll probably be upgrading it to a 5 TB box which is €13.41.

How do these work? Well, what they really are is extremely limited Linux boxes. After you obtain access to one, it will appear on your Hetzner account and you have a few options to enable or disable depending on how you intend to access the box (so you can increase your security,) these are: WebDAV, Samba and SSH protocols, along with the ability to restrict access outside of Hetzner. If you only intend to use this in conjunction with a service you're hosting on a Hetzner VPS, the box won't be accessible outside of Hetzner's own network, which is a very neat option, especially if you're just going to use this as a NextCloud storage option or something similar. You can SSH directly to the box but it's quite restricted and you can run help to get a list of commands which mostly consists of file navigation commands and du/df for storage usage. However, they also come with borg, rsync and rclone installed on them.

As for using them? Personally, at least for the moment, I've exclusively been using my storage box by mounting it with sshfs onto my computer and my Pi. Now here's the thing, I've never had a cloud backup, I know I should have, but I was content with just manually copying (well, rsyncing) my stuff to that external drive. Even though I'm very aware I should have also had a cloud backup. Since I did finally have some cloud storage, I decided to look into properly doing this stuff. I'd heard of borg backup before, at least 2 of my friends use it, but I had a quick glance at it in the past and to be honest, just couldn't be bothered (silly me!) as it turns out, it's not actually that complicated at all.

If you don't know ‘borg backup’ is, in their own words, a "deduplicating archiver with compression and encryption." It's also purely a command-line program (though I believe GUIs exist.) You start by creating the folder that will become your ‘borg repo,’ mine of course, is located in my storage box mount, so everything is saved to that. To be honest, I can't explain the setup any simpler than borg's own quick start page on their wiki but in case you're wondering, I'm using zstd compression with the "repokey" encryption option. One of borg's features that I really like is the ability to mount these repos so you can browse them like a regular directory in case you just need to grab file or 2 without pulling down the entire backup, super handy. For the moment, I am manually running my backup command, but I'll probably script it later, especially to add the archive pruning that borg can do so I don't end up with a ridiculously over-sized repo.

I had borg set-up, but I also wanted some general use cloud-storage, borg is great but it is primarily built around being a backup solution. I wanted something similar, an encrypted space that I could mount locally and unencrypted, to use with little hassle, because even though I like Hetzner, I don't really trust any cloud provider; it's encrypted or nothing. A quick search brought up a few options: ecryptfs, encfs and gocryptfs. Encfs I immediately stopped looking at when I found that it isn't really maintained and has a lot of known security issues. Ecryptfs is quite old, but still being maintained and seems to have a decent reputation, but it's a little more involved than I was looking for and as far as I can tell, requires an actual partition to be made, which is a no go for cloud storage. Gocryptfs was the last one I looked at, and as it turned out, was exactly what I was looking for. After doing some searching and asking a friend, I found that it's well-known and secure, so, solution found. Gocryptfs is ridiculously simple. After you install it, either via your package manager or the binary download from github, you make your folder you want. For me, one on my storage box. Then simply run gocryptfs -init /path/to/folder, enter a passphrase and you can then mount the unencrypted version with gocryptfs /path/to/encrypted/folder /path/to/local/mount. That's literally it. All files placed in the unencrypted mount will show encrypted in the first folder, filenames and all.

There are some other options on the init you probably want to use, most importantly, the -config option, which will let you specify where to put the gocryptfs.conf file, because by default, it will be in the repo in plain text, which contains an "EncryptedKey" and salt value, not enough to read your files but better it’s better practice to keep it separate anyway; though that of course complicates things a little if you want to mount it on multiple computers, though that doesn't bother me.

That was my adventure in encryption, backups and cloud storage, also my first real blog post, was quite fun, might do more in the future.