top | item 25873156

(no title)

a20eac1d | 5 years ago

The one thing that gives me hypertension when self hosting a code repository is backups.

The hard drive of your server can fail at any time and when self hosting you are responsible for your backups.

This is giving me night terrors, especially when its on a cloud server and I don't have access to the hardware.

Currently, I'm running a cron task once per day executing a simple backup script that does the following:

Stop the Gitea container, copy the entire Gitea directory (including the docker-compose.yml and the data directory) to a backup folder, restart the container, sync that folder to a Backblaze bucket, delete the backup folder.

Restoring the backup is (should) be as easy as downloading the bucket from Backblaze and simply docker-composing it up.

I'm looking for other ideas advice that will help me sleep at night. Thanks!

discuss

order

LinuxBender|5 years ago

For what its worth, another thing you might consider looking into is using RSnapshot [0]. RSnapshot helps me sleep at night. This will create multiple directories that are hard links to files that did not change, thus saving disk space and giving you multiple days or weeks of snapshots, in the even something was corrupted and you want to roll back at the filesystem level. Most questions one could come up about rsnapshot with are answered on Serverfault [1] There are many how-to sites [2] with usage examples. You can create snapshots locally or remote. On mac you can brew install rsnapshot.

[0] - https://rsnapshot.org/

[1] - https://serverfault.com

[2] - https://linuxconfig.org/guide-to-rsnapshot-and-incremental-b...

hnlmorg|5 years ago

Snapshots aren't a backup. However they do obviously have their merits too. Personally I'd recommend ZFS over RSnapshot and then make use of raidz so you have redundancy at the hardware level as well as at the snapshots.

GuB-42|5 years ago

One of the advantages of systems like git is that you just need to clone and you have a complete copy of the original repository, you can synchronize with push/pull. No need to mess with backblaze and containers unless you want to back up your server configuration too. If you lose your server, just push your working copy on a new server. With enough people, losing data becomes almost impossible.

In fact you don't even need a server, just push/pull between your machines, a server just makes things more convenient.

hellcow|5 years ago

Do you not have all your code locally as well? It seems like if your drive failed for a personal git host, you could just re-push everything back up to it.

3np|5 years ago

Have you considered a distributed filesystem?

For example:

Glusterfs Replicated 3 (1 arbiter/parity), put on top of zfs filesystems with checksumming.

Users/consuming services can Fuse mount over the network

Incremental backups of bricks (1 should be enough) to a mirror or spinning rust

Then all you need to worry about is offsite

If you really only need to solve for gitea, this is prob overkill but if you have more services putting stuff on disk it could be worth it. Works great for me.