gitlab-backup: Set GZIP_RSYNCABLE=yes so borg can dedup
Compare changes
@@ -12,4 +12,4 @@ echo "emptying backup directory ${backupdir}"
Due to an influx of spam, we have had to temporarily disable account registrations. Please write an email to accountsupport@archlinux.org, with your desired username, if you want to get access. Sorry for the inconvenience.
gitlab-backup produces gzipped tarballs that cannot be meaningfully deduplicated by borg. This can be mitigated by passing --rsyncable to gzip.
The above is verified by creating two new borg repositories, adding
the two most recent gitlab.archlinux.org archives to both, with the
difference of re-compressing the tarballs with gzip -1 --rsyncable
before adding them to the second repository.
In the first case, the 215.97 GB backup archive gets compressed and deduplicated down to 176.24 GB. With --rsyncable it gets reduced to just 12.79 GB. These numbers are for /srv/gitlab/data/backups only, but the other non-tarballed files get sufficiently deduped already.
Based on the above, I am hoping to see the borg repository for gitlab shrink over time from the current 3 TB to around 600 GB which is more manageable.