Skip to content
  • Evangelos Foutras's avatar
    gitlab-backup: Set GZIP_RSYNCABLE=yes so borg can dedup · 2222767c
    Evangelos Foutras authored
    gitlab-backup produces gzipped tarballs that cannot be meaningfully
    deduplicated by borg.  This can be mitigated by passing --rsyncable
    to gzip.
    
    The above is verified by creating two new borg repositories, adding
    the two most recent gitlab.archlinux.org archives to both, with the
    difference of re-compressing the tarballs with `gzip -1 --rsyncable`
    before adding them to the second repository.
    
    In the first case, the 215.97 GB backup archive gets compressed and
    deduplicated down to 176.24 GB. With --rsyncable it gets reduced to
    just 12.79 GB. These numbers are for /srv/gitlab/data/backups only,
    but the other non-tarballed files get sufficiently deduped already.
    
    Based on the above, I am hoping to see the borg repository for gitlab
    shrink over time from the current 3 TB to around 600 GB which is more
    manageable.
    2222767c