Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found

Target

Select target project
  • antiz/infrastructure
  • okabe/infrastructure
  • eworm/infrastructure
  • polyzen/infrastructure
  • pitastrudl/infrastructure
  • sjon/infrastructure
  • torxed/infrastructure
  • jinmiaoluo/infrastructure
  • moson/infrastructure
  • serebit/infrastructure
  • ivabus/infrastructure
  • lb-wilson/infrastructure
  • gromit/infrastructure
  • matt-1-2-3/infrastructure
  • jocke-l/infrastructure
  • alucryd/infrastructure
  • maximbaz/infrastructure
  • ainola/infrastructure
  • segaja/infrastructure
  • nl6720/infrastructure
  • peanutduck/infrastructure
  • aminvakil/infrastructure
  • xenrox/infrastructure
  • felixonmars/infrastructure
  • denisse/infrastructure
  • artafinde/infrastructure
  • jleclanche/infrastructure
  • kpcyrd/infrastructure
  • metalmatze/infrastructure
  • kevr/infrastructure
  • dvzrv/infrastructure
  • dhoppe/infrastructure
  • ekkelett/infrastructure
  • seblu/infrastructure
  • lahwaacz/infrastructure
  • klausenbusk/infrastructure
  • alerque/infrastructure
  • hashworks/infrastructure
  • foxboron/infrastructure
  • shibumi/infrastructure
  • lambdaclan/infrastructure
  • ffy00/infrastructure
  • freswa/infrastructure
  • archlinux/infrastructure
44 results
Show changes
Commits on Source (2)
  • Evangelos Foutras's avatar
    gitlab-backup: Set GZIP_RSYNCABLE=yes so borg can dedup · 2222767c
    Evangelos Foutras authored 
    gitlab-backup produces gzipped tarballs that cannot be meaningfully
deduplicated by borg.  This can be mitigated by passing --rsyncable
to gzip.

The above is verified by creating two new borg repositories, adding
the two most recent gitlab.archlinux.org archives to both, with the
difference of re-compressing the tarballs with `gzip -1 --rsyncable`
before adding them to the second repository.

In the first case, the 215.97 GB backup archive gets compressed and
deduplicated down to 176.24 GB. With --rsyncable it gets reduced to
just 12.79 GB. These numbers are for /srv/gitlab/data/backups only,
but the other non-tarballed files get sufficiently deduped already.

Based on the above, I am hoping to see the borg repository for gitlab
shrink over time from the current 3 TB to around 600 GB which is more
manageable.
    Verified
    2222767c
  • Evangelos Foutras's avatar
    Merge branch 'gitlab-backup-gzip-rsyncable' into 'master' · 90a51d33
    Evangelos Foutras authored 
    gitlab-backup: Set GZIP_RSYNCABLE=yes so borg can dedup

See merge request archlinux/infrastructure!429
    90a51d33
Hide whitespace changes
Inline Side-by-side
roles/borg_client/templates/backup-gitlab.sh.j2
View file @ 90a51d33
......@@ -12,4 +12,4 @@ echo "emptying backup directory ${backupdir}"
# Verify that the gitlab_backupdir in ansible was defined, otherwise we will rm /* and remove the previous backup
rm -r "${backupdir:?backup dir unset}/"*
docker exec gitlab gitlab-backup create STRATEGY=copy SKIP=tar
docker exec gitlab gitlab-backup create STRATEGY=copy SKIP=tar GZIP_RSYNCABLE=yes