Skip to content
Snippets Groups Projects
  1. Jul 30, 2021
  2. Jul 29, 2021
    • Evangelos Foutras's avatar
      syncarchive: Drop --delay-updates; breaks incremental scan · 9fd5d458
      Evangelos Foutras authored
      The --delay-updates option results in 6G memory usage per archive mirror
      for a total of ~18G memory used on gemini when all three archive mirrors
      are syncing. Less important (but still revelant!) is the memory usage on
      each mirror, which climbs to about 11G during each synchronization.
      
      Removing the --delay-updates option should be OK considering the archive
      hosts data that almost never changes. Without this option, rsync is able
      to do a sequential scan which uses 90M of memory (per archive mirror) on
      gemini and about 250M on each mirror individually.
      9fd5d458
    • Evangelos Foutras's avatar
      syncarchive: Tweak rsync options and drop temp dir · 49ba41bf
      Evangelos Foutras authored
      This reflects the recent changes made to syncrepo.
      Verified
      49ba41bf
    • Evangelos Foutras's avatar
      syncrepo: Drop --temp-dir option as unneeded · 4fe427a2
      Evangelos Foutras authored
      Using a temporary directory outside of /srv/ftp was meant to protect
      against incomplete files from being synced by downstream mirrors. It
      does not achieve this to much effect though; each file gets uploaded
      to the temporary directory but then immediately moved under a .~tmp~
      directory at its target location (.~tmp~ because of --delay-updates,
      otherwise the file would be renamed to its final path).
      
      The `--delay-updates` option by itself sufficiently protects against
      temp files being transferred to downstream mirrors; when used by the
      receiver, it automatically adds an exclude rule for ~.tmp~, behaving
      exactly like we want it to. As such, the `--temp-dir` option doesn't
      provide any further benefit and can be removed.
      Verified
      4fe427a2
  3. Jul 28, 2021
  4. Jul 25, 2021
  5. Jul 24, 2021
  6. Jul 20, 2021
  7. Jul 18, 2021
    • Evangelos Foutras's avatar
      Split storage box monitoring into new text collector · c844d0cb
      Evangelos Foutras authored
      This was previously monitored as part of the borg text collector, but
      now that it only runs after each backup (instead of hourly) the stats
      from monitoring.archlinux.org do not remain accurate for long. Switch
      back to hourly checks of the storage box's disk usage by adding a new
      text collector just for this purpose.
      Verified
      c844d0cb
    • Evangelos Foutras's avatar
      Run borg-textcollector after each backup completes · 68def695
      Evangelos Foutras authored
      Instead of gathering borg statistics every hour or so, run the text
      collector script only once after each borg-backup service finishes.
      
      Also split the borg text collector script into two similar scripts,
      where each one gathers borg statistics for its respective borg host.
      Verified
      68def695
  8. Jul 17, 2021
    • Evangelos Foutras's avatar
      Use RandomizedDelaySec=30min in Borg TextCollector · 3aa4d49f
      Evangelos Foutras authored
      Doing this in an attempt to be kind to our Borg hosts in cases where the
      prometheus-borg-textcollector.timer is restarted on all hosts and avoids
      having all machines querying the Borg hosts within the same minute. Only
      downside is that the timers will trigger every 75-ish minutes instead of
      exactly every hour, but this should not be a problem.
      Verified
      3aa4d49f
  9. Jul 16, 2021
  10. Jul 15, 2021
  11. Jul 14, 2021
  12. Jul 13, 2021
  13. Jul 12, 2021
  14. Jul 11, 2021
Loading