Skip to content
Snippets Groups Projects
  1. Aug 22, 2021
  2. Aug 19, 2021
  3. Aug 18, 2021
  4. Aug 17, 2021
  5. Aug 16, 2021
  6. Aug 15, 2021
  7. Aug 14, 2021
  8. Aug 12, 2021
  9. Aug 11, 2021
  10. Aug 10, 2021
  11. Aug 05, 2021
  12. Aug 01, 2021
  13. Jul 31, 2021
  14. Jul 30, 2021
  15. Jul 29, 2021
    • Evangelos Foutras's avatar
      syncarchive: Drop --delay-updates; breaks incremental scan · 9fd5d458
      Evangelos Foutras authored
      The --delay-updates option results in 6G memory usage per archive mirror
      for a total of ~18G memory used on gemini when all three archive mirrors
      are syncing. Less important (but still revelant!) is the memory usage on
      each mirror, which climbs to about 11G during each synchronization.
      
      Removing the --delay-updates option should be OK considering the archive
      hosts data that almost never changes. Without this option, rsync is able
      to do a sequential scan which uses 90M of memory (per archive mirror) on
      gemini and about 250M on each mirror individually.
      9fd5d458
    • Evangelos Foutras's avatar
      syncarchive: Tweak rsync options and drop temp dir · 49ba41bf
      Evangelos Foutras authored
      This reflects the recent changes made to syncrepo.
      49ba41bf
    • Evangelos Foutras's avatar
      syncrepo: Drop --temp-dir option as unneeded · 4fe427a2
      Evangelos Foutras authored
      Using a temporary directory outside of /srv/ftp was meant to protect
      against incomplete files from being synced by downstream mirrors. It
      does not achieve this to much effect though; each file gets uploaded
      to the temporary directory but then immediately moved under a .~tmp~
      directory at its target location (.~tmp~ because of --delay-updates,
      otherwise the file would be renamed to its final path).
      
      The `--delay-updates` option by itself sufficiently protects against
      temp files being transferred to downstream mirrors; when used by the
      receiver, it automatically adds an exclude rule for ~.tmp~, behaving
      exactly like we want it to. As such, the `--temp-dir` option doesn't
      provide any further benefit and can be removed.
      4fe427a2
  16. Jul 28, 2021
  17. Jul 25, 2021
  18. Jul 24, 2021
  19. Jul 20, 2021
Loading