Skip to content
  • Florian Pritz's avatar
    Upload files in chunks of 20 files at a time · 9112c623
    Florian Pritz authored
    
    
    Some pacakges have very many files (stack has ~1376) that are currently
    uploaded all at once. If there is an exception during this run or if the
    program is killed (happened due to 'open files' limit), the sucessfully
    uploaded files are not added to the sqlite db. Next time they are
    uploaded again and the upload may be interrupted again, leading to huge
    time loss.
    
    To fix this, we now upload in chunks of 20 files at a time and add them
    to the db after this. Thus, as long as we manage to upload 20 files
    without crashing, the next time, there will be fewer files to upload.
    
    Signed-off-by: default avatarFlorian Pritz <bluewind@xinu.at>
    9112c623