Handle archive.org's request limits
The systemd service is now failing on and off due to reaching the archive.org's upload limit. The code should handle this and apply a timeout.
See the log output:
May 22 12:00:47 orion.archlinux.org find[3612439]: archlinux_pkg_grafana-zabbix: exception raised
May 22 12:00:47 orion.archlinux.org find[3612439]: Traceback (most recent call last):
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 958, in upload_file
May 22 12:00:47 orion.archlinux.org find[3612439]: response.raise_for_status()
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
May 22 12:00:47 orion.archlinux.org find[3612439]: raise HTTPError(http_error_msg, response=self)
May 22 12:00:47 orion.archlinux.org find[3612439]: requests.exceptions.HTTPError: 503 Server Error: Slow Down for url: https://s3.us.archive.org/archlinux_pkg_grafana-zabbix/grafana-zabbix-3.12.0-1-any.pkg.tar.zst
May 22 12:00:47 orion.archlinux.org find[3612439]: During handling of the above exception, another exception occurred:
May 22 12:00:47 orion.archlinux.org find[3612439]: Traceback (most recent call last):
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/home/archive/archive-uploader/upload_pkg_internetarchive.py", line 86, in upload_pkg
May 22 12:00:47 orion.archlinux.org find[3612439]: res = self.ia.upload(identifier, files=files, metadata=metadata)
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/usr/lib/python3.8/site-packages/internetarchive/api.py", line 284, in upload
May 22 12:00:47 orion.archlinux.org find[3612439]: return item.upload(files,
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 1103, in upload
May 22 12:00:47 orion.archlinux.org find[3612439]: resp = self.upload_file(body,
May 22 12:00:47 orion.archlinux.org find[3612439]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 979, in upload_file
May 22 12:00:47 orion.archlinux.org find[3612439]: raise type(exc)(error_msg, response=exc.response, request=exc.request)
May 22 12:00:47 orion.archlinux.org find[3612439]: requests.exceptions.HTTPError: error uploading grafana-zabbix-3.12.0-1-any.pkg.tar.zst to archlinux_pkg_grafana-zabbix, Please reduce your request rate. - total_tasks_queued exceeds global_limit
May 22 12:00:47 orion.archlinux.org find[3612439]: /srv/archive/packages/g/grafana-zabbix
May 22 12:01:43 orion.archlinux.org find[3638236]: archlinux_pkg_libreoffice-fresh-ca-valencia: exception raised
May 22 12:01:43 orion.archlinux.org find[3638236]: Traceback (most recent call last):
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 958, in upload_file
May 22 12:01:43 orion.archlinux.org find[3638236]: response.raise_for_status()
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/usr/lib/python3.8/site-packages/requests/models.py", line 941, in raise_for_status
May 22 12:01:43 orion.archlinux.org find[3638236]: raise HTTPError(http_error_msg, response=self)
May 22 12:01:43 orion.archlinux.org find[3638236]: requests.exceptions.HTTPError: 503 Server Error: Slow Down for url: https://s3.us.archive.org/archlinux_pkg_libreoffice-fresh-ca-valencia/libreoffice-fresh-ca-valencia-6.4.4-1-any.pkg.tar.zst
May 22 12:01:43 orion.archlinux.org find[3638236]: During handling of the above exception, another exception occurred:
May 22 12:01:43 orion.archlinux.org find[3638236]: Traceback (most recent call last):
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/home/archive/archive-uploader/upload_pkg_internetarchive.py", line 86, in upload_pkg
May 22 12:01:43 orion.archlinux.org find[3638236]: res = self.ia.upload(identifier, files=files, metadata=metadata)
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/usr/lib/python3.8/site-packages/internetarchive/api.py", line 284, in upload
May 22 12:01:43 orion.archlinux.org find[3638236]: return item.upload(files,
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 1103, in upload
May 22 12:01:43 orion.archlinux.org find[3638236]: resp = self.upload_file(body,
May 22 12:01:43 orion.archlinux.org find[3638236]: File "/usr/lib/python3.8/site-packages/internetarchive/item.py", line 979, in upload_file
May 22 12:01:43 orion.archlinux.org find[3638236]: raise type(exc)(error_msg, response=exc.response, request=exc.request)