Skip to content
Snippets Groups Projects

Mirror definition - Proposal for new mirror administration routines and mirror requirements

Merged Anton Hvornum requested to merge torxed/rfcs:mirror-definition into master
1 file
+ 12
12
Compare changes
  • Side-by-side
  • Inline
@@ -81,22 +81,22 @@ Tier4 is user controlled mirrors, not listed in any official mirror lists, but g
Mirror Specification
^^^^^^^^^^^^^^^^^^^^
A `mirror specification`_ has been created and is hereby proposed as the new going forward standard for new mirror submissions. The specification is a living entity, but is versioned and should aim to be backwards compatible but no restriction on such is put in place.
A `mirror specification`_ has been created and is hereby proposed as the new standard for new mirror submissions going forward. The specification is a living entity, but is versioned and should aim to be backwards compatible but no restriction on such is put in place.
The manifest aims to define what a mirror is, and must be in a machine readable format as well as being easy for humans to read. TOML is proposed as an alternative to JSON as it supports comments as well as fulfill both requirements of being human and machine readable.
The manifest aims to define what a mirror is, and must be in a machine readable format as well as being easy for humans to read. TOML is proposed as an alternative to JSON as it supports comments as well as fulfills both requirements of being human and machine readable.
Specific mirror specification versions will become deprecated followed by a discontinue as new versions are created.
Specific mirror specification versions will become deprecated and later discontinue as new versions are created.
Managing Mirrors
^^^^^^^^^^^^^^^^
Before this proposal, mirrors have been managed via `flyspray`_ and `archweb`_. This have been great in the past, but is somewhat slow and tedious for `mirror administrators`_ to manage. As such, it takes more work and time than it could.
Before this proposal, mirrors have been managed via `flyspray`_ and `archweb`_. This has been great in the past, but is somewhat slow and tedious for `mirror administrators`_ to manage. As such, it takes more work and time than it could.
The proposal is therefore that mirrors are managed via `GitLab`_ and more specifically the `mirror project`_. The latest proposed workflow is located at the `mirror project`_.
This automates the process to such a degree that `mirror administrators`_ only have to sign off on new mirrors via new mirror releases, and the mirror administrator have to do no more or less work than already required.
This automates the process to such a degree that `mirror administrators`_ only have to sign off on new mirrors via new mirror releases, and the mirror administrator has to do no more or less work than already required.
Another benefit is that Arch Linux can move away from publicly revieling the mirror operators e-mail address. As GitLab contact information can be used instead. Thus complying with regulatory demands.
Another benefit is that Arch Linux can move away from publicly revieling the mirror operators e-mail address. The GitLab contact information can be used instead, thus complying with regulatory demands.
Mirror decommission
^^^^^^^^^^^^^^^^^^^
@@ -117,20 +117,20 @@ The workflow for decommissioning a mirror is quite similar:
2. Delete the `mirror definition`_ that you want to decommission or just disable the mirror due to maintenance (flip a bool).
3. Submit a new pull request *(PR)*
4. `mirror administrators`_ sign off the *PR* and merge
5. An automatic runner creates a source of truth in the root of the `mirror project`_ in a JSON format. (The runner also checks if the person doing the MR is either a `mirror administrators`_ or mirror owner. ?)
5. An automatic runner creates a source of truth in the root of the `mirror project`_ in a JSON format. (The runner also checks if the person doing the MR is either a `mirror administrators`_ or mirror owner.)
6. The source of truth is parsed by other Arch Linux projects, such as `archweb`_ and `reflector`_
Highly Recommended Mirror Specifications and Requirements
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
As outlined in a rough draft of this RFC during the `Arch Summit 2023 - Mirror Notes <https://md.archlinux.org/5I61f0yFSmysBkV8QQFU-g>`_, new requirements/recommendations should be put into place. This is to make it more clear to a mirror operators what the expectations are, especially when *"upgrading"* a mirror. This RFC is the developed version of the mirror notes during the summit.
As outlined in a rough draft of this RFC during the `Arch Summit 2023 - Mirror Notes <https://md.archlinux.org/5I61f0yFSmysBkV8QQFU-g>`_, new requirements/recommendations should be put into place. This is to make it more clear to mirror operators what the expectations are, especially when *"upgrading"* a mirror. This RFC is the developed version of the mirror notes during the summit.
The requirements will not be enforced immediately in existing mirrors but is heavily encouraged to do as soon as possible. The requirements also acts as a form of `SLA`_ between Arch Linux and the mirror operator(s). At some point in time, the transition period will have to begin in which a proposed grace period is enacted:
- Tier 1 and 2 mirrors will have 6 months to upgrade or change their servers, which can be extended if the procedure might take a longer time.
- Tier 3 mirrors will have 12 months to transition to the new specification.
In case of missing the grace period, additional warnings will be sent out which are then followed by removal from the official mirror list. Tier1 and Tier2 mirrors that don't follow the spec within the grace period will be downgraded to one level below after two attempts of contact without response over a period of two months. The new requirements will first be enacted only on **new** mirrors.
In case of missing the grace period, additional warnings will be sent out which are then followed by removal from the official mirror list. Tier 1 and 2 mirrors that don't follow the spec within the grace period will be downgraded to one level below after two attempts of contact without response over a period of two months. The new requirements will first be enacted only on **new** mirrors.
**Note:** Certain deviations will be made based on regional data. As we know certain regions have different ammounts of mirrors, average infrastructure levels differ vastly across the glob etc. So Arch Linux mirror staff (mirror admins) have the right to not enforce certain requirements after discussions with the mirror operators in certain regions.
@@ -202,7 +202,7 @@ Status Endpoint
This proposes a new way for mirror operators(s) to communicate technical data to `mirror administrators`_ in an automated fashion. This will allow `mirror administrators`_ to build tooling to automatically detect these technical messages and act on them.
The proposed method is via a JSON `service specification`_. This service specification must be discoverable in a automated fashion via either DNS TXT lookup and/or via a `well known`_ URI. The DNS record used for lookup is ``_archmirror_v1.<mirror domain>`` and the format is a URL to the JSON `service specification`_ file hosted somewhere.
The proposed method is via a JSON `service specification`_. This service specification must be discoverable in an automated fashion via either DNS TXT lookup and/or via a `well known`_ URI. The DNS record used for lookup is ``_archmirror_v1.<mirror domain>`` and the format is a URL to the JSON `service specification`_ file hosted somewhere.
The status endpoint can be hosted either on a secondary domain, a `text storage site`_ such as https://0x0.st or directly on the mirror domain.
@@ -224,7 +224,7 @@ Mirrors are then automatically hidden if signatures or content is considered inv
Arch Linux commitment
^^^^^^^^^^^^^^^^^^^^^
The following are proposed commitments from Arch Linux. The commitments aim to ease the way the community interact with Arch Linux in the context of mirrors and mirror management. The task itself is not very complex, but this would also have a positive side effect of ensuring a consistent behaviour across mirrors by providing the following:
The following are proposed commitments from Arch Linux. The commitments aim to ease the way the community interacts with Arch Linux in the context of mirrors and mirror management. The task itself is not very complex, but this would also have a positive side effect of ensuring a consistent behaviour across mirrors by providing the following:
- Provide *(rootless)* container based images for different tiers *(with docker-compose)*
- Provide `pacman` package for different tiers *(for EdgeTier this is required)*
@@ -312,7 +312,7 @@ Suggested future projects
Alternatives Considered
-----------------------
Alternative discussions was not mentioned in the `Arch Summit 2023`_ as the conclusion was that this is a good step forward.
A discussion of alternatives was not mentioned in the `Arch Summit 2023`_ as the conclusion was that this is a good step forward.
.. _scarecely doccumented: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
Loading