diff --git a/rfcs/0029-mirror-definition.rst b/rfcs/0029-mirror-definition.rst
index 4f3cdb6279259147952b8eb40a5e9a4aa68bc81e..78a1407dd1d8a654d345fc357d81255caa8c7f9d 100644
--- a/rfcs/0029-mirror-definition.rst
+++ b/rfcs/0029-mirror-definition.rst
@@ -18,7 +18,9 @@ Before this proposal, there was a lack of requirements and definition of what a
Motivation
----------
-Mirrors are a well known concept on an abstract level as they are one of the foundations of most Linux distributions. However, the Arch Linux specific requirements put on mirrors are unknown or `scarecely doccumented`_. The process as of writing this is also tedious and manual for mirror administrations which is not ideal for anyone participating in the mirror workload.
+Mirrors are a well known concept on an abstract level as they are one of the foundations of most Linux distributions.
+However, the Arch Linux specific requirements put on mirrors are unknown or `scarcely documented`_.
+The process at the time of writing is tedious and manual for mirror operators which is not ideal for anyone participating in the mirror workload.
This RFC aims to rectify this by first defining the different types of mirrors and then introducing requirements for those types of mirrors. This RFC also outlines a new method of becoming a listed mirror and managing the mirror list via `GitLab`_ rather than `archweb`_ and `flyspray`_.
@@ -40,18 +42,23 @@ The following points define the scope of this RFC:
What will happen with existing mirrors
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Existing Arch Linux mirror metadata in the archweb will be transformed and migrated to the GitLab repository. Mirror owners will be able to open issues and merge requests to interact with their mirror. Mirrors will be allowed a certain grace period until they might be considered for the decommission process or deactivation. The criteria for what happens after the grace period depends on the Tier level, region of operation and mirror administrators best effort judgement of the particular mirror.
+Existing Arch Linux mirror metadata in archweb will be transformed and migrated to a GitLab repository.
+Mirror owners will be able to open issues and merge requests to interact with their mirror data.
+Mirrors will be allowed a certain grace period until they may be considered for the decommission or deactivation process.
+The criteria for what happens after the grace period depends on the Tier level, region of operation and the mirror administrators best effort judgement of the particular mirror.
-Depending on the mirror tier and mirror configuration, the mirror might be lowered or upgraded on the new tiering scheme. During the initial migration period the mirrors shall be sorted depending on the metadata arch linux mirror administrators have at hand. The mirror owners will be notified of this change if a means of valid contact was provided.
+Depending on the mirror tier and mirror configuration, the mirror might be lowered or upgraded on the new tiering scheme.
+During the initial migration period the mirrors are sorted depending on the metadata available to the Arch Linux mirror administrators.
+The mirror owners will be notified of this change, if a means of valid contact was provided.
-The transition period will be long and any improvements may still be made to the whole process - as well as Arch Linux mirror administrators best judgement will be applied to each individual situation.
+The transition period will be long and any improvements may still be made to the whole process.
Timeline
--------
| 3 months notice period of change | 3 months migration/grace period | 2 months clean-up phase |
-Suggested start date for these changes is one week after the RFC has been accepted and one announcement has been e-mailed to all mirror operators primary point of contact.
+Suggested start date for the proposed changes is one week after the RFC has been accepted and a first announcement has been sent to each mirror operator's primary point of contact.
Specification
-------------
@@ -59,8 +66,9 @@ Specification
Mirror Types introduction
^^^^^^^^^^^^^^^^^^^^^^^^^
-Mirror types are vaguely named according to a tier-model.
-We `mirror administrators`_ propose the current mirror model stays the same and be represented as an integer for optimization purposes. And their short name *(T0, T1 etc)* remain the same where applicable. But wherever applicable should be represented in a *"human readable form"* to what they represent, as well as introduce a new "Tier 3" and "Tier 4" mirror type:
+Mirror types are assigned in accordance with a tier-model.
+The current `mirror administrators`_ propose to largely reuse the existing mirror model and extend it with new "Tier 3" and "Tier 4" mirror types.
+Short names (*T0*, *T1*, *T2*, *T3* and *T4*) as well as human readable forms (*SourceTier*, *DistributionTier*, *ContentDeliveryTier*, *EdgeTier* and *LocalMirror*) will be used where applicable.
- Tier0(T0): SourceTier
- Tier1(T1): DistributionTier
@@ -68,56 +76,61 @@ We `mirror administrators`_ propose the current mirror model stays the same and
- Tier3(T3): EdgeTier
- Tier4(T4): LocalMirror
-Tier0 This is the mirror Arch Linux Packager's update packages against, and is hosted by the Arch Linux Devops team.
+Tier0 This is the mirror Arch Linux Packagers build packages against, and is hosted by the Arch Linux Devops team.
Tier1 This mirror type is intended to be a near real time copy of the Tier0 mirror. This puts higher demand on this mirror type as it needs to update as soon as possible with little to no delay, and uptime is paramount for the success of other mirrors following this tier.
-Tier2 would be used for the mirrors that don't have a static geographical point, but rather optimizes based on the clients geographical position.
+Tier2 would be used for the mirrors that don't have a static geographical point, but rather optimizes based on the client's geographical position.
Tier3 would be a best-effort mirror closer to individual users, and will have far less requirements and follow-ups from the `mirror administrators`_.
-Tier4 is user controlled mirrors, not listed in any official mirror lists, but gets a name so that we can have a well defined term for those types of mirrors.
+Tier4 are user controlled mirrors, not listed in any official mirror lists, but get a name so that we can have a well defined term for those types of mirrors.
Mirror Specification
^^^^^^^^^^^^^^^^^^^^
-A `mirror specification`_ has been created and is hereby proposed as the new standard for new mirror submissions going forward. The specification is a living entity, but is versioned and should aim to be backwards compatible but no restriction on such is put in place.
+A `mirror specification`_ has been created and is hereby proposed as the new standard for new mirror submissions going forward.
+The specification is a living document, but is versioned and should aim to be backwards compatible but no restriction is put in place in this regard.
-The manifest aims to define what a mirror is, and must be in a machine readable format as well as being easy for humans to read. TOML is proposed as an alternative to JSON as it supports comments as well as fulfills both requirements of being human and machine readable.
+The specification aims to define what a mirror is, and must be in a machine readable format as well as being easy for humans to read.
+TOML is proposed as an alternative to JSON as it supports comments as well as fulfills both requirements of being human and machine readable.
-Specific mirror specification versions will become deprecated and later discontinue as new versions are created.
+In the future, older versions of the mirror specification may be entirely superseded by newer ones.
Managing Mirrors
^^^^^^^^^^^^^^^^
Before this proposal, mirrors have been managed via `flyspray`_ and `archweb`_. This has been great in the past, but is somewhat slow and tedious for `mirror administrators`_ to manage. As such, it takes more work and time than it could.
-The proposal is therefore that mirrors are managed via `GitLab`_ and more specifically the `mirror project`_. The latest proposed workflow is located at the `mirror project`_.
+The proposal is therefore that mirrors are managed via the `mirror project`_ (a `GitLab`_ project) instead.
+The latest proposed workflow for this is documented in the `mirror project`_.
-This automates the process to such a degree that `mirror administrators`_ only have to sign off on new mirrors via new mirror releases, and the mirror administrator has to do no more or less work than already required.
+This allows automating the process to such a degree that `mirror administrators`_ only have to sign off on new mirrors via new mirror releases, and mirror operators are offered a streamlined process.
-Another benefit is that Arch Linux can move away from publicly revieling the mirror operators e-mail address. The GitLab contact information can be used instead, thus complying with regulatory demands.
+As an additional upside to the previous workflow, mirror operator contact information does not have to be publicly revealed, as the GitLab contact information complies with regulatory demands.
Mirror decommission
^^^^^^^^^^^^^^^^^^^
-As unfortunate as it may be, a mirror can be up for decommissioning. This action is taken to preserve and maintain the quality of mirroring for the distribution. Some issues may be handled differently if an edge case arises.
+Under certain circumstances a mirror needs to be decommissioned.
+This action is taken to preserve and maintain the quality of mirroring for the distribution as a whole.
-Mirrors may be decommissioned due to several reasons:
+Mirrors may be decommissioned due to the following reasons, which are evaluated on a case-by-case basis:
- The mirror is unreachable or unable to fulfill its service as a mirror.
- Voluntary withdrawal by the mirror owner.
- Malicious behaviour, such as attempting to serve malicious files, or domain hijacking.
- Failure to follow specifications for a prolonged amount of time, even after given grace periods.
-Decommissioning is the last step after deactivation, as there would be grace periods and ongoing communication between the involved parties.
+Decommissioning is the last step in the process of mirror removal.
+It is embedded in grace periods and ongoing communication between the involved parties.
-The workflow for decommissioning a mirror is quite similar:
+The workflow for decommissioning a mirror is defined by the following steps:
1. Fork the `mirror project`_.
-2. Delete the `mirror definition`_ that you want to decommission or just disable the mirror due to maintenance (flip a bool).
-3. Submit a new pull request *(PR)*
+2. Delete the `mirror definition`_ that you want to decommission or just disable the mirror due to maintenance (flip a bool).
+3. Submit a new merge request (*MR*) for the changes
4. `mirror administrators`_ sign off the *PR* and merge
-5. An automatic runner creates a source of truth in the root of the `mirror project`_ in a JSON format. (The runner also checks if the person doing the MR is either a `mirror administrators`_ or mirror owner.)
+5. A continuous integration pipeline creates a JSON formatted file, which serves as source of truth for the overall state of mirrors providing Arch Linux.
6. The source of truth is parsed by other Arch Linux projects, such as `archweb`_ and `reflector`_
Highly Recommended Mirror Specifications and Requirements
@@ -125,25 +138,31 @@ Highly Recommended Mirror Specifications and Requirements
As outlined in a rough draft of this RFC during the `Arch Summit 2023 - Mirror Notes <https://md.archlinux.org/5I61f0yFSmysBkV8QQFU-g>`_, new requirements/recommendations should be put into place. This is to make it more clear to mirror operators what the expectations are, especially when *"upgrading"* a mirror. This RFC is the developed version of the mirror notes during the summit.
-The requirements will not be enforced immediately in existing mirrors but is heavily encouraged to do as soon as possible. The requirements also acts as a form of `SLA`_ between Arch Linux and the mirror operator(s). At some point in time, the transition period will have to begin in which a proposed grace period is enacted:
+The requirements will not be enforced immediately in existing mirrors but is heavily encouraged to do as soon as possible. The requirements also acts as a form of agreement between Arch Linux and the mirror operator(s). At some point in time, the transition period will have to begin in which a proposed grace period is enacted:
-- Tier 1 and 2 mirrors will have 6 months to upgrade or change their servers, which can be extended if the procedure might take a longer time.
+- Tier 1 and 2 mirrors will have 6 months to upgrade or change their servers, which may be extended in case the procedure takes longer.
- Tier 3 mirrors will have 12 months to transition to the new specification.
-In case of missing the grace period, additional warnings will be sent out which are then followed by removal from the official mirror list. Tier 1 and 2 mirrors that don't follow the spec within the grace period will be downgraded to one level below after two attempts of contact without response over a period of two months. The new requirements will first be enacted only on **new** mirrors.
+The new requirements will at first only be enacted on **new** mirrors.
+In case of missing the grace period, additional warnings will be sent out which are then followed by removal from the official mirror list.
+Tier 1 and 2 mirrors that are not able to follow the new requirements within the grace period will be downgraded to one level below after two attempts of contact without response over a period of two months.
-**Note:** Certain deviations will be made based on regional data. As we know certain regions have different ammounts of mirrors, average infrastructure levels differ vastly across the glob etc. So Arch Linux mirror staff (mirror admins) have the right to not enforce certain requirements after discussions with the mirror operators in certain regions.
+**Note:** Certain deviations will be made based on regional data.
+We are aware that certain regions have different amounts of mirrors and average infrastructure levels differ vastly across the globe.
+The `mirror administrators`_ reserve the right to not enforce certain requirements after discussions with the mirror operators in certain regions.
Recommendation and Requirements for different mirror types
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-- **Tier 0** *(SourceTier)* - This is the Arch Linux source of truth for packages. It's workflow is out of this RFC's scope and should be documented elsewhere.
-- **Tier 1** *(DistributionTier)* - This mirror should aim to only be a distribution to the region *(or otherwise)* and not face end users directly if possible. With no restriction imposed to face users however - especially in mirror-scarce regions.
+- **Tier 0** *(SourceTier)* - This is the Arch Linux source of truth for packages.
+ Its workflow is out of scope for this RFC and should be documented separately.
+- **Tier 1** *(DistributionTier)* - Mirrors of this type should aim to only act as distributor to the region by serving lower tiered mirrors and - if possible - not end users directly.
+ However, no restriction are imposed on serving end users however, especially in mirror-scarce regions.
Tier 1 Requirements
- Minimum two active e-mail contacts on minimum one GitLab account
- - Active monitoring of tagged GitLab issues (initial response within 1-2 days)
+ - Active monitoring of mentions in GitLab issues and merge requests (initial response within 1-2 days)
- Uptime above 99.5% per year
- 1Gbit/s throughput
- 1:1 copy of T0
@@ -165,7 +184,7 @@ Recommendation and Requirements for different mirror types
- Required SLA *(response time)*
- Stricter error checks
- Signed domain+lastupdate
- - No fail2ban/rate-limiting
+ - No fail2ban/rate-limiting for lower tier mirrors
- “Status JSON manifest†discovered via DNS (TXT?)
- Paste:able via Pastebins, or on a secondary webserver
- Also paste:able in a GitLab issue
@@ -190,7 +209,7 @@ Recommendation and Requirements for different mirror types
Tier 3 Requirements
- - Support the latest HTTPS best practice ciphers and version of TLS *(as of writing TLS1.3 for instance)*
+ - Support HTTPS
- Sync at least once per day
- Downtime can last no longer than 48 hours *(after which it gets disabled, not removed)*
@@ -241,32 +260,43 @@ The following are proposed commitments from Arch Linux. The commitments aim to e
Tier Structure
^^^^^^^^^^^^^^
-This RFC outlines a new proposal for the tier model, along side the naming of the different tiers. The new model consists of promoting more decentralized mirrors both by protocol but also by tier levels.
+This RFC outlines a new proposal for the tier model, alongside the naming of the different tiers.
+The new model consists of promoting more decentralized mirrors both by protocol but also by tier levels.
This aims to combat overcrowding regions with high tier mirrors as well as enhance the user experience. Despite it being uplifting to have many high tier mirrors, it defeats the purpose of the tier model slightly.
-The proposal is therefore to reorganize the current T1 in the regions where there's an abundance of them to T2, and this should not be considered a downgrade - but a sideways migration of duties. The same goes for T2 in overpopulated regions where some will be changed to a T3 status.
+The proposal is therefore to reorganize some of the current *T1* mirrors in regions where there's an abundance of them to *T2*.
+This should not be considered a downgrade - but a sideways migration of duties.
+The same goes for *T2* in overpopulated regions where some will be changed to a *T3* status.
The migration will depend on gathered service statistics among geographical factors where applicable.
Mirror Protocols
^^^^^^^^^^^^^^^^
-This RFC proposes that supported protocols should foremost be defined by the `mirror specification`_. Specifically this RFC aims to define the structure of which torrents could be used between inter-mirror communication.
+The `mirror specification`_ will define supported file transfer protocols.
+Additionally, this RFC aims to propose the use of torrents for inter-mirror communication.
-This aims to be a viable method of offloading mainly the SourceTier and DistributionTier mirrors as the protocol has built-in offloading of data sharing. But torrent-only mirrors will not be allowed in the first version. Instead in such scenarios `seed box`_ endpoints can be used and thus not being listed.
+This aims to be a viable method of offloading mainly the *SourceTier* and *DistributionTier* mirrors as the torrent protocol has built-in offloading of data sharing.
+However, torrent-only mirrors will not be supported and listed in the first version of the `mirror specification`_.
+It is suggested to instead use `seed box`_ endpoints in such scenarios.
-The RFC proposes Arch Linux runs and maintains a torrent tracker for direct announcements. This would allow Arch Linux to prioritize different peer-listing depending on the requesting peer. Tier2 peers could have Tier1 prioritized in the peer listing for instance. It would also allow for geographical optimizations based on known locations of mirrors where applicable.
+This RFC encourages Arch Linux to run and maintain a torrent tracker for direct announcements.
+This would allow Arch Linux to prioritize different peer-listing depending on the requesting peer.
+*Tier2* peers could have *Tier1* prioritized in their peer listing for instance.
+It would also allow for geographical optimizations based on known locations of mirrors where applicable.
Future work could be done to allow for `middleware`_ between `pacman`_ and mirrors, allowing for torrent to be a viable source of package downloads in high-capacity connections. A proof of concept for such a scenario was created after `Arch Summit 2023`_ called `pactor`_. The proof of concept was slow and requires further work on `pacman`_ to allow for *"mirrors"* with potential delays.
-It could also allow for a `seed box`_ mirror type to be created with the soul purpose of speeding up package distribution between mirrors of all types.
+It could also allow for a `seed box`_ mirror type to be created with the sole purpose of speeding up package distribution between mirrors of all types.
-Introducing torrent into the mirror-to-mirror communication, some what nullifies the tier model as all mirrors can simultaneously offload each other. But keeping the tier model could allow Arch Linux to artificially speed up the propagation from the server side.
+Introducing torrent into the mirror-to-mirror communication, somewhat nullifies the tier model as all mirrors can simultaneously offload each other.
+But keeping the tier model could allow Arch Linux to additionally speed up the propagation from the server side.
Mirror Tooling
^^^^^^^^^^^^^^
-To facilitate the changes outlined in this RFC, tooling would not only be beneficial but crucial for managing things going forward. To preface, all features workflows of ``mirrorctl`` should also be possible to do manually from the mirror administrators.
+To facilitate the changes outlined in this RFC, tooling would not only be beneficial but crucial for managing things going forward.
+All `mirror administrators`_ workflows outlined below should also be possible manually.
This RFC proposes a tool called ``mirrorctl`` which should have at least the following features:
@@ -284,11 +314,14 @@ Container images for setting up a mirror should be separated into the different
Drawbacks
---------
-The main drawback would be that old mirrors might not updated to the proposed changes. The RFC has taken this into account and it produces no unwanted results other than old mirrors not supporting new features.
+The main drawback would be that old mirrors might not update to the proposed changes.
+This RFC has taken this into account and it produces no unwanted results other than old mirrors not supporting new features.
-The RFC proposes gradual changes to the mirror layout to allow old mirrors to conform with new requirements. If this change would greatly impact package distribution, a re-evaluation of the process shall be done to minimize the impact of package distribution.
+Gradual changes to the `mirror specification`_ may impact package distribution.
+In such a case the proposed process will be re-evaluated.
-It would require some more maintenance from Arch Linux staff, specifically the devops team. However, the added workload is considered to be less than or equal to that of the current `mirror administrators`_.
+A slight maintenance overhead is imposed on Arch Linux staff, specifically the devops team.
+However, the added workload is considered to be less than or equal to that of the current `mirror administrators`_.
Unresolved Questions
--------------------
@@ -315,7 +348,7 @@ Alternatives Considered
A discussion of alternatives was not mentioned in the `Arch Summit 2023`_ as the conclusion was that this is a good step forward.
-.. _scarecely doccumented: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
+.. _scarcely documented: https://wiki.archlinux.org/title/DeveloperWiki:NewMirrors
.. _archweb: https://github.com/archlinux/archweb
.. _flyspray: https://bugs.archlinux.org/
.. _mirror administrators: https://gitlab.archlinux.org/groups/archlinux/teams/mirror-administrator/-/group_members