Import all flyspray issues from the keyring project

added hinthelp-wanted scopeenhancement labels

@kevr has created a very cool tool for this: https://gitlab.archlinux.org/kevr/flyspray2gitlab and I did a trial import when this project was just a pull mirror. Since then he has improved the tool and implemented all the features I asked for (thanks @kevr!).

I just did another trial import here: https://gitlab.archlinux.org/flyspray-migration-test/archlinux-keyring/-/issues/, please check it out and report back if anything looks wrong.

I just redid the import after @kevr fixed a few things. Tasks with multiple assigness is handled properly now and the Additional comments about closing is commented by the correct user.

Edit: I need to do at least one more trial import as I forgot a username-mapping for @freswa.

Sounds good. Thanks for looking into this! :)

Here is the final mapping list (Flyspray user to GitLab user):

{
  "brain0": null,
  "pierre": "pierre",
  "allan": "allan",
  "bluewind": "bluewind",
  "barthalion": "bpiotrowski",
  "diabonas": "diabonas",
  "anthraxx": "anthraxx",
  "eschwartz": "eschwartz",
  "toofishes": null,
  "wonder": null,
  "snowman": null,
  "xyne": null,
  "city-busz": "bgyorgy",
  "lfleischer": "lfleischer",
  "jelly": "jelle",
  "eworm": "eworm",
  "vesath": null,
  "tpowa": null,
  "grazzolini": "grazzolini",
  "foxboron": "foxboron",
  "ricsch": null,
  "sergej": null,
  "arcan1s": null,
  "falconindy": null,
  "anatolik": "anatolik",
  "muflone": "muflone",
  "felixonmars": "felixonmars",
  "bash": null,
  "arojas": "arojas",
  "svenstaro": "svenstaro",
  "xyproto": null,
  "ambrevar": null,
  "foutrelis": "foutrelis",
  "tensor5": null,
  "alucryd": "alucryd",
  "nicohood": null,
  "foxxx0": null,
  "demize": "demize",
  "schivmeister": null,
  "dvzrv": "dvzrv",
  "andrewsc": "andrewsc",
  "alad": "alad",
  "escondida": null,
  "cesura": null,
  "wild": "wild",
  "ffy00": "ffy00",
  "shibumi": "shibumi",
  "yan12125": "yan12125",
  "archangegabriel": null,
  "zorun": null,
  "polyzen": "polyzen",
  "kgizdov": "kgizdov",
  "kpcyrd": "kpcyrd",
  "heftig": "heftig",
  "frederik": "freswa",
  "hashworks": "hashworks",
  "raster": null,
  "daurnimator": "daurnimator"
}

I haven't checked all the users without a mapping (null), perhaps you could do a quick skimming of the list.

The following are not with us anymore and have no SSO account:

The following are not with us anymore, but have an SSO account:

falconindy - @dreisner

The following I have no clue about, they seem not to exist:

ricsch
bash

These are all active staff and I can send a mail to remind them to sign up now:

Let's give it a week from now?

ricsch

Looks like he accidentally created a task for the Keyring project: https://bugs.archlinux.org/task/37121

bash

It is https://archlinux.org/people/developer-fellows/#andrea (Flyspray user: https://bugs.archlinux.org/user/1846).

Let's give it a week from now?

Sounds good to me.

mentioned in issue infrastructure#320 (closed)

"archangegabriel": archange

I’m hopping to get everything consolidated to that pseudo once every Arch services use SSO. ;)

I've done the requested steps, hope it is all working fine

"escondida": "escondida"

Edit: quotes

I am here:

https://bugs.archlinux.org/user/1098

"sergej": "spupykin"

I am here: https://bugs.archlinux.org/user/19564

"foxxx0":"foxxx0"

I'm here: https://bugs.archlinux.org/user/20791

"tensor5": "tensor5"

The final import will happen with these commands and username mapping:

$ GITLAB_TOKEN="xxx"
$ python flyspray.py --password xxx --project-id 7 --db-type mysql > flyspray.json
$ ./gitlab.py --attachments "/srv/http/flyspray/attachments" --default-target flyspray-migration-test/archlinux-keyring-test -t "${GITLAB_TOKEN}" -b "https://gitlab.archlinux.org" -u "https://bugs.archlinux.org" --username-mapping username_mappings.json --id-mapping-output task-id-to-issue-url-mapping.json --promote --dump-file flyspray.json import

{
  "Alad": "alad",
  "Alucryd": "alucryd",
  "Barthalion": "bpiotrowski",
  "City-busz": "bgyorgy",
  "Foxboron": "foxboron",
  "jelly": "jelle",
  "Shibumi": "shibumi",
  "Svenstaro": "svenstaro",
  "Wild": "wild",
  "frederik": "freswa"
}

First a trial import will be done, if everything looks good, I will do the final import. Due to GitLab limitations, the users neeed to be "owner" if we want to comment as them and have the correct timestamp*.

* sudo is used to create the issue/comment as the relevant user but created_at requires administrator or project/group owner rights.

So the plan is:

Do the trial migration
Check that everything looks good
Do the real migration
1. Move this project to: https://gitlab.archlinux.org/flyspray-migration-test/archlinux-keyring
2. Do the migration
3. Check that everything looks good
4. Move the project back
5. Delete all the promoted users from the flyspray-migration-test group
6. Profit

A final trial migration has been done per ^ (the project is located here) with this patch to flyspray2gitlab:

diff --git a/gitlab.py b/gitlab.py
index c6737d1..7a5950d 100755
--- a/gitlab.py
+++ b/gitlab.py
@@ -569,11 +569,6 @@ def restore(token, repository, group, is_group):
     if len(users_to_remove):
         logging.info("Removing users we added...")
 
-    for user in users_to_remove:
-        request(requests.delete, member_endpoint(group, user), json={
-            "access_token": token
-        })
-        logging.info(f"Removed {user.get('username')} from the project.")
     to_remove = set()
 
     if len(email_settings):

As we want to avoid adding/removing the users as "owner" for every single run.

Beforehand all the existing test issues was deleted with:

$ curl --request DELETE --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" "https://gitlab.archlinux.org/api/v4/projects/19589/issues/"{386..480}

task-id-to-issue-url-mapping.json

The final migration has been done now per ^ and all the members removed from the flyspray-migration group with this command:

$ curl -s --header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" "https://gitlab.archlinux.org/api/v4/groups/flyspray-migration-test/members" | jq .[].id | xargs -I{} -n1 -P1 curl --request DELETE --header "PRIVATE-TOKEN: $GITLAB_TOKEN" "https://gitlab.archlinux.org/api/v4/groups/flyspray-migration-test/billable_members/{}"