Font rendering broken as of pango 1.56.0

Description:

GTK apps are completely broken in latest ArchLinux testing branch. (Both GTK3 and GTK4 apps and desktops are affected)

I would usually have used IRC to contact the ArchLinux testing team to figure out which exact package is causing the issue, but due to my IRC client using GTK I'm currently unable to do that, and due to the severity and emergency of the issue, I'm opening an issue here.

While I'll try to get an IRC client that doesn't use GTK, configuring it might take me precious time, if you can, please contact the ArchLinux testing team ASAP!

A bug like this must not make it's way into the regular ArchLinux repos if it's not already too late.
(I only run ArchLinux with testing repos since I'm a ArchLinux tester)

I update my system daily, I updated my system yesterday before going to bed, so it's a recent breakage.

Steps to reproduce:

Just open any GTK apps, attached is a screenshot of polari (IRC client I currently cannot use to contact ArchLinux testing team).

GTK Desktop like Cinnamon crash too with weird visual artifacts.

Qt apps/desktops like KDE and it's apps are working fine.

assigned to @freswa, @toolybird, and @gromit

added priority3-normal severity4-low statusunconfirmed labels

assigned to @freswa, @toolybird, and @gromit

Logs? What do you get when starting them from a terminal?

Edit to make clear I can't reproduce, either.

FWIW, I'm not able to reproduce that (up-to-date machine using testing repositories).

Update: gtk2 / gtk3 issues seems related to the pango package.

Source of the issues with gtk4 apps is still unknown.

I'm able to use gtk2 / gtk3 apps just fines with latest pango from testing on my side (as seen with e.g. hexchat and remmina).

However, we got someone in #archlinux-testing that is able to reproduce the pango issue somehow (even if it works as expected for @ptr1337 and I).

Is that other person running NVIDIA on wayland? Are any of the people not able to reproduce running this configuration?

Doesn't seem to be related. The other person affected is running intel igpu (on Wayland), while @ptr1337 is running Nvidia (on Wayland) and isn't affected.

I'm on AMD + Wayland and I'm not affected

coredump from remmina provided by someone whose affected by the "pango" issue: https://paste.systemli.org/?b534063dfe72ef8c#D8vyKMTYToH1NdpHVBHEN5HPdNVnP9kvoakwAcDXZcJ9

I'm using Intel igpu on Wayland and so far no problems.

pango is a font rendering library, it's possible gtk4 was broken for a while and I didn't notice, I don't use Adwaita apps outside or IRC (Polari) or VR (Envision), and I haven't done either in a while as I haven't been feeling good enough to do VR during Christmas season.

https://gitlab.gnome.org/GNOME/pango:

• "Pango is a library for layout and rendering of text, with an emphasis on internationalization."

I was also unable to reproduce this issue. My system is up-to-date with the testing repository enabled.

My systemd logs as asked on testing IRC: jrlnctl.txt

I got Konversation working.

polari doesn't produce any logs when ran from terminal sadly, I'll send logs of another gtk4 apps that have issues one I reported pango issue.

Could you please post a list of all your non-repo packages (just post the output of pacman -Qm)?

Links provided by OP in #archlinux-testing:
https://hastebin.skyra.pw/hidazaxura.less & https://hastebin.skyra.pw/otacaxobez.pgsql

Thanks for the pointer, IRC had too many messages for me to go through all! This looks a bit sus:

chaotic-aur mint-themes 2.2.2-1 [installed]

As a foreign theme could easily break in such a manner

I had issues on KDE, mint-themes are just the Cinnamon themes, and I'm not using them outside of Cinnamon.
(Cinnamon is made by the LinuxMint team)

On KDE I just use the default KDE theme.

Default KDE dark theme

Well, I'm not using cinnamon for a while, so I uninstalled mint-themes.

I'm dedicated to staying on testing branch even if I can't use Cinnamon.

This is so I can report on the progression of this issue.

I can desktop share my computer via ruskdesk if needed to help solve the issue.

As long as I can have a contact on Discord/Telegram/Matrix. (Only one of theses is needed)

The difficulty of reproducing the issue is a real problem.

I'll keep testing more stuff and keep everyone updated if I find any useful info.

Removing ~/.cache or just cleaning font cache (fc-cache -f -v) is all that is needed apparently.

fc-cache -f -v fixed all issues for me, WTF.

Well, now it's time to see if we can add something to the install script.

Well, we need something on ArchLinux that runs fc-cache -f -v on login for every users or something.

If me and @zbe got this same issue in testing, users will have the same issues as us when things go into regular ArchLinux branch.

I don't think gtk4 is the right package for that, but this should be discussed here until a package is chosen by the peoples here.

we need something on ArchLinux that runs fc-cache -f -v on login for every users

No, that's not a solution. We stopped doing that years ago. For example: FS#40134 FS#41742

Just to be clear, fontconfig maintains 2 caches: 1 owned by root in /var/cache/fontconfig and the other in user's homedir ~/.cache/fontconfig.

I forgot to backup my fontconfig before deleting it with fc-cache -f -v, if someone has the same issue, can you please upload your fontconfig cache.

Thanks for the input everyone. This is clearly not a packaging issue. Not sure if the PM of affected pkgs is aware (gtk4 pango cairo fontconfig) so let's cc @heftig just in case.

Ideally someone needs to come up with a reproducer then submit it upstream. FWIW, I cannot repro either...

Well, I do have old ~/.cache/fontconfig that causes it, if that's of any help. The only problem is my firefox crashes now every time file dialog window tries to open to upload it (anywhere), heh.

Managed to do it finally: fontconfig.tar.zst

This also affect gtk2 and gtk3, I mainly noticed it was a big issue because it locked me out of my Cinnamon desktop.

This is a major issue as it locks users out of some desktops.

If I only had Cinnamon installed with this bug, I would have been stuck with only the tty to help me.

Thankfully I was lucky to have KDE Plasma installed too when the bug occurred.

@zbe thank you so much for sending in the file.

The cache files don't help reproduce it here. I assume because they immediately get invalidated because they don't match my available fonts.

@zbe Can you perhaps reduce it to a single file by copying from the backup back to your cache dir one-by-one? Only the ones ending in -le64.cache-9 should be relevant.

Sure, I'll do it in couple of hours, when I have time.

Could someone who is still affected please run a bisection on pango between 1.54.0 and 1.56.0 so we know which commit is causing this?

For what it's worth...

fe7cc1d4cd57ee188e4676b65bbef367b1201118 is the first bad commit
commit fe7cc1d4cd57ee188e4676b65bbef367b1201118 (origin/reject-fileless-patterns)
Author: Matthias Clasen <mclasen@redhat.com>
Date:   Tue Nov 26 21:15:30 2024 -0500

Rewrite pango_fc_is_supported_font_format

Since we now require fontconig 2.15, we can use FC_FONT_WRAPPER,
which is a much better fit. At the same time, explicitly reject
patterns without FC_FILE, since our caching is based on the
filename.

pango/pangofc-fontmap.c | 23 ++++++++---------------
1 file changed, 8 insertions(+), 15 deletions(-)

Thanks! The code looks fine to me, so this looks like a dead end.

Does running an affected app with Valgrind get you any complaints? The backtrace linked above seems entirely unrelated to pango, so I'm thinking this is some kind of stack or heap corruption.

I'm not sure this will be of any help either, but here it goes..

valgrind_remmina.log

Yeah, unfortunately this also looks like a dud. Thanks, anyway.

moved from gtk4#6 (moved)

mentioned in issue gtk4#6 (moved)

changed the description