1.1.0 breaking firewalld

It was a routine pkgver update so this is unlikely to be an Arch packaging issue. Would guess that upstream firewalld needs to adapt to changes in latest nftables-1.1.0.

You could always perform a git bisection on nftables to identify which commit is causing the issue then notify upstream of your findings. See also:

https://github.com/firewalld/firewalld/issues/1366

added priority3-normal scoperegression severity4-low statusunconfirmed labels

assigned to @freswa, @toolybird, and @gromit

added statusconfirmed label and removed statusunconfirmed label

assigned to @seblu and unassigned @freswa, @toolybird, and @gromit

@bugbuddy assign @eworm

assigned to @eworm

Since this also causes breakages with docker network this also might need downstream attention

Yeah, as @gromit said, this causes issue with docker by cascade (as it uses firewalld to create bridged networks and firewall rules if firewalld is installed). The docker.service ends up failing to start because of conflicting bridged network (as it is not able to create them properly). Not sure if this is enough to justify an epoch while waiting to find/fix the issue?

For those that needs docker back after facing this issue:

1 - # pacman -U /var/cache/pacman/pkg/libnftnl-1.2.6-1-x86_64.pkg.tar.zst /var/cache/pacman/pkg/nftables-1\:1.0.9-3-x86_64.pkg.tar.zst
2 - # systemctl restart firewalld.service
3 - # systemctl restart nftables.service
4 - # mv /var/lib/docker/network/files/local-kv.db /var/lib/docker/network/files/local-kv.db.bak
5 - # systemctl restart docker.service

Alternatively, one can use the iptables backend for firewalld (instead of nftables) in the mean time.

Maybe a delete with the docker cli for the network also does the trick? Something like docker network rm bridge? I'm really not liking the rm -rf

Unfortunately you can't use docker cli if the service is not started... So it's not an option unfortunately x)

So lets atleast change it to something like mv /var/lib/docker/network /var/lib/docker/network.bak

/var/lib/docker/network itself is a folder, but maybe mv the db file it contains is enough. Let me try that!

Indeed, that works. Thanks for the suggestion! I edited my initial comment accordingly ;)

It's difficult to confirm just yet but, according to my tests, chances are that the real issue is in libnftnl. I was able to get firewalld and docker back on track just by downgrading libnftnl (instead of both libnftl and nftables).
There is still one specific error in journalctl -u firewalld.service that only disappeared after I also downgraded nftables though, but it might just be because latest nftables malfunctions with older libnftnl?

Forget it, I think I confused myself during tests. Downgrading nftables alone is indeed enough to fix the issue. So nftables v1.1.0 indeed seems to be the cause here. Sorry

I did a git bisection which points to:

e791dbe109b6dd891a63a4236df5dc29d7a4b863 is the first bad commit
commit e791dbe109b6dd891a63a4236df5dc29d7a4b863
Author: Pablo Neira Ayuso <pablo@netfilter.org>
Date:   Mon May 27 13:26:01 2024 +0200

    cache: recycle existing cache with incremental updates

    Cache tracking has improved over time by incrementally adding/deleting
    objects when evaluating commands that are going to be sent to the kernel.

    nft_cache_is_complete() already checks that the cache contains objects
    that are required to handle this batch of commands by comparing cache
    flags.

    Infer from the current generation ID if no other transaction has
    invalidated the existing cache, this allows to skip unnecessary cache
    flush then refill situations which slow down incremental updates.

    Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>

 src/cache.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)

git bisect log:

git bisect start
# status: waiting for both good and bad commits
# good: [49151cd0709acbfb7c6c7f35c44fc12060b695d1] build: Bump version to 1.0.9
git bisect good 49151cd0709acbfb7c6c7f35c44fc12060b695d1
# status: waiting for bad commit, 1 good commit known
# bad: [d946842f576b422972212a2b83601bbd6204337c] build: Bump version to 1.1.0
git bisect bad d946842f576b422972212a2b83601bbd6204337c
# good: [4ee6f7d8c676ca6cd96aabd8dd9b52d90f4127d5] tests: add a test case for double-flush bug in pipapo
git bisect good 4ee6f7d8c676ca6cd96aabd8dd9b52d90f4127d5
# good: [b11b6c68e61ea294eb4c313705ccfe3e7b0eda87] netlink_delinearize: restore binop syntax when listing ruleset for flags
git bisect good b11b6c68e61ea294eb4c313705ccfe3e7b0eda87
# good: [f6b579344eee17e5587b6a7fcc444fe997cd8cb6] evaluate: Fix incorrect checking the `base` variable in case of IPV6
git bisect good f6b579344eee17e5587b6a7fcc444fe997cd8cb6
# bad: [f13fd62ef1e2755ed5eef11adbe0e8a479528f72] tests: py: drop redundant JSON outputs
git bisect bad f13fd62ef1e2755ed5eef11adbe0e8a479528f72
# bad: [c624578ee18131492e7f72b662d5faf7c042e0d8] tests: shell: skip ip option tests if kernel does not support it
git bisect bad c624578ee18131492e7f72b662d5faf7c042e0d8
# bad: [016f37f1268fa1003c46c66655697d3f58d86598] monitor: too large shift exponent displaying payload expression
git bisect bad 016f37f1268fa1003c46c66655697d3f58d86598
# bad: [e791dbe109b6dd891a63a4236df5dc29d7a4b863] cache: recycle existing cache with incremental updates
git bisect bad e791dbe109b6dd891a63a4236df5dc29d7a4b863
# good: [bbb0e944b59d355085e8f50b4b7b5057ae0d33a4] cache: check for NFT_CACHE_REFRESH in current requested cache too
git bisect good bbb0e944b59d355085e8f50b4b7b5057ae0d33a4
# first bad commit: [e791dbe109b6dd891a63a4236df5dc29d7a4b863] cache: recycle existing cache with incremental updates

I posted the git bisection result in the upstream ticket

Maybe you could also edit your comment there saying that reverting the bisected commit does indeed fix the issue

Yup, done

closed with commit f3ca10b4

1.1.0 breaking firewalld

Description:

Additional info:

Child items ...

Activity

Admin message

1.1.0 breaking firewalld

Description:

Additional info:

Activity