BTRFS spin locks way too much when accessing many files concurrently

Description:

Accessing many files concurrently at once causes BTRFS to spin lock too much and basically lock up all other processes for seconds at a time.

Mostly happens when accessing their content, not the other metadata.

Additional info:

• package version(s): linux 6.10.1-arch1-1
• config and/or log files: I collected a perf report showing the problem, using sudo perf record -F 99 --call-graph dwarf,16384 -g -m 32M -- rg uncommon-phrase ~/Documents: perf.data, flamegraph.svg
• link to upstream bug report, if any: https://lore.kernel.org/linux-btrfs/CAHPNGSSt-a4ZZWrtJdVyYnJFscFjP9S7rMcvEMaNSpR556DdLA@mail.gmail.com/T/#u

Steps to reproduce:

1. Collect a lot of files / file content in BTRFS (In my test I had 1,425,292 files comprising 424GB)
2. Run rg anything-here ./folder, or any other highly-parallel file content consuming command, e.g. I also encountered this with restic during backups.
3. The system slows to an absolute crawl.

added priority3-normal severity4-low statusunconfirmed labels

assigned to @freswa, @anthraxx, @toolybird, and @gromit

I was unsure if upstream would accept my report

I see no reason not to "accept" a sane detailed report. Posting it here doesn't help anyway, as it is clearly an upstream kernel thing.

So better report it upstream https://bugzilla.kernel.org/

Is this maybe this issue? https://lore.kernel.org/all/CABXGCsMmmb36ym8hVNGTiU8yfUS_cGvoUmGCcBrGWq9OxTrs+A@mail.gmail.com/

If yes then it could be possibly fixed by https://lore.kernel.org/all/cover.1720448663.git.fdmanana@suse.com/

added scoperegression label

added statuswaiting-input label and removed statusunconfirmed label

assigned to @heftig, @dvzrv, and @tpowa and unassigned @freswa, @toolybird, and @gromit

changed the description

Patch cherry-picked into linux 6.10.4.arch2-1.

closed

added resolutioncompleted label