BTRFS spin locks way too much when accessing many files concurrently

Description:

Accessing many files concurrently at once causes BTRFS to spin lock too much and basically lock up all other processes for seconds at a time.

Mostly happens when accessing their content, not the other metadata.

Additional info:

Steps to reproduce:

  1. Collect a lot of files / file content in BTRFS (In my test I had 1,425,292 files comprising 424GB)
  2. Run rg anything-here ./folder, or any other highly-parallel file content consuming command, e.g. I also encountered this with restic during backups.
  3. The system slows to an absolute crawl.
Edited by Octavia Togami