Skip to content

BTRFS spin locks way too much when accessing many files concurrently

Description:

Accessing many files concurrently at once causes BTRFS to spin lock too much and basically lock up all other processes for seconds at a time.

Mostly happens when accessing their content, not the other metadata.

Additional info:

Steps to reproduce:

  1. Collect a lot of files / file content in BTRFS (In my test I had 1,425,292 files comprising 424GB)
  2. Run rg anything-here ./folder, or any other highly-parallel file content consuming command, e.g. I also encountered this with restic during backups.
  3. The system slows to an absolute crawl.
Edited by Octavia Togami
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information