Edit 2: Through all of my shenanigans I ended up on a read-only snapshot for root. The error I got just seemed similar to previous out-of-space errors. I went to a later snapshot as default and everything is working great!
My OpenSUSE Tumbleweed is wonky since I last did a dist-upgrade with about 4000 packages. Midway through it errord out with an error that indicated that the filesystem was full althou df showed plenty of free space.
BTRFS seemed to be the culprit. Removing snapshots let me continue the upgrade until it errored out again. Rinse and repeat until it was done.
Edit: My root subvolume is read only. So there must be some error in that. The other subvolumes work correctly. So I guess it isn’t about free space after all.
But now the BTRFS seems to be almost full and I cannot update anymore.
...
Checking for file conflicts: .....................[done]error: can't create transaction lock on /usr/lib/sysimage/rpm/.rpm.lock (Read-only file system) ( 1/40) Removing: ovpn-dco-kmp-default-0.2.202412[error]Removal of (76899)ovpn-dco-kmp-default-0.2.20241216~git0.a08b2fd_k6.13.7_1-2.2.x86_64(@System) failed: Error: Subprocess failed. Error: RPM failed: Command exited with status 1. Abort, retry, ignore? [a/r/i] (a): Problem occurred during or after installation or removal of packages: Installation has been aborted as directed. Please see the above error message for a hint.
I’ve tried a full balance but that didn’t even seem to help. So I suspect that the space is caught up in snapshots, but I can’t delete them.
# snapper list
# │ Type │ Pre # │ Date │ User │ Used Space │ Cleanup │ Description │ Userdata ─────┼────────┼───────┼──────────────────────────────────┼──────┼────────────┼─────────┼───────────────────────┼───────────── 0 │ single │ │ │ root │ │ │ current │ 1 │ single │ │ Thu 18 Apr 2024 05:58:31 PM CEST │ root │ 12.51 GiB │ number │ first root filesystem │365* │ pre │ │ Wed 26 Mar 2025 04:28:33 PM CET │ root │ 16.00 KiB │ number │ zypp(zypper) │ important=no 366 │ pre │ │ Wed 26 Mar 2025 07:28:09 PM CET │ root │ 16.00 KiB │ number │ zypp(zypper) │ important=no 367 │ pre │ │ Wed 26 Mar 2025 07:36:53 PM CET │ root │ 16.00 KiB │ number │ zypp(zypper) │ important=no
# snapper rm 1
Deleting snapshot failed.
# snapper rm 365
Cannot delete snapshot 365 since it is the currently mounted snapshot.
# btrfs filesystem usage /
Overall: Device size: 476.44GiB Device allocated: 389.06GiB Device unallocated: 87.37GiB Device missing: 0.00B Device slack: 3.50KiB Used: 382.53GiB Free (estimated): 90.80GiB (min: 47.12GiB) Free (statfs, df): 90.80GiB Data ratio: 1.00 Metadata ratio: 2.00 Global reserve: 512.00MiB (used: 0.00B) Multiple profiles: no Data,single: Size:381.00GiB, Used:377.57GiB (99.10%) /dev/mapper/cr_root 381.00GiB Metadata,DUP: Size:4.00GiB, Used:2.48GiB (61.97%) /dev/mapper/cr_root 8.00GiB System,DUP: Size:32.00MiB, Used:80.00KiB (0.24%) /dev/mapper/cr_root 64.00MiB Unallocated: /dev/mapper/cr_root 87.37GiB
# btrfs qgroup show /
Qgroupid Referenced Exclusive Path -------- ---------- --------- ---- 0/5 16.00KiB 16.00KiB <toplevel> 0/256 16.00KiB 16.00KiB @ 0/257 14.25GiB 14.25GiB @/var 0/258 16.00KiB 16.00KiB @/usr/local 0/259 16.00KiB 16.00KiB @/srv 0/260 54.32MiB 54.32MiB @/root 0/261 24.09GiB 24.09GiB @/opt 0/262 289.02GiB 288.95GiB @/home 0/263 16.00KiB 16.00KiB @/boot/grub2/x86_64-efi 0/264 16.00KiB 16.00KiB @/boot/grub2/i386-pc 0/265 16.00KiB 16.00KiB @/.snapshots 0/266 24.00GiB 12.51GiB @/.snapshots/1/snapshot 0/473 16.00GiB 16.00GiB @/.snapshots/1/snapshot/swap 0/657 23.68GiB 16.00KiB @/.snapshots/365/snapshot 0/661 23.68GiB 16.00KiB @/.snapshots/366/snapshot 0/662 23.68GiB 16.00KiB @/.snapshots/367/snapshot 1/0 36.19GiB 36.12GiB <0 member qgroups>
Any tips?
No real advice but I’ve heard of people having issues with their BTRFS filesystem running out of free inodes and reporting the filesystem as full due to that. Note that the df command is not expected to work properly for a BTRFS filesystem.
Are you sure the right partitions have enough space? On my tumbleweed the installer made several subvolumes i think
If I understand subvolumes correctly they share their space when they reside on the same device. I only have two partitions. One for
/boot/efiand one for the rest.You are actually kinda right. My error is that
/is read only. The other subvolumes are writable. No idea why or how I can find out why.
This sounds like you’ll need to do a balance operation. Try this first and see if it helps:
btrfs balance start -dusage=0 -musage=0 /If not you can increase the number to 5 or 10. This operation reallocates chunks on the disk and ensures they’re filled - check https://btrfs.readthedocs.io/en/latest/Balance.html for details.
I did that. Turns out my error had nothing to do with space. I was just booting into a read-only snapshot by default. Rolling back to a later snapshot fixed everything.
Weird, but good that you found out!
Yeah, another good example of the XY-Problem.
I always do a df -h before updating. And this recent update was brutally large. I was unable to free enough space by deleting snapshots as I’ve taken to doing lately. What I’ve now started doing is finding directories using a lot of space and moving them from my root partition to another partition I have with more space and then linking that directory back to a directory with the original path. The most recent culprit I found and moved is /var/lib/flatpack. The program filelight is a great tool for finding culprits.
