Read-only filesystem error when running sudo timeshift --check

[kekkon]

Any hints on what to look for in dmesg? Or how to filter out the errors from the noise?

The check reported

╰─λ sudo btrfs check /dev/nvme0n1
Opening filesystem to check...
No valid Btrfs found on /dev/nvme0n1
ERROR: cannot open file system

I am not sure why you'd want me to check that device though, since that's the SSD itself, not a specific partition.

EDIT: this is what dmesg had to say about btrfs:

[✔️] ╰─λ  sudo dmesg | grep -i btrfs
[    3.745425] Btrfs loaded, crc32c=crc32c-intel, zoned=yes
[    3.745847] BTRFS: device fsid a6eae74a-2326-42b3-af7d-a0ea0be07a31 devid 1 transid 8312 /dev/nvme0n1p3 scanned by systemd-udevd (271)
[    4.000588] BTRFS info (device nvme0n1p3): disk space caching is enabled
[    4.000591] BTRFS info (device nvme0n1p3): has skinny extents
[    4.024458] BTRFS info (device nvme0n1p3): enabling ssd optimizations
[    6.461024] BTRFS info (device nvme0n1p3): enabling auto defrag
[    6.461028] BTRFS info (device nvme0n1p3): use zstd compression, level 3
[    6.461029] BTRFS info (device nvme0n1p3): disk space caching is enabled
[  269.291091] audit: type=1130 audit(1631957262.086:135): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=grub-btrfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  269.291096] audit: type=1131 audit(1631957262.086:136): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=grub-btrfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  608.376346] audit: type=1130 audit(1631957601.171:197): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=grub-btrfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  608.376351] audit: type=1131 audit(1631957601.171:198): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=grub-btrfs comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

[filo]

Btrfs check must be launched from a live USB, with target unmounted.
The dmesg seems ok. Please check
sudo btrfs check /dev/nvme0n1p3

[kekkon]

This is what I got back after the check:

╰─λ sudo btrfs check /dev/nvme0n1p3
Opening filesystem to check...
Checking filesystem on /dev/nvme0n1p3
UUID: a6eae74a-2326-42b3-af7d-a0ea0be07a31
[1/7] checking root items
[2/7] checking extents
[3/7] checking free space cache
[4/7] checking fs roots
[5/7] checking only csums items (without verifying data)
[6/7] checking root refs
[7/7] checking quota groups
Counts for qgroup id: 0/465 are different
our:            referenced 93389197312 referenced compressed 93389197312
disk:           referenced 93389197312 referenced compressed 93389197312
our:            exclusive 325410816 exclusive compressed 325410816
disk:           exclusive 325349376 exclusive compressed 325349376
diff:           exclusive 61440 exclusive compressed 61440
found 134934302720 bytes used, error(s) found
total csum bytes: 128034596
total tree bytes: 1554268160
total fs tree bytes: 1293139968
total extent tree bytes: 105922560
btree space waste bytes: 273923387
file data blocks allocated: 178626359296
referenced 234386849792

[tbg]

Might be worth deleting all your snapshots, then do a btrfs balance.

After completion create a new snapshot, then reboot.

Then check to see if the same errors are still present.

[kekkon]

I have done that, but sudo timeshift --check still keeps throwing errors about not being able to remove the files in the snapshots. The rest of the system works fine, so I think this is specifically something with timeshift.

sudo timeshift --list-devices gives me

/dev/nvme0n1p3 is mounted at: /run/timeshift/backup, options: rw,relatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=5,subvol=/

but that is also my root partition. Can it be mounted twice?

EDIT: my mtab

proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
sys /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
dev /dev devtmpfs rw,nosuid,relatime,size=16373272k,nr_inodes=4093318,mode=755,inode64 0 0
run /run tmpfs rw,nosuid,nodev,relatime,mode=755,inode64 0 0
efivarfs /sys/firmware/efi/efivars efivarfs rw,nosuid,nodev,noexec,relatime 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
/dev/nvme0n1p3 / btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=256,subvol=/@ 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0
cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
none /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16535 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
binder /dev/binderfs binder rw,relatime,max=1048576 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0
/dev/nvme0n1p3 /root btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=258,subvol=/@root 0 0
/dev/nvme0n1p3 /home btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=257,subvol=/@home 0 0
/dev/nvme0n1p3 /srv btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=259,subvol=/@srv 0 0
/dev/nvme0n1p3 /var/cache btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=260,subvol=/@cache 0 0
/dev/nvme0n1p3 /var/log btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=261,subvol=/@log 0 0
/dev/nvme0n1p3 /var/tmp btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=262,subvol=/@tmp 0 0
tmpfs /home/guest tmpfs rw,nosuid,nodev,mode=750,uid=1001,gid=984,inode64 0 0
tmpfs /tmp tmpfs rw,nosuid,nodev,size=16405996k,nr_inodes=409600,inode64 0 0
/dev/nvme0n1p1 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro 0 0
/dev/sdb2 /mnt/data fuseblk rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other,blksize=4096 0 0
/dev/loop0 /var/lib/anbox/rootfs squashfs ro,relatime,errors=continue 0 0
/dev/nvme0n1p3 /var/lib/anbox/rootfs/cache btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=256,subvol=/@ 0 0
/dev/nvme0n1p3 /var/lib/anbox/rootfs/data btrfs rw,noatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=256,subvol=/@ 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3281196k,nr_inodes=820299,mode=700,uid=1000,gid=1000,inode64 0 0
/dev/nvme0n1p3 /run/timeshift/backup btrfs rw,relatime,compress=zstd:3,ssd,space_cache,autodefrag,subvolid=5,subvol=/ 0 0

[filo]

I think
/dev/nvme0n1p3 is mounted at: /run/timeshift/backup
is normal when timeshift is working or the gui open.
Just a dumb idea: if you're dualbooting with Windows, could you please check that fast boot is disabled (sometimes Windows updates turn it on automatically)?

[kekkon]

It was turned on, but the problem persists even after disabling fastboot.

[dalto]

The timeshift package was recently patched to create read-only snapshots. Has anyone else tested this on a snapshot taken with the latest version of timeshift to see if this is a side-effect of that patch?

[kekkon]

That would explain it. Seems like it's doing that by default and doesn't know how to delete them again.

[kekkon]

Any idea on how to fix this? I'm willing to be a test person for a fix. I just tried going to the directory and doing sudo rm -rf ./* but ofcourse I still get the same error.

In mtab the timeshift directory is mounted as rw, so I don't understand why it's saying that it's mounted as read only, but I am very new to BTRFS.

[filo]

I haven't done anything like this before, but it is easily rollable back.
I see in fstab all the file systems are mount as read only (isn't this the final 0 on each line?)
Maybe you could fix that from a live USB and reboot.
I think 1 for / and 2 for the others.

[kekkon]

No, the last two numbers are dump and pass parameters (from Wikipedia):

• dump – A number indicating whether and how often the file system should be backed up by the dump program; a zero indicates the file system will never be automatically backed up.
• pass – A number indicating the order in which the fsck program will check the devices for errors at boot time:

read-write or read-only should be ro, or rw in the options line. But neither are specified. Also, the rest of the file system is read-write. Just the snapshots directory from timeshift isn't.

[filo]

Yes i was just posting the same after reading the man page, sorry...
Although I don't know the reason for those 0's.

[kekkon]

That's how Garuda set them up on install. I just followed the installer.

[filo]

I have 1 for / and 2 for the others, and the same I saw checking a few fstab in other posts. Although I see now in the Arch wiki that for btrfs they should be 0.
Never mind, I tried, let's wait for a dev...

[kekkon]

Mind you, the read only part is not in fstab, it is only in mtab, so that directory is mounted AFTER boot and is probably part of the BTRFS metadata somehow, it's a separate subvolume.

When I run sudo btrfs subvolume list -r / I get the following output:

ID 372 gen 1731 top level 5 path timeshift-btrfs/snapshots/2021-09-13_08-14-13/@
ID 373 gen 1733 top level 5 path timeshift-btrfs/snapshots/2021-09-13_08-14-13/@home
ID 399 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_08-00-01/@
ID 400 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_08-00-01/@home
ID 401 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_08-01-02/@
ID 402 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_08-01-02/@home
ID 446 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_17-42-34/@
ID 447 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_17-42-34/@home
ID 448 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_17-50-17/@
ID 449 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_17-50-17/@home
ID 450 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_18-09-17/@
ID 451 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_18-09-17/@home
ID 452 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_18-10-56/@
ID 453 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_18-10-56/@home
ID 454 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_20-00-19/@
ID 455 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_20-00-19/@home
ID 456 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_23-06-17/@
ID 457 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-14_23-06-17/@home
ID 458 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-15_18-25-16/@
ID 459 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-15_18-25-16/@home
ID 460 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-15_18-34-38/@
ID 461 gen 8530 top level 5 path timeshift-btrfs/snapshots/2021-09-15_18-34-38/@home
ID 494 gen 8535 top level 5 path timeshift-btrfs/snapshots/2021-09-18_12-36-40/@
ID 495 gen 8537 top level 5 path timeshift-btrfs/snapshots/2021-09-18_12-36-40/@home
ID 496 gen 8558 top level 5 path timeshift-btrfs/snapshots/2021-09-18_12-47-59/@
ID 497 gen 8560 top level 5 path timeshift-btrfs/snapshots/2021-09-18_12-47-59/@home
ID 500 gen 8747 top level 5 path timeshift-btrfs/snapshots/2021-09-18_14-07-31/@
ID 501 gen 8749 top level 5 path timeshift-btrfs/snapshots/2021-09-18_14-07-31/@home
ID 502 gen 8771 top level 5 path timeshift-btrfs/snapshots/2021-09-18_14-16-57/@
ID 503 gen 8773 top level 5 path timeshift-btrfs/snapshots/2021-09-18_14-16-57/@home
ID 505 gen 8899 top level 5 path timeshift-btrfs/snapshots/2021-09-18_17-32-25/@
ID 506 gen 8901 top level 5 path timeshift-btrfs/snapshots/2021-09-18_17-32-25/@home
ID 508 gen 9042 top level 5 path timeshift-btrfs/snapshots/2021-09-18_18-32-24/@
ID 509 gen 9044 top level 5 path timeshift-btrfs/snapshots/2021-09-18_18-32-24/@home
ID 513 gen 9560 top level 5 path timeshift-btrfs/snapshots/2021-09-19_09-28-37/@
ID 514 gen 9562 top level 5 path timeshift-btrfs/snapshots/2021-09-19_09-28-37/@home
ID 515 gen 9573 top level 5 path timeshift-btrfs/snapshots/2021-09-19_09-32-34/@
ID 516 gen 9575 top level 5 path timeshift-btrfs/snapshots/2021-09-19_09-32-34/@home

Those are the snapshots that timeshift can't delete. So the question really is: how do I file a bug for Timeshift and how do I deleade read only btrfs snapshots?

EDIT: I ended up deleting all the read only snapshots with the following command:

for id in (sudo btrfs subvolume list -r /run/timeshift/backup/ | awk '{ print $2 }'); sudo btrfs subvolume delete -i $id /run/timeshift/backup; end

After that I ran sudo timeshift --check and it ran normally.

[dalto]

You deleted all the read-only snapshots but won't the future snapshots also be read-only as well?

[kekkon]

I am testing that right now. Maybe it was a temporary issue. Even if it's not, I could add my delete script to the pacman hook for timeshift and cobble together a temporary fix that way.

[kekkon]

FYI, it seems that the read-only snapshots haven't returned so far, but I am still clueless as to what produced them to begin with.