[root@garuda-gnome /]# findmnt --real
TARGET SOURCE FSTYPE OPTIONS
/ /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216[/@]
| btrfs rw,relatime,ssd,space_cache=v2,subvolid=256,subvol=/@
`-/boot/efi
/dev/nvme1n1p1
vfat rw,noatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,error
[root@garuda-gnome /]#
Something is wrong with your system. There should be way more mounts than that. For some reason you are not getting all the mounts in your fstab.
When you examine the files on the disk, are they intact?
The files are yes, but running any software is a no no for some reason. Firedragon won’t run, the default video player Lollypop I think it is won’t run, pictures work, that’s all. I tried btrfs/snapper, restores won’t work, barely anything does.
I only updated it the same way I usually did so whatever happened has messed it up big time.
From the chroot, let’s see this output:
btrfs subvolume list / | grep "top level 5 "
Fresh day for me, sorry for delay.
╭─garuda@garuda in ~ as 🧙 took 12ms
╰─λ garuda-chroot -a
==> Mounting (Garuda_Linux) [/dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216]
--> mount: [/mnt]
--> mount: [/mnt/boot/efi]
[root@garuda-gnome /]# btrfs subvolume list / | grep "top level 5 "
ID 256 gen 220347 top level 5 path @
ID 257 gen 221625 top level 5 path @home
ID 258 gen 220329 top level 5 path @root
ID 259 gen 133923 top level 5 path @srv
ID 260 gen 220329 top level 5 path @cache
ID 261 gen 220713 top level 5 path @log
ID 262 gen 220254 top level 5 path @tmp
ID 681 gen 133836 top level 5 path @_backup_20241108140647380
ID 682 gen 133811 top level 5 path @_backup_20241908145646908
[root@garuda-gnome /]#
All your subvolumes are there, but for some reason the garuda-chroot
tool is not correctly setting up the mounts for them. You will need to mount them manually and just use arch-chroot
instead.
Exit out of the chroot if you are still in it, then set it up by specifying each mount point like this:
sudo mount -o subvol=@ /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt
sudo mount -o subvol=@home /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/home
sudo mount -o subvol=@root /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/root
sudo mount -o subvol=@srv /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/srv
sudo mount -o subvol=@cache /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/cache
sudo mount -o subvol=@log /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/log
sudo mount -o subvol=@tmp /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/tmp
sudo mount /dev/nvme1n1p1 /mnt/boot/efi
arch-chroot /mnt
Once you are inside the chroot try updating your system again.
Do I just do garuda-update now?
╭─garuda@garuda in ~ as 🧙 took 3ms
[🔴] × sudo mount -o subvol=@ /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount -o subvol=@home /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/home
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount -o subvol=@root /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/root
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount -o subvol=@srv /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/srv
╭─garuda@garuda in ~ as 🧙 took 14ms
╰─λ sudo mount -o subvol=@cache /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/cache
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount -o subvol=@log /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/log
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount -o subvol=@tmp /dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 /mnt/var/tmp
╭─garuda@garuda in ~ as 🧙 took 13ms
╰─λ sudo mount /dev/nvme1n1p1 /mnt/boot/efi
╭─garuda@garuda in ~ as 🧙 took 34ms
╰─λ arch-chroot /mnt
==> ERROR: This script must be run with root privileges
╭─garuda@garuda in ~ as 🧙 took 3ms
[🔴] × sudo arch-chroot /mnt
[root@garuda-gnome /]#
I attempted the update and got this:
╭─garuda@garuda in ~ as 🧙 took 3ms
[🔴] × sudo arch-chroot /mnt
[root@garuda-gnome /]# garuda-update
:: Synchronizing package databases...
error: failed to synchronize all databases (unable to lock database)
:: Synchronizing package databases...
error: failed to synchronize all databases (unable to lock database)
/usr/lib/garuda/garuda-update/main-update: line 10: echo: write error: No space left on device
[root@garuda-gnome /]#
Let’s take a closer look:
df -i
btrfs filesystem df /
btrfs filesystem usage /
Let’s also try deleting the package cache and see if it works now that all the subvolumes are mounted.
paccache -rk0
df -i
[root@garuda-gnome /]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/dm-0 0 0 0 - /
/dev/dm-0 0 0 0 - /home
/dev/dm-0 0 0 0 - /root
/dev/dm-0 0 0 0 - /srv
/dev/dm-0 0 0 0 - /var/cache
/dev/dm-0 0 0 0 - /var/log
/dev/dm-0 0 0 0 - /var/tmp
/dev/nvme1n1p1 0 0 0 - /boot/efi
efivarfs 0 0 0 - /sys/firmware/efi/efivars
udev 3981376 921 3980455 1% /dev
shm 3992139 1 3992138 1% /dev/shm
run 3992139 1 3992138 1% /run
tmp 3992139 2 3992137 1% /tmp
overlay 3992139 3941 3988198 1% /etc/resolv.conf
btrfs filesystem df /
[root@garuda-gnome /]# btrfs filesystem df /
Data, single: total=459.34GiB, used=417.58GiB
System, DUP: total=32.00MiB, used=80.00KiB
Metadata, DUP: total=3.03GiB, used=2.53GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
btrfs filesystem usage /
[root@garuda-gnome /]# btrfs filesystem usage /
Overall:
Device size: 465.46GiB
Device allocated: 465.46GiB
Device unallocated: 1.03MiB
Device missing: 0.00B
Device slack: 512.00B
Used: 422.65GiB
Free (estimated): 41.75GiB (min: 41.75GiB)
Free (statfs, df): 41.75GiB
Data ratio: 1.00
Metadata ratio: 2.00
Global reserve: 512.00MiB (used: 0.00B)
Multiple profiles: no
Data,single: Size:459.34GiB, Used:417.58GiB (90.91%)
/dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 459.34GiB
Metadata,DUP: Size:3.03GiB, Used:2.53GiB (83.51%)
/dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 6.06GiB
System,DUP: Size:32.00MiB, Used:80.00KiB (0.24%)
/dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 64.00MiB
Unallocated:
/dev/mapper/luks-f2314da2-7bb3-4bfa-83ba-9a58d102e216 1.03MiB
paccache -rk0
==> finished: 4195 packages removed (disk space saved: 13.42 GiB)
[root@garuda-gnome /]#
That’s a good sign at least. Let’s try the update again.
Man, this truly hates me.
[root@garuda-gnome /]# garuda-update
:: Synchronizing package databases...
garuda 64.2 KiB 128 KiB/s 00:01 [------------------------------------] 100%
core 118.7 KiB 333 KiB/s 00:00 [------------------------------------] 100%
extra 7.3 MiB 3.63 MiB/s 00:02 [------------------------------------] 100%
multilib 136.9 KiB 343 KiB/s 00:00 [------------------------------------] 100%
chaotic-aur 753.5 KiB 1288 KiB/s 00:01 [------------------------------------] 100%
--> Refreshing mirrorlists using reflector, please be patient..🍵
[2024-08-28 17:36:51] WARNING: failed to rate http(s) download (https://mirror.theash.xyz/arch/extra/os/x86_64/extra.db): HTTP Error 403: Forbidden
:: Synchronizing package databases...
garuda downloading...
core downloading...
extra downloading...
multilib downloading...
chaotic-aur downloading...
spawn pacman -Su
error: failed to init transaction (unable to lock database)
error: could not lock database: No space left on device
[root@garuda-gnome /]# sed: couldn't flush stdout: No space left on device
Even though there is technically free space left on the device, Btrfs can’t use it because there isn’t enough contiguous free space or unallocated chunks left. Here you can see there is only 1.03 MiB of unallocated space:
Open Btrfs Assistant and balance the filesystem using the tool on the first window when it opens. It’s possible it will take a while to run.
It would be good to delete unneeded snapshots while you are in there as well.
I did just that, it did something for about 0.5 seconds and then went back to:
“No balance running.”
If I delete snapshots will it delete all my stuff? I’ll have to be cautious with that as I’ve never used that on Linux before.
Hmm, it’s not normally known for being a fast operation.
It will delete previous versions of your stuff. Each snapshot “remembers” the state your filesystem was in at the moment the snapshot was taken. It allows rolling back the system, grabbing a previous version of a file or a file which has been deleted, etc.
They are useful but take up space. In your case the lack of filesystem space is creating an issue so I think it will be in your best interest to lose a few snapshots if you can. Maybe keep the three most recent and delete some old ones to see if that takes the curse off.
Yeah it does it in like 0.5 seconds, so not sure what that’s about. To get that screenshot of it doing something was tricky.
As for the snapshots, I’ll follow your advice.
This is the before:
This is the after:
Good, hopefully that will free up some space.
Let’s try running a full balance from the command line like this:
btrfs balance start --full-balance /
I’m at a bit of a loss what to do really, the snapshots didn’t do anything. Do I have to free up some space on the drive it’s self? If so, I think I can do that. Or if you tell me where I need to go or a command to put in to free up space, I can also do that. I’m not worried about losing any programs, just thing’s like documents, pictures and videos etc.
[root@garuda-gnome /]# btrfs balance start --full-balance /
ERROR: error during balancing '/': No space left on device
There may be more info in syslog - try dmesg | tail
[root@garuda-gnome /]#
in terminal, maybe we see what’s going on.
[root@garuda-gnome /]# dmesg | tail
[ 2385.288791] BTRFS info (device dm-0): balance: start -d -m -s
[ 2385.289014] BTRFS info (device dm-0): relocating block group 515230400512 flags data
[ 2386.692062] BTRFS info (device dm-0): 470 enospc errors during balance
[ 2386.692064] BTRFS info (device dm-0): balance: ended with status: -28
[ 2804.117093] BTRFS info (device dm-0): balance: start -d -m -s
[ 2805.800395] BTRFS info (device dm-0): 470 enospc errors during balance
[ 2805.800397] BTRFS info (device dm-0): balance: ended with status: -28
[ 2813.956414] BTRFS info (device dm-0): balance: start -d -m -s
[ 2815.639094] BTRFS info (device dm-0): 470 enospc errors during balance
[ 2815.639097] BTRFS info (device dm-0): balance: ended with status: -28
[root@garuda-gnome /]#
Commonly the snapshots take up very little space.
Yes, if that is an option it would be helpful. For example if you can uninstall some applications, at least for now until the filesystem can be rebalanced.