Cannot boot anymore (root full, snapshot not working)

Hi there,

excuse me if I cannot post my garuda-inxi and have to resort to screenshots, but since I cannot boot I’m afraid this is the best I can do. :cry:
I couldn’t update since my root partition (50gb) was apparently full.
So I deleted some snapshots, cleared cache and orphans via the Garuda tools, then tried rebooting and… I cannot boot anymore, and the five snapshot I have do not work as well!

This is what I see if I try to boot linux-zen or linux-lts (I have both installed, keeping LTS as a fallback):

And this if I try to boot any snapshot:

Can I do something or am I simply done? :frowning:
I have a separate /home partition, in case it’s useful for recovering data.
Thanks in advance! :pray:

P.S. btw would it be possible to resize the root partition?

Boot from a live USB, muont your system partition and free-up/move as much space as possible, if this is really the problem.
From the live USB, once the system is mounted, you should also be able to access your snapshots in the Btrfs Assistant and delete some old snapshots.
If that’s not enough, you should consider chrooting from the live USB into your system and, having made some space, you should be able to update or just reinstall the kernel, if that’s the problem.
Before updating, you could free up a lot of space with:

sudo pacman -Scc

(well sudo is not needed when inside a chroot)
Ops, I just noticed:

You could try chrooting anyway and updating, now that the disk space should be available.

Nope.

and post the garuda-inxi.

1 Like

I thought that in the live system the inxi would have been related to, well, the live system.
It seems not, so here it is:

System:
Kernel: 6.2.13-zen-1-zen arch: x86_64 bits: 64 compiler: gcc v: 12.2.1
parameters: BOOT_IMAGE=/boot/vmlinuz-x86_64 lang=en_US keytable=it tz=UTC
misobasedir=garuda root=miso:LABEL=GARUDA_DR460NIZED_RAPTOR quiet
systemd.show_status=1 ibt=off driver=free nouveau.modeset=1
i915.modeset=1 radeon.modeset=1
Desktop: KDE Plasma v: 5.27.4 tk: Qt v: 5.15.9 wm: kwin_x11 vt: 1 dm: SDDM
Distro: Garuda Linux base: Arch Linux
Machine:
Type: Desktop Mobo: ASRock model: P67 Extreme4 serial: <superuser required>
BIOS: American Megatrends v: P3.10 date: 04/24/2012
Battery:
Device-1: hidpp_battery_0 model: Logitech G604 Wireless Gaming Mouse
serial: <filter> charge: 100% (should be ignored) rechargeable: yes
status: discharging
CPU:
Info: model: Intel Core i5-2500K bits: 64 type: MCP arch: Sandy Bridge
gen: core 2 level: v2 built: 2010-12 process: Intel 32nm family: 6
model-id: 0x2A (42) stepping: 7 microcode: 0x2F
Topology: cpus: 1x cores: 4 smt: <unsupported> cache: L1: 256 KiB
desc: d-4x32 KiB; i-4x32 KiB L2: 1024 KiB desc: 4x256 KiB L3: 6 MiB
desc: 1x6 MiB
Speed (MHz): avg: 2995 high: 4390 min/max: 1600/5900 scaling:
driver: intel_cpufreq governor: schedutil cores: 1: 4390 2: 1600 3: 1600
4: 4390 bogomips: 26340
Flags: avx ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
Vulnerabilities: <filter>
Graphics:
Device-1: NVIDIA GM204 [GeForce GTX 970] vendor: ZOTAC driver: nouveau
v: kernel non-free: 530.xx+ status: current (as of 2023-03) arch: Maxwell
code: GMxxx process: TSMC 28nm built: 2014-19 pcie: gen: 1 speed: 2.5 GT/s
lanes: 16 link-max: gen: 3 speed: 8 GT/s ports: active: DP-1,DVI-I-1
empty: DVI-D-1,HDMI-A-1 bus-ID: 01:00.0 chip-ID: 10de:13c2 class-ID: 0300
temp: 33.0 C
Display: x11 server: X.Org v: 21.1.8 with: Xwayland v: 23.1.1
compositor: kwin_x11 driver: X: loaded: nouveau unloaded: modesetting
alternate: fbdev,nv,vesa dri: nouveau gpu: nouveau display-ID: :0
screens: 1
Screen-1: 0 s-res: 1920x1080 s-dpi: 96 s-size: 506x285mm (19.92x11.22")
s-diag: 581mm (22.86")
Monitor-1: DP-1 model: Panasonic Panasonic-TV serial: <filter> built: 2010
res: N/A gamma: 1.2 size: 698x392mm (27.48x15.43") modes: max: 1920x1080
min: 640x480
Monitor-2: DVI-I-1 pos: primary model: LG (GoldStar) W2363D
serial: <filter> built: 2011 res: 1920x1080 hz: 60 dpi: 96 gamma: 1.2
size: 510x287mm (20.08x11.3") diag: 582mm (22.9") ratio: 16:9 modes:
max: 1920x1080 min: 720x400
API: OpenGL v: 4.3 Mesa 23.0.3 renderer: NV124 direct-render: Yes
Audio:
Device-1: Intel 6 Series/C200 Series Family High Definition Audio
vendor: ASRock driver: snd_hda_intel bus-ID: 1-1.2:4 v: kernel
bus-ID: 00:1b.0 chip-ID: 046d:0a4d class-ID: 0300 chip-ID: 8086:1c20
class-ID: 0403
Device-2: NVIDIA GM204 High Definition Audio vendor: ZOTAC
driver: snd_hda_intel v: kernel pcie: gen: 1 speed: 2.5 GT/s lanes: 16
link-max: gen: 3 speed: 8 GT/s bus-ID: 01:00.1 chip-ID: 10de:0fbb
class-ID: 0403
Device-3: Logitech G430 Surround Sound Gaming Headset type: USB
driver: hid-generic,snd-usb-audio,usbhid
Device-4: M-Audio AIR 192 6 type: USB driver: snd-usb-audio
bus-ID: 2-1.5:3 chip-ID: 0763:410c class-ID: fe01
API: ALSA v: k6.2.13-zen-1-zen status: kernel-api tools: N/A
Server-1: PipeWire v: 0.3.70 status: active with: 1: pipewire-pulse
status: active 2: wireplumber status: active 3: pipewire-alsa type: plugin
4: pw-jack type: plugin tools: pactl,pw-cat,pw-cli,wpctl
Network:
Device-1: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet
vendor: ASRock driver: r8169 v: kernel pcie: gen: 1 speed: 2.5 GT/s lanes: 1
port: b000 bus-ID: 0c:00.0 chip-ID: 10ec:8168 class-ID: 0200
IF: enp12s0 state: up speed: 1000 Mbps duplex: full mac: <filter>
Device-2: Realtek RTL88x2bu [AC1200 Techkey] type: USB driver: rtw_8822bu
bus-ID: 2-1.6:4 chip-ID: 0bda:b812 class-ID: 0000 serial: <filter>
IF: wlp0s26u1u6 state: down mac: <filter>
Bluetooth:
Device-1: Cambridge Silicon Radio Bluetooth Dongle (HCI mode) type: USB
driver: btusb v: 0.8 bus-ID: 1-1.3:5 chip-ID: 0a12:0001 class-ID: e001
Report: bt-adapter ID: hci0 rfk-id: 0 state: up address: <filter>
Drives:
Local Storage: total: 2.85 TiB used: 45.34 GiB (1.6%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/sda maj-min: 8:0 vendor: Samsung model: SSD 860 EVO 1TB
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 3B6Q scheme: MBR
ID-2: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST2000DX001-1NS164
size: 1.82 TiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
type: HDD rpm: 7200 serial: <filter> rev: CC41 scheme: MBR
ID-3: /dev/sdc maj-min: 8:32 vendor: Crucial model: M4-CT128M4SSD2
size: 119.24 GiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
type: SSD serial: <filter> rev: 070H scheme: MBR
ID-4: /dev/sdd maj-min: 8:48 type: USB model: USB Flash Disk
size: 3.75 GiB block-size: physical: 512 B logical: 512 B type: SSD
serial: <filter> rev: 1100 scheme: MBR
SMART Message: Unknown USB bridge. Flash drive/Unsupported enclosure?
Partition:
Message: No partition data found.
Swap:
Kernel: swappiness: 133 (default 60) cache-pressure: 100 (default)
ID-1: swap-1 type: zram size: 15.58 GiB used: 0 KiB (0.0%) priority: 100
dev: /dev/zram0
Sensors:
System Temperatures: cpu: 49.0 C mobo: N/A gpu: nouveau temp: 33.0 C
Fan Speeds (RPM): N/A gpu: nouveau fan: 1122
Info:
Processes: 215 Uptime: 4m wakeups: 2 Memory: 15.58 GiB
used: 2.33 GiB (15.0%) Init: systemd v: 253 default: graphical
tool: systemctl Compilers: gcc: 12.2.1 Packages: pm: pacman pkgs: 1200
libs: 327 tools: octopi,paru Shell: fish v: 3.6.1 default: Bash v: 5.1.16
running-in: konsole inxi: 3.3.26
warning: database file for 'garuda' does not exist (use '-Sy' to download)
warning: database file for 'core' does not exist (use '-Sy' to download)
warning: database file for 'extra' does not exist (use '-Sy' to download)
warning: database file for 'community' does not exist (use '-Sy' to download)
warning: database file for 'multilib' does not exist (use '-Sy' to download)
warning: database file for 'chaotic-aur' does not exist (use '-Sy' to download)
Garuda (2.6.16-1):
System install date:     2024-04-12
Last full system update: 2024-04-12 ↻
Is partially upgraded:   No
Relevant software:       snapper NetworkManager dracut
Windows dual boot:       <superuser required>
Failed units:

I suppose I could try… could you please tell me how to do that correctly? I only chrooted once and it was following exact instructions. I do not really know what I’m doing and wouldn’t want to do any more damage.
Thank you both, @filo and @SGS! :pray:

In the meantime I recovered the old instructions that @NaN kindly gave me at the time and tried garuda-chroot -a, but:

╭─garuda@garuda in ~ as 🧙 took 48ms
[🔴] × garuda-chroot -a
mount: /var/lib/os-prober/mount: mount(2) system call failed: No such file or directory.
dmesg(1) may have more information after failed mount system call.
grub-probe: error: cannot find a GRUB drive for /dev/sdd1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sdd1.  Check your device.map.
==> ERROR: No Linux partitions detected!

The root partition is indeed mounted via Dolphin, i can navigate it. :man_shrugging:

and from your garuda-inxi (07.04.2024):

ID-1: / raw-size: 50 GiB size: 46.57 GiB (93.13%) used: 42.7 GiB (91.7%)
fs: btrfs dev: /dev/sda2 maj-min: 8:2

Personally, I wouldn’t even try to repair anything.

Make a backup of /home.
Repartition the SSD and give the root partition about 100GB.
Do a fresh install.

3 Likes

Thank you @nepti, I’m just waiting to check if this is indeed the only real solution.
That would be really sad, given the long time spent in installing, fixing, tuning, adapting the system.

If I’ll need to reinstall:

  1. is it worth it to keep / and /home separate? or is it an unnecessary complication? (if you do not install the exact same version of the exact same distro maybe keeping the /home intact would be a problem anyway)
  2. the / must be at least 100Gb? (pretty huge, on my dual boot system)

In the end, I think maybe it is my fault: I’m a real noob in the Linux world and maybe choosing an Arch base rolling release wasn’t the wisest decision, even if I’m loving the Garuda approach (rolling, but assisted).
Maybe if I’ll have to reinstall I’ll give Mint or Fedora or Zorin a chance, to see if they are more suited to this noob stage of my Linux life. :slight_smile:
But I’m still hoping to remain on Garuda! :crossed_fingers:

In case you still want to go the chrooting route, there is a button in the Garuda Welcome on the live USB to do that.
Otherwise, detailed instructions to do it manually are provided here:

I see from your previous inxi that your /dev/sda has a MBR, so you should follow the relevant instructions in the tutorial if you want to try reinstalling the GRUB as well (which won’t be a bad choice).

1 Like

Thanks, I missed it, but unfortunately it gives me the same error I reported up here. :frowning:
I’ll try the manual steps.

I think I managed to chroot and I garuda-updated the system twice, with an error at the end but mostly succesfully updating packages it seemed.

:: Starting full system upgrade...
there is nothing to doRunning in chroot, ignoring command 'start'
Running in chroot, ignoring command 'start'
Failed to connect to bus: No such file or directory
Failed to start transient service unit: Transport endpoint is not connected

I deleted a strange directory on sda2 but outside /root that was taking up space (I think it was an old backup, since it had a name that I assigned to specific backup/snapshot in the past).

But… when I rebooted the result was the same as before.
I’ll do it again maybe trying to reinstall GRUB, even if I’m afraid to mess up my Windows 10 installation as well. :cold_sweat:

EDIT: W10 partition seems to stil boot up regularly, so I’m really way to mess it up. I feel stuck.

Nope: I tried garuda-updating from chroot several times, always with the same results.
I still cannot boot nor restore any snapshot, even if I now have 7.8Gb of free space on the root partition. :cold_sweat:

If you don’t have room, none do.

I boot 1 OS per 1 TB drive, no separate “home” to ever wonder if I’m gonna run out of room.

2 Likes

So the root partition cannot be resized?
However if I’m not able to reboot the topic is sadly postponed. :disappointed_relieved:

I don’t have the time for a detailed answer at the moment.

From what I can see, you ran out of free space during the update.

too late…

ok…you can do it that way too…still too late.

Please don’t explain to us that with the constellation:

Drives:
Local Storage: total: 2.85 TiB used: 216.55 GiB (7.4%)
SMART Message: Unable to run smartctl. Root privileges required.
ID-1: /dev/sda maj-min: 8:0 vendor: Samsung model: SSD 860 EVO 1TB
size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
tech: SSD serial: <filter> fw-rev: 3B6Q scheme: MBR
ID-2: /dev/sdb maj-min: 8:16 vendor: Seagate model: ST2000DX001-1NS164
size: 1.82 TiB block-size: physical: 4096 B logical: 512 B speed: 6.0 Gb/s
tech: HDD rpm: 7200 serial: <filter> fw-rev: CC41 scheme: MBR
ID-3: /dev/sdc maj-min: 8:32 vendor: Crucial model: M4-CT128M4SSD2
size: 119.24 GiB block-size: physical: 512 B logical: 512 B speed: 6.0 Gb/s
tech: SSD serial: <filter> fw-rev: 070H scheme: MBR
Partition:
ID-1: / raw-size: 50 GiB size: 46.57 GiB (93.13%) used: 42.7 GiB (91.7%)
fs: btrfs dev: /dev/sda2 maj-min: 8:2
ID-2: /home raw-size: 414.91 GiB size: 414.91 GiB (100.00%)
used: 173.85 GiB (41.9%) fs: btrfs dev: /dev/sda3 maj-min: 8:3
ID-3: /var/log raw-size: 50 GiB size: 46.57 GiB (93.13%)
used: 42.7 GiB (91.7%) fs: btrfs dev: /dev/sda2 maj-min: 8:2
ID-4: /var/tmp raw-size: 50 GiB size: 46.57 GiB (93.13%)
used: 42.7 GiB (91.7%) fs: btrfs dev: /dev/sda2 maj-min: 8:2

windows is installed on sda1…a few users here from the forum including me would otherwise have a heart attack :slight_smile:

Or even shorter: enlarge the root partition, or you’ll run into the same problem again at some point.

EDIT: @bro already mentioned it. If he has the time and the desire, he might explain it to you. I know that he knows about Windows and Linux.

3 Likes

No, Windows is on sdc, so you can spare the heart attack - this time. :wink:
Still, if I don’t find a way to reboot, nuking everything will be the only “solution”, since unfortunately updating from chroot seems to be ineffective.
I’m really way about reinstalling GRUB, not to invalidate the opportunity to boot Windows via sdc…

You saved a lot of lives with this :rofl:

Whether you need a separate /home depends on your usercase. Since you obviously don’t mount /home into any other Linux installations, this would be unnecessary.

My suggestion would be the same as mentioned above.
Back up all personal data and reinstall garuda. Either with more space for root or you use the entire SSD for the installation (like @bro already said). Check beforehand whether there is a bootloader or other important stuff on the SSD.

Whether you try to fix your garuda or whether you want to reinstall it, that is your decision :slight_smile:

4 Likes

If you re-install Garuda, please note that Windows is installed in MBR, not GPT, so you’ll want Garuda to be the same. Which is kinda a shame, but…

4 Likes

@Ezahn, switching distros is not the solution. It really doesn’t matter which distro you are using if you don’t make backups. Any and all OS’s will likely fail or require reinstallation at some point, (if you use them long enough). A static distro may be less likely to fail during updating, but it usually requires reinstallion at some point.

If you want to avoid reinstalling regularly your best route is to use a rolling distro and keep full backups of the system and your user data.

Without backups you will eventually suffer a breakage at some point, regardless of the OS in use. BTRFS snapshots are a great recovery method, but they are not foolproof, (as you have now have discovered).

A 120GB SSD can be purchased for ±$20 nowadays. Keep each OS on its own individual drive and you can hopefully avoid some pitfalls. Keep regular backups and you’re golden.

If you don’t keep backups, it really doesn’t matter which OS you use, you will end up having to reinstall at some point. It’s just a matter of how painful recovering your system will be. Backups generally make digital life painless. Unfortunately most people always have to learn that fact the hard way.

4 Likes

Thanks so much @nepti, @Bro and @tbg for taking the time to help and counsel.
I’ll try some more times to repair the installation, if only to learn something, and if everything fails I’ll reinstall, maybe with a single partition as suggested.

Regarding backups: wise words and I totally agree. I have a very recent backup of most of the home partition (all the documents), but I confess that I’m struggling to find a suitable backup strategy and software for both system and documents.
At the moment I just manually copied documents, pictures etc. to an external hdd.
On Windows I also use iDrive as cloud backup but I couldn’t make it work on Linux (even if I installed the relevant package)
If you have a strategy and software / cloud to suggest I’m all ears! :slight_smile:

Thanks again, all the best. :pray:

1 Like