Garuda worked flawless for months now. After update I'm being dropped to emergency shell :-( How to fix?

ckid · 20 February 2022 20:09

Okay I stopped trying this idea as it is about the main boot partition.

But in my case EFI works as it asks for my LUKS password and correctly decrypts and then shows the grub menu.

But from then on everything I do drops me to emergency shell.

I will check again here tomorrow and hope for new ideas otherwise I have to reinstall garuda …which I really want to avoid…this would take me a day or two.

I understand this is a difficult problem to solve but this is happening linux distro after distro for years…

filo · 20 February 2022 20:32

Why don't you try at least to update-grub when you're in chroot?

ckid · 21 February 2022 06:51

because pacman -Syyu & pacman -Qqn | pacman -S - already do that no?

It also executed mkinitcpio and all that.

jonathon · 21 February 2022 12:12

If you didn’t see GRUB “update itself” (by which I assume you mean “generate the boot menu”) then no, GRUB won’t do that.

This implies it’s something you’re doing to your system rather than something being broken in every Linux distro?

ckid · 21 February 2022 18:01

@jonathon
yes ofc it will do and I clearly explained that in a post above that it did.....

ckid · 21 February 2022 18:03

I talk about 2 decades of computers and linux…

This sooner or later happened to me on any distro and you can google and you will see to how many people this happens regularly…

Again I only did a normal system update like every 2 days for months and suddenly the system is toast…

ckid · 21 February 2022 18:08

For completnes sake I ran update-grub as well and also attached the
garuda-inxi output as well.

sh-5.1# grub-update
sh: grub-update: command not found
sh-5.1# update-grub 
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-linux-zen
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux-zen.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-zen-fallback.img
Found linux image: /boot/vmlinuz-linux-lts
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux-lts.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-lts-fallback.img
Found linux image: /boot/vmlinuz-linux
Found initrd image: /boot/intel-ucode.img /boot/initramfs-linux.img
Found fallback initrd image(s) in /boot:  intel-ucode.img initramfs-linux-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
grub-probe: error: cannot find a GRUB drive for /dev/sdb1.  Check your device.map.
grub-probe: error: cannot find a GRUB drive for /dev/sdb1.  Check your device.map.
Found Ubuntu 21.10 on /dev/mapper/mydrive
Found Ubuntu 21.04 on /dev/mapper/mydrive
Found Ubuntu 18.04.4 LTS on /dev/mapper/mydrive
Adding boot menu entry for UEFI Firmware Settings ...
Detecting snapshots ...
Found snapshot: 2022-02-20 20:32:27 | @/.snapshots/390/snapshot | post | 1password a52dec aalib abiword abseil-cpp accountsservice acl adobe-sour |
Found snapshot: 2022-02-20 20:25:33 | @/.snapshots/389/snapshot | pre  | [rcu_gp]                                                                 |
Found snapshot: 2022-02-20 19:52:33 | @/.snapshots/388/snapshot | post | 1password accountsservice acl alsa-card-profiles aom attr audit bind bin |
Found snapshot: 2022-02-20 19:47:10 | @/.snapshots/387/snapshot | pre  | [kworker/3:0H-kblockd]                                                   |
Found snapshot: 2022-02-02 07:59:19 | @/.snapshots/386/snapshot | post | gendesk                                                                  |
Found snapshot: 2022-02-02 07:59:18 | @/.snapshots/385/snapshot | pre  | pacman -R -u --noconfirm --config /etc/pacman.conf -- gendesk            |
Found snapshot: 2022-02-02 07:59:17 | @/.snapshots/384/snapshot | post | cudatext-qt5-bin imdone-bin obsidian python-pulsectl syncthing-bin synct |
Found snapshot: 2022-02-02 07:59:15 | @/.snapshots/383/snapshot | pre  | pacman -U --noconfirm --config /etc/pacman.conf -- /home/cocoonkid/.cach |
Found snapshot: 2022-02-02 07:57:51 | @/.snapshots/382/snapshot | post | burpsuite-pro                                                            |
Found snapshot: 2022-02-02 07:57:50 | @/.snapshots/381/snapshot | pre  | pacman -U --noconfirm --config /etc/pacman.conf -- /home/cocoonkid/.cach |
Found snapshot: 2022-02-02 07:50:40 | @/.snapshots/380/snapshot | post | accountsservice alsa-card-profiles amdvlk at bat bolt broadcom-wl broot  |
Found snapshot: 2022-02-02 07:48:58 | @/.snapshots/379/snapshot | pre  | pacman -S -u --config /etc/pacman.conf -- core/zstd core/curl core/libel |
Found snapshot: 2022-02-02 07:41:50 | @/.snapshots/378/snapshot | post | garuda-hotfixes                                                          |
Found snapshot: 2022-02-02 07:41:49 | @/.snapshots/377/snapshot | pre  | pacman -Udd --noconfirm --needed archlinux-keyring-20220125-1-any.pkg.ta |
Found snapshot: 2022-01-31 06:29:58 | @/.snapshots/376/snapshot | post | chaotic-keyring                                                          |
Found snapshot: 2022-01-31 06:29:57 | @/.snapshots/375/snapshot | pre  | pacman -Udd --noconfirm --needed archlinux-keyring-20220125-1-any.pkg.ta |
Found snapshot: 2022-01-27 10:25:56 | @/.snapshots/374/snapshot | post | archlinux-keyring                                                        |
Found snapshot: 2022-01-27 10:25:53 | @/.snapshots/373/snapshot | pre  | pacman -Udd --noconfirm --needed archlinux-keyring-20220125-1-any.pkg.ta |
Found snapshot: 2022-01-22 10:38:57 | @/.snapshots/372/snapshot | post | gendesk                                                                  |
Found snapshot: 2022-01-22 10:38:56 | @/.snapshots/371/snapshot | pre  | pacman -R -u --noconfirm --config /etc/pacman.conf -- gendesk            |
Found snapshot: 2022-01-22 10:38:55 | @/.snapshots/370/snapshot | post | cudatext-qt5-bin lens-bin pgcli python-cli_helpers python-pulsectl synct |
Found snapshot: 2022-01-22 10:38:52 | @/.snapshots/369/snapshot | pre  | pacman -U --noconfirm --config /etc/pacman.conf -- /home/cocoonkid/.cach |
Found snapshot: 2022-01-22 10:38:19 | @/.snapshots/368/snapshot | post | anbox-modules-dkms buku                                                  |
Found snapshot: 2022-01-22 10:37:29 | @/.snapshots/367/snapshot | pre  | pacman -U --noconfirm --config /etc/pacman.conf -- /home/cocoonkid/.cach |
Found 24 snapshot(s)
Unmount /tmp/grub-btrfs.YycRojeO3a .. Success
done

12System:
  12Kernel 5.15.18-1-lts x86_64 12bits 64 12compiler gcc 12v 11.1.0
    12parameters BOOT_IMAGE=/boot/vmlinuz-x86_64 lang=en_US keytable=us tz=UTC misobasedir=garuda
    misolabel=GARUDA_XFCE_WHITETAILEDEAGLE quiet systemd.show_status=1
    systemd.unified_cgroup_hierarchy=1 driver=nonfree nouveau.modeset=0 i915.modeset=1
    radeon.modeset=1
  12Desktop Xfce 12info xfce4-panel 12wm xfwm 12dm N/A 12Distro Garuda Linux 12base Arch Linux
12Machine:
  12Type Desktop 12Mobo ASUSTeK 12model WS X299 SAGE/10G 12v Rev 1.xx 12serial <filter>
    12UEFI American Megatrends 12v 3302 12date 11/04/2020
12CPU:
  12Info 12model Intel Core i9-10920X 12socket LGA 2066 R4 12bits 64 12type MT MCP 12arch Cascade Lake
    12family 6 12model-id 0x55 (85) 12stepping 7 12microcode 0x5003102
  12Topology 12cpus 1x 12cores 12 12tpc 2 12threads 24 12smt enabled 12cache 12L1 768 KiB
    12desc d-12x32 KiB; i-12x32 KiB 12L2 12 MiB 12desc 12x1024 KiB 12L3 19.2 MiB 12desc 1x19.2 MiB
  12Speed (MHz) 12avg 1200 12high 1201 12min/max 1200/4800:4600:4700 12base/boost 3500/4000 12scaling
    12driver intel_pstate 12governor powersave 12volts 1.6 V 12ext-clock 100 MHz 12cores 121 1200 122 1200 123 1200
    124 1201 125 1201 126 1200 127 1200 128 1200 129 1200 1210 1200 1211 1200 1212 1200 1213 1200 1214 1200 1215 1201
    1216 1200 1217 1200 1218 1200 1219 1200 1220 1200 1221 1200 1222 1200 1223 1200 1224 1200 12bogomips 167995
  12Flags avx avx2 ht lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
  12Vulnerabilities
  12Type itlb_multihit 12status KVM: VMX disabled
  12Type l1tf 12status Not affected
  12Type mds 12status Not affected
  12Type meltdown 12status Not affected
  12Type spec_store_bypass 12mitigation Speculative Store Bypass disabled via prctl and seccomp
  12Type spectre_v1 12mitigation usercopy/swapgs barriers and __user pointer sanitization
  12Type spectre_v2 12mitigation Enhanced IBRS, IBPB: conditional, RSB filling
  12Type srbds 12status Not affected
  12Type tsx_async_abort 12mitigation TSX disabled
12Graphics:
  12Device-1 AMD Navi 14 [Radeon RX 5500/5500M / Pro 5500M] 12vendor Micro-Star MSI 12driver amdgpu
    12v kernel 12bus-ID 1c:00.0 12chip-ID 1002:7340 12class-ID 0300
  12Device-2 Logitech HD Pro Webcam C920 12type USB 12driver snd-usb-audio,uvcvideo 12bus-ID 1-5:4
    12chip-ID 046d:082d 12class-ID 0102 12serial <filter>
  12Display 12server X.org 1.21.1.3 12compositor xfwm4 12driver 12loaded amdgpu
    12note n/a (using device driver)
  12Message Unable to show advanced data. Required tool glxinfo missing.
12Audio:
  12Device-1 Intel 200 Series PCH HD Audio 12vendor ASUSTeK 12driver snd_hda_intel 12v kernel
    12bus-ID 00:1f.3 12chip-ID 8086:a2f0 12class-ID 0403
  12Device-2 AMD Navi 10 HDMI Audio 12vendor Micro-Star MSI 12driver snd_hda_intel 12v kernel
    12bus-ID 1c:00.1 12chip-ID 1002:ab38 12class-ID 0403
  12Device-3 Logitech HD Pro Webcam C920 12type USB 12driver snd-usb-audio,uvcvideo 12bus-ID 1-5:4
    12chip-ID 046d:082d 12class-ID 0102 12serial <filter>
  12Sound Server-1 ALSA 12v k5.15.18-1-lts 12running yes
  12Sound Server-2 PulseAudio 12v 15.0 12running no
  12Sound Server-3 PipeWire 12v 0.3.47 12running yes
12Network:
  12Device-1 Intel Ethernet 10G X550T 12vendor ASUSTeK 12driver ixgbe 12v kernel 12port N/A 12bus-ID b4:00.0
    12chip-ID 8086:1563 12class-ID 0200
  12IF enp180s0f0 12state up 12speed 1000 Mbps 12duplex full 12mac <filter>
  12Device-2 Intel Ethernet 10G X550T 12vendor ASUSTeK 12driver ixgbe 12v kernel 12port N/A 12bus-ID b4:00.1
    12chip-ID 8086:1563 12class-ID 0200
  12IF enp180s0f1 12state down 12mac <filter>
12Bluetooth:
  12Device-1 Cambridge Silicon Radio Bluetooth Dongle (HCI mode) 12type USB 12driver btusb 12v 0.8
    12bus-ID 1-1:2 12chip-ID 0a12:0001 12class-ID e001
  12Report bt-adapter 12ID hci0 12rfk-id 0 12state down 12bt-service N/A 12rfk-block 12hardware no
    12software yes 12address N/A
12Drives:
  12Local Storage 12total 2.79 TiB 12used 530.49 GiB (18.6%)
  12SMART Message Required tool smartctl not installed. Check --recommends
  12ID-1 /dev/nvme0n1 12maj-min 259:0 12vendor Western Digital 12model WDS500G3X0C-00SJG0
    12size 465.76 GiB 12block-size 12physical 512 B 12logical 512 B 12speed 31.6 Gb/s 12lanes 4 12type SSD
    12serial <filter> 12rev 111110WD 12temp 49.9 C
  12ID-2 /dev/nvme1n1 12maj-min 259:1 12vendor Western Digital 12model WDS200T1X0E-00AFY0 12size 1.82 TiB
    12block-size 12physical 512 B 12logical 512 B 12speed 63.2 Gb/s 12lanes 4 12type SSD 12serial <filter>
    12rev 611100WD 12temp 45.9 C
  12ID-3 /dev/sda 12maj-min 8:0 12type USB 12vendor SanDisk 12model SD9SN8W512G 12size 465.76 GiB 12block-size
    12physical 4096 B 12logical 512 B 12type N/A 12serial <filter> 12rev 1012
  12ID-4 /dev/sdb 12maj-min 8:16 12type USB 12vendor Kingston 12model DTR30G2 12size 57.69 GiB 12block-size
    12physical 512 B 12logical 512 B 12type N/A 12serial <filter> 12rev PMAP
12Partition:
  12ID-1 / 12raw-size 1.82 TiB 12size 1.82 TiB (100.00%) 12used 530.49 GiB (28.5%) 12fs btrfs
    12block-size 4096 B 12dev /dev/dm-0 12maj-min 254:0 12mapped mydrive
12Swap:
  12Kernel 12swappiness 133 (default 60) 12cache-pressure 100 (default)
  12ID-1 swap-1 12type zram 12size 62.49 GiB 12used 0 KiB (0.0%) 12priority 100 12dev /dev/zram0
12Sensors:
  12System Temperatures 12cpu 36.0 C 12mobo N/A 12gpu amdgpu 12temp 59.0 C 12mem 0.0 C
  12Fan Speeds (RPM) N/A 12gpu amdgpu 12fan 0
12Info:
  12Processes 425 12Uptime 13m 12wakeups 1 12Memory 62.49 GiB 12used 4.23 GiB (6.8%) 12Init systemd 12v 250
  12tool systemctl 12Compilers 12gcc 11.2.0 12clang 13.0.1 12Packages 12pacman 1882 12lib 349 12Client N/A
  12inxi 3.3.12
Garuda (2.5.5-1):
  System install date:     2022-02-20
  Last full system update: 2022-02-20
  Is partially upgraded:   No
  Relevant software:       NetworkManager connman

ckid · 21 February 2022 18:35

Okay what I am trying right now:

I added btrfs to HOOKS in /etc/mkinitcpio.conf as it was not there.

Also I checked /etc/default/grub and there "boot from cryptodisks" was commented out. I enabled it and ran again
mkinitcpio -P

Result: Now grub is scanning for BTRFS filesystems before crashing to emergency shell..

I smell the issue is hidden here somewhere..

Moved block before autodetect hooks but still no dice

ckid · 21 February 2022 19:17

So I have a suspicion.

set root='cryptouuid/4f00be9f88d047109d5464b39bffd653'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint='cryptouuid/4f00be9f88d047109d5464b39bffd653'  fe781c27-e3fe-49f6-896e-9611aa0f7838
	else

When I manually mount the partition from live system via sudo cryptsetup luksOpen /dev/nvme1n1p2 mydrive mydrive gets the UUID fe781c27-e3fe-49f6-896e-9611aa0f7838 which is precisely the UUID grub doesn`t find on boot.

But on boot grub asks for my password and it works so doesn't that mean it is mounted and available?

running blkid in the emergency shell does not list the luks-mapper ..

ckid · 21 February 2022 19:56

In emergency shell

Okay I was able to cryptsetup luksOpen the /nvme1n1p2 as new_rot and mount it to /new_root.

Then exit and I was kind of able to boot into the system.

There I ran a update-grub again and now the bootmenu is back from original grub to garuda linx and shows everything.

Still getting dropped to emergency shell.

Apparently grub does not mount the device correctly after having entered my LUKS password during normal boot.

Attached photos of what I did:

How and where can I see what is happening and why the BTRFS volume isn't properly mounted to /new_root ?

I sincerely hope somebody can help me here. I feel there is not much missing anymore to this riddle.

filo · 21 February 2022 20:22

This GRUB with LUKS isn’t really my thing but I thought to just leave you a hint found here:
https://wiki.archlinux.org/title/GRUB#Encrypted_/boot

If you experience issues getting the prompt for a password to display (errors regarding cryptouuid, cryptodisk, or “device not found”), try reinstalling GRUB and appending --modules="part_gpt part_msdos" to the end of your grub-install command.

Feel free to disregard if useless…

ckid · 22 February 2022 08:12

It accepts the password. thats not the issue. But it doesn't mount the BTRF volumes correctly to new_root.

There lies the problem. I will reinstall tomorrow and use etckeeper for keeping track of any changes in the future and get to the bottom of this issue.

We cannot continue telling anybody who's linux is crashed by simple normal updates that they have to reinstall...

mandog · 22 February 2022 09:59

We don't tell them to reinstall what we can't do is know what they have done themselves to cause the problems in the 1st place, we can't tell them to not enter some code they found on the internet that they c/p.
Why does this only happen to certain 00001% of users the rest are fine.
I've been using Garuda from before it was released nothing has happened that could not be fixed again Arch Linux for over 20yrs never had to reinstall

BluishHumility · 22 February 2022 11:53

Did you try reinstalling grub? I know it is taking your password, but this bit here seems to describe your situation accurately.

Worth a shot before you reinstall I think!

I have been following this thread with great interest, hoping to see what the solution is. Sorry I have not been helpful, I have practically no experience with encrypted drives. If you must reinstall I understand, but it seems like you are so close!

ckid · 5 March 2022 07:45

Will reply & update soon with my final trials. Thanks for all the comments.

ckid · 15 March 2022 14:27

Unfortunately none of the suggestions made it work.

So I reinstalled and moved my /home/ back and was up and running within a few hours.

Mostly some small fixes which took time but I am a happy camper again.

What I did now to at least make sure that I will be able to pinpoint what changed is:

Installed and setup etckeeper & etckeeper-packages.

Created backups of the relevant data.

/etc/fstab
/boot/EFI


sudo sgdisk --backup=nvme1n1p1.gpt.header.backup /dev/nvme1n1p1

sudo dd if=/dev/nvme1n1p1 of=mbr.bak bs=512 count=1

(lsblk -o +partuuid; efibootmgr -v) > boot-entry-repair-kit.txt

checked with strings to make sure that there is the correct data in there.

If it happens again I will report here

Thanks to everyone!

PS: Is there a notable difference between 512 and 446 or 448 ?

system · 29 March 2022 14:27

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.