Dang! I did it AGAIN: updatin' broke my system

[XORxECor]

Someone please clarify for me: is this my own doing, once again, or is something TOTALLY AMISS WITH the update functionality in Garuda Linux? Last time this happened, it turned out to be my own damned fault, and I know, I know, I read plenty of times to not install stuff -- especially system-level stuff, from the AUR. Yet I thought it would be cool to add some kernel dkms modules from the AUR, which then BUSTED my dkms and royally left me with an unbootable system on reboot.

No surprises to discover a ROYAL F*CK-UP that would leave me without a bootable machine this time around, 'cause I have been here far too many times already than to have any confidence in the dependability of the distro. I now make sure to read the output of the update with a magnifying glass and a fine-toothed comb and always hold my breath when I reboot -- something I try to avoid as much as possible... Don't get me wrong: I love Garuda and I like a painful operating system experience just as much as the next nerd in so much as it provides to me the opportunity to delve deeper in my Linux know-how and kung-fu.

Yet I am also a musician, and use my machine for the production of music. At this point, I want to run back SCREAMING ALL THE WAY to the welcoming, faithful, loyal, dependable and loving arms of a BSD operating system so fscking hard that I can already hear the violins playing in my head... I ALWAYS DID LOVE YOU, BSD!!! ...yet if only you had more modern driver support for hardware, more software development on the platform (like music production software) we might NEVER HAVE EVER, EVER, EVER PARTED NOR EVER WOULD... Instead, I became a cheater and had to betray you, BSD, for music production's sake and just being able to run something not Microsoft on my 'puter with the Realtek 8188ce wireless network interface and AMD audio chipset still as yet unsupported in BSD territory... Martin Luther King had a world-impacting dream. I also dream, albeit not so majestically or grandiose: I dream a much smaller dream of running Bitwig Studio natively on a BSD and helping to give to the world the first-ever, bonafide, workable, easy, and very capable system of microtonality in modern music production. Maybe some day...

I digress. What broke in the update: kernel updated to 6.0 yet the mkinitcpio portion of pacman's duties failed horrifically with this error:

running '90-mkinitcpio-install.hook'...
call to execv failed (No such file or directory)

I then read the manpage on mkinitcpio and deduce that I need to call it thusly, manually:

mkinitcpio -v -k 6.0.2-zen1-1-zen -g /tmp/vmlinux-6.0.2-zen1-1-zen

I did my homework, found a forum post where '-P' is used. Yet this proves useless and makes the command fail due to no presets in /etc/mkinitcpio.d/. So I deduce there must have been a preset and wonder what in the update process might have erased it??? Sure enough, I untar a backup of /etc and find the linux-zen.preset.

But before I F*CK up the already broken system even more than garuda-update did, I opened the /etc/mkinitcpio.conf file, and discovered a garuda-update shortcoming: /etc/mkinitcpio.conf.pacnew. Worse, the .pacnew has no useable values for my system. So I manually fix the .conf file, adding the new directives from the .pacnew, and specify uncompressed and set COMPRESSION=cat as per the manpage. Mkinitcpio reports a successful generation. I would like some confirmation:

From reading the /boot/grub/grub.cfg I deduce that the output of mkinitcpio must be /boot/vmlinuz-linux-zen. Can I 'cp /tmp/vmlinux-6.0.2-zen1-1-zen /boot/vmlinuz-linux-zen' and then do a 'grub-install' and hope to be able to reboot without tears, or do I need to write another love letter to BSD? Anyone but Bob, feel free to answer correctly the question... I know that this might work for the 6.0 kernel.

I offer my help to give garuda-update a serious kick in the pants and update. Yet what might have possibly fscked up the mkinitcpio, and deleted the linux-zen.preset? The ONLY thing I can think of is my having added that cachyos repo per the instructions in this forum. If cachyos repo turns out to be the culprit, then PLEASE REMOVE THAT F**KING POST IN THE FORUMS AND ALERT OTHER GARUDERS TO THE PROBLEM AND OFFER KUNG-FU FIXES... I WILL HELP!

Help!

F.

[TNE]

Well none of this is really garuda-update's doing, all of this happened in pacman, even the pacnew file and everything, it could've been anything, very hard to debug without access to the system or any kind of logs.

Just roll back a snapshot and try again, this time save the logs somewhere, so you can post them if it happens again.

This is what snapshots are for.

[SGS]

Boot simply a snapshot.

Why, why, why you have to find out yourself.

You are your own system administrator and the only one who has problems with cachyOS?

If so, it is not the OS.

[BluishHumility]

@XORxECor

The ONLY thing I can think of is my having added that cachyos repo per the instructions in this forum.

If you read through the thread, a few other folks did run into some problems after setting up this repo. This person had trouble using some of their normal packages:

Performance optimized repositories - cachyos-aur

Got a first problem: WPS Office no longer launches. Also, in the package manager, programs appear twice, as chaotic-aur and as cachyos. Uninstalled the repos, and immediately WPS Office started working again.

This person ran into some problems getting the kernel installed:

Performance optimized repositories - cachyos-aur

I'm trying to install the kernel and I get execv: No such file or directory when running sudo pacman -S linux-cachyos linux-cachyos-headers. The initcpios are never created and thus the kernel cannot be loaded. Any ideas what might be going wrong?

This person had to hack around a little bit to get the kernel working:

Performance optimized repositories - cachyos-aur

well ... in my case: I had to copy the kernel into boot and manually run mkinitcpio the kernel is in /lib/modules/$VERSION/vmlinuz dunny why but didn't work like "other arch kernels".

This person, like yourself, wound up with an un-bootable system:

Performance optimized repositories - cachyos-aur

Hmm, after installing a cachyos kernel and upgrading with garuda-update, I have the next error :: Running post-transaction hooks... ( 1/19) Restoring Linux kernel modules... ++ uname -r + KVER=6.0.0-1-cachyos + test -e /usr/lib/modules/backup/6.0.0-1-cachyos + rsync -AHXal --ignore-existing /usr/lib/modules/backup/6.0.0-1-cachyos /usr/lib/modules/ + rm -rf /usr/lib/modules/backup ( 2/19) Creating system user accounts... ( 3/19) Reloading system manager configuration... ( 4/19) Creating temporar…

As for this comment:

@XORxECor

If cachyos repo turns out to be the culprit, then PLEASE REMOVE THAT ******* POST IN THE FORUMS AND ALERT OTHER GARUDERS TO THE PROBLEM

If you read through the posts in the thread, you’ll find quite a few community members were very happy with the results they got adding these repos and kernels to their system. Just because you used this resource and had bad results doesn’t automatically make the post problematic.

If every Arch package that gave someone problems was taken down, Arch would have nothing.

Best to leave it up to the user. Those who wish to hack around making tweaks and trying new or experimental repos and kernels have every right to do so. If you would rather not risk taking your system out of production, then your approach should be more prudent.

@XORxECor

before I **** up the already broken system even more than garuda-update did,

Not to be disagreeable, but I think it is unfair to throw garuda-update under the bus for your broken system. “A bad workman blames his tools,” as they say.

[XORxECor]

Thank you, my friend. I found the other posts I had not read informative. I want to be the guy who blames not, yet knows what broke and by what hands. What kind of workman is it that finds his tools broken, learns to fix 'em himself, and then offers to fix the offending bandit who broke 'em so that it does not happen again and so that other workmen and workwomen do not have to suffer the same stupidity of broken tools.

garuda-update relied upon pacman, the condemned bandit who broke my initcpio and ate my linux-zen.preset. I say we hollar at the Arch Ranch over yonder and let 'em know. Or hell, posse up and fix it ourselves... He may have been the coolest dude back in like 1983, but these days his life has degenerated into robbin' old ladies of their pellets, gettin' drunk with ghosts, and messin' with the initcpio.

And 'cause now it seems official that Cachyos done messes up your Garuda, will somebody please take down or stuff that post to the rim with CAVEATS to spare others the stupidity of kernel install attempts from their repos. More fun, less fixin', for all.

Broke tools, no music, no fun...
...Oh, if only, BSD.

F.

[Hanuman]

Still -- snapshot can save your day

It reverts everything EXCEPT the home folder

[alkapwn3d]

install regular archlinux to a USB and boot to it.
then do a fresh reinstall of the linux.
that usually fixes an unbootable system for me.

[leonavis]

Hehe. I also installed the CachyOS-repos and kernel. And to be honest: I’ve had never done that if I hadn’t had a snap ready to go if things went south.

Such meddling with the OS can always turn out badly for your config. To add foreign sources into Pacman can always make your system unusable. But:

@XORxECor

And 'cause now it seems official that Cachyos done messes up your Garuda, […]

It did not mess up anything for my system. Everything works just as it did beforehand. There might be improvements, I’m actually to lazy to check. If you decide to try third-party hacks or solutions there’s always a chance it’ll break your system some way or the other.

Part of the fun of Linux for me actually is fixing broken messes I made for myself - until the OS is broken enough to reinstall it. Backup-SSD be praised! ^^

[XORxECor]

Amigos!

First a shout out and thanks to Alexjp, whose kung-fu saved the day, and BluishHumility for having taken the time to learn me to the fix. https://forum.garudalinux.org/t/performance-optimized-repositories-cachyos-aur/23773

1. Commented-out the cachyos repos in /etc/pacman.conf
2. Did as root a 'pacman -Syyuu'.
3. Migrated previous configuration from /etc/mkinitcpio.conf into /etc/mkinitcpio.conf.pacnew and then 'mv /etc/mkinitcpio.conf.pacnew /etc/mkinitcpio.conf'
4. Found the "vmlinuz" linux kernel for the 6.0.2.linux-zen in /lib/modules/6.0.2.linux-zen/
5. copied to /boot the 'vmlinuz' to 'vmlinuz-linux-zen' as per the grub.cfg .
6. Restored the linux-zen.preset to /etc/mkinitcpio.d/
7. Built the missing cpio initramfs archives in /boot by doing a 'mkinitcpio -v -P'
8. Did a 'grub-install', double checked the /boot/grub/grub.cfg entry I intended to boot from, ensuring the files made by step #3 above were named exactly the same.
9. 'update-grub'

Then a 'sync ; shutdown -r now', and it was smooth sailing from there... Don't worry: I then did a garuda-update to re-Garudaize everything after reboot. I chose to do the 'pacman -Syyuu' instead of garuda-update because garuda-update would oddly fail on the mkinitcpio portion. In regressing to the ARCH version, my linux-zen.preset suddenly reappeared... Odd...

[XORxECor]

leonavis, my Friend:

"Part of the fun of Linux for me actually is fixing broken messes I made for myself - until the OS is broken enough to reinstall it. "

You and me both, Brother! That's why I've become a regular poster and reader in this forum... Heck, I confess that I sometimes enjoy getting reckless and installing kernel modules from repos of ill-repute and questionable character the likes of the AUR... Total CRAZY!

F.