I’ve been using Linux for the better part of 4 years so I’m not new to it, but I’ve always learned stuff on an as-needed basis. Today I ran into an issue that I want to prevent in the future since I had a mini heart attack thinking about how my last backup on this system was… Never since I’m an idiot who forgot to set it up like I have on my laptop. Here are my steps:

  • Ran sudo pacman -Syu; sudo pacman -Syy like I do every few days
  • packages updated
  • restarted computer
  • can only boot into emergency mode

The journal was really long so I moved past it and went to the pacman logs, linux had updated from 6.4.3.1-1 to 6.4.3.1-2. Nothing else was important enough to cause the system to only boot into emergency (gcc, vbox, some libs) so I did a quick pacman -U to the cached 6.4.3.1-1 version for both Linux and Linux headers and rebooted - hurrah it was fixed! But I have no idea why it happened, or how to prevent it.

Has anyone else ran into this issue when updating? Any advice for preventing future crashes or issues like this so I don’t fear updating?

Edit: Thanks to everyone for your advice! I ended up following multiple bits of advice. I reinstalled arch to get btrfs as the filesystem (didn’t have anything important other than some docked-compose files and books yet) and grabbed the linux-lts kernal as a backup as well. I haven’t configured snapper yet, but it’s on my list of things to do.

  • Minty95@lemm.ee
    3·
    1 year ago

    Use timeshift, easy peasy to set up. It’s saved my bacon a couple of times now, only last week, when the kernel 6.4 came out, but my old nvidia driver wouldn’t work with it. You just jump into tty, run timeshift --restore and chose a previous backup, takes all of ten minutes. I have it doing a back up every day to a second HD, keeping the last five, doesn’t take up much space. I don’t think there’s an easier option than that.

  • auth@lemmy.ml
    1·
    1 year ago

    Arch breaks once in a while… Like anything else in my experience

      • auth@lemmy.ml
        1·
        1 year ago

        In about two years, the update process broke twice… I had to manually add or remove packages so that the update process would complete successfully

  • Fryboyter@discuss.tchncs.de
    1·
    1 year ago

    Ran sudo pacman -Syu; sudo pacman -Syy like I do every few days

    Syy forces the package database to be updated even if no updates are available.

    In my opinion, this makes no sense, especially after you have already run pacman -Syu before. Basically, you only generate additional, unnecessary traffic on the mirror you are using. Pacman -Syu is normally always sufficient.

    The journal was really long so I moved past it

    The display of the systemd journal can be easily filtered. For example, with journalctl -p err -b -1, all entries of the last boot process that are marked as error, critical, alarm or emergency are displayed.

    Has anyone else ran into this issue when updating?

    Not me. But other users do. Some of them also use a distribution other than Arch (or a distribution based on it). When I look at the problems, the current kernel is probably quite a minefield as far as problems are concerned.

    Any advice for preventing future crashes or issues like this so I don’t fear updating?

    As other users have already recommended, you could additionally install the LTS kernel. And if you use BTRFS as a file system, create snapshots before an update (https://wiki.archlinux.org/title/snapper#Wrapping_pacman_transactions_in_snapshots).

    And it should be obvious that important data should be backed up on a regular basis.

  • feral_hedgehog@pawb.social
    1·
    1 year ago

    Maybe rebuilding the ramdisk failed during the original upgrade?
    One of the post-install stages after upgrading the kernel is rebuilding the initramfs - a tiny environment for bootstrapping the main OS.
    If you trigger it manually with mkinitcpio --allpresets you’ll notice it has fancy colorful output, with clearly visible warnings/errors.
    However when invoked as part of an upgrade this coloring is removed, making errors difficult to spot.
    I had this stage randomly fail a few times, resulting in an unbootable system like you described - solution was to just trigger a manual rebuild or reinstall the kernel with pacman -U.
    It’s possible that this is what actually fixed things when you downgraded the kernel.

  • whoami@lemmygrad.ml
    1·
    1 year ago

    btrfs and snapshots is one solution

    Also, it looks like you were able to solve the problem, even if you aren’t 100% sure why. Maybe the difference in package versions lead to some problem with the bootloader, causing it to go to emergency mode, and updating it fixed the problem?

    Another is using a more stable distro…

  • happyhippo@feddit.it
    11·
    1 year ago

    Advice? Sure, setup timeshift backups.

    Or if possible, switch to btrfs and install snapper + the grub integration. Will make it possible to go back to a previous state even from grub.

    Not if you mess up grub tho, then you’re screwed.

  • Hamartiogonic@sopuli.xyz
    0·
    1 year ago

    Sadly, I can’t help you there, but I must applaud to your attitude of figuring out what happend and why.

    When reading forum posts about Windows, Android or iOS stuff, it’s infuriatingly common to find a list of potential fixes without any explanations. Many people don’t know what went wrong, or why, but they do have some ideas what might fix it. Unfortunately, they just can’t tell you why a particular action is supposed to fix anything, because they don’t understand the root causes.