snap
stores all of its packages - including core dependencies such as bare
and core
as well as every software package as single SquashFS .snap
files and then mounts these files (usually in /snap/
directory) in order to access the contents. This is how it looks in action (output of mount | grep snap
):
/var/lib/snapd/snaps/snapd_17029.snap on /snap/snapd/17029 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/bare_5.snap on /snap/bare/5 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/prusa-slicer_60.snap on /snap/prusa-slicer/60 type squashfs (ro,nodev,relatime,x-gdu.hide)
/var/lib/snapd/snaps/gtk-common-themes_1535.snap on /snap/gtk-common-themes/1535 type squashfs (ro,nodev,relatime,x-gdu.hide)
snap
also does not have a command for reinstalling the package like, for example, apt reinstall
. In order to do a reinstall one needs to remove the package and install it again.
Given these two properties, it is possible to break all of the software installed via snap
by simply corrupting a SquashFS file containing some of the core dependencies.
It's exactly what happened to me yesterday - I couldn't run any software that has been installed using snap
. Every attempt ended up in exactly the same stacktrace:
unexpected fault address 0x6208c0
fatal error: fault
[signal SIGBUS: bus error code=0x2 addr=0x6208c0 pc=0x6208c0]
goroutine 1 [running, locked to thread]:
runtime.throw(0x85ef48, 0x5)
/usr/lib/go-1.13/src/runtime/panic.go:774 +0x72 fp=0xc0000dde38 sp=0xc0000dde08 pc=0x42e582
runtime.sigpanic()
/usr/lib/go-1.13/src/runtime/signal_unix.go:391 +0x455 fp=0xc0000dde68 sp=0xc0000dde38 pc=0x4426c5
compress/bzip2.init.0()
/usr/lib/go-1.13/src/compress/bzip2/bzip2.go:479 fp=0xc0000dde70 sp=0xc0000dde68 pc=0x6208c0
runtime.doInit(0xd0c400)
/usr/lib/go-1.13/src/runtime/proc.go:5222 +0x8a fp=0xc0000ddea0 sp=0xc0000dde70 pc=0x43bf1a
runtime.doInit(0xd11da0)
/usr/lib/go-1.13/src/runtime/proc.go:5217 +0x57 fp=0xc0000dded0 sp=0xc0000ddea0 pc=0x43bee7
runtime.doInit(0xd12160)
/usr/lib/go-1.13/src/runtime/proc.go:5217 +0x57 fp=0xc0000ddf00 sp=0xc0000dded0 pc=0x43bee7
runtime.doInit(0xd117e0)
/usr/lib/go-1.13/src/runtime/proc.go:5217 +0x57 fp=0xc0000ddf30 sp=0xc0000ddf00 pc=0x43bee7
runtime.doInit(0xd0fdc0)
/usr/lib/go-1.13/src/runtime/proc.go:5217 +0x57 fp=0xc0000ddf60 sp=0xc0000ddf30 pc=0x43bee7
runtime.main()
/usr/lib/go-1.13/src/runtime/proc.go:190 +0x1da fp=0xc0000ddfe0 sp=0xc0000ddf60 pc=0x42feda
runtime.goexit()
/usr/lib/go-1.13/src/runtime/asm_amd64.s:1357 +0x1 fp=0xc0000ddfe8 sp=0xc0000ddfe0 pc=0x45a541
snap-update-ns failed with code 2
Kernel log contained some more hints:
[19136.436337] SQUASHFS error: xz decompression failed, data probably corrupt
[19136.436339] SQUASHFS error: squashfs_read_data failed to read block 0xfbf09
[19136.436361] SQUASHFS error: xz decompression failed, data probably corrupt
[19136.436362] SQUASHFS error: squashfs_read_data failed to read block 0xfbf09
It was more or less clear to me that one of the core snaps must have been somehow damaged.
The question is, how to fix such an issue in an elegant way? There's no easy way to find which file has been corrupted and even if I'd have found that - snap provides no option to reinstall a package other that removing and installing it again.
Because of that, the only solution I see is removing all packages and installing them again - and this is what I ended up doing. Isn't there any other, less invasive option?
In order to replicate this issue, one can simply put random data in one of the core snaps - making it unreadable (backup the file first):
head -c 50M </dev/urandom > /var/lib/snapd/snaps/core20_1623.snap