zpool import Failure: can I recover data?

Skig

6/20/23, 7:43 PM

I have a personal zpool server that was going down, and I want to know if there is a way to recover what I have. Most of my critical data (photos, tax into, etc) got backuped to OneDrive, so I am mostly ok, but would like the rest of the data if possible.

I watched the LTT video on ZFS maintenance/data loss, saw I have never done a scrub either, so I ran it, hit a bunch of checksum errors, a couple read errors. So, I am planning to upgrade from a 3+1 2TB drives to 3+1 6TB drives and have the drives now. I took the server down to put in a 4TB drive to back everything up (only ~1.5TB ZFS full), and I think I started all my issues by doing that, as at least the zpool was up, now it won't even power up anymore...

When I run zpool import, I get this report:

   pool: raid
     id: 12795258809798297580
  state: FAULTED
 status: One or more devices contains corrupted data.
 action: The pool cannot be imported due to damaged devices or data.
        The pool may be active on another system, but can be imported using
        the '-f' flag.
   see: http://zfsonlinux.org/msg/ZFS-8000-5E
 config:

        raid        FAULTED  corrupted data
          raidz1-0  DEGRADED
            sdc     ONLINE
            sdc     FAULTED  corrupted data
            sde     ONLINE
            sdf     ONLINE

Now, this shows an issue, sdc is appearing twice, and sdd is missing. When I run lsblk, I see this:

...
sdb      8:16   0   3.7T  0 disk
├─sdb1   8:17   0    16M  0 part
└─sdb2   8:18   0   3.7T  0 part
sdc      8:32   0   1.8T  0 disk
├─sdc1   8:33   0   1.8T  0 part
└─sdc9   8:41   0     8M  0 part
sdd      8:48   0   1.8T  0 disk
sde      8:64   0   1.8T  0 disk
├─sde1   8:65   0   1.8T  0 part
└─sde9   8:73   0     8M  0 part
sdf      8:80   0   1.8T  0 disk
├─sdf1   8:81   0   1.8T  0 part
└─sdf9   8:89   0     8M  0 part

sdb would be drive I was going to back up to. sdc, sde, and sdf are part of the zfs recovery, but it looks like sdd is now gone/failed.

Is there a way that I can get the zpool back up and running to recover whatever data I can back? I thought the whole point of raidz1 was any 1 drive could fail, and it was still recoverable... zpool import -f raid fails with:

cannot import 'raid': I/O error
        Destroy and re-create the pool from
        a backup source.

PS: Notes for rebuild... I think I need to do a zpool export when I get the new ZFS array running. Also, I should refer to disks by ID instead of devid.

0 + 0

zfs

Score:0

Server

tquang

6/21/23, 9:51 AM

If you want to backup zpool, you can research more with key "zfs snapshot", now you can use 2 external tools (I'm using them for backup and restore)

https://www.zfsnap.org/

https://github.com/someone1/zfsbackup-go

Take snapshot zpool with name "music-pool" and rolling-out after 3 days

./zfsnap.sh snapshot -rv -a 3d music-pool

Full backup snapshot to external storage

#List
zfs list -t snapshot

#Do store backup to path /backup/
./zfsbackup-go send music-pool@FULL--3d file://backup/

Incremental backup snapshot to external storage

zfs list -t snapshot
./zfsbackup-go send -i music-pool@FULL--3d music-pool@FULL_inc001--3d file://backup/

Restore from full

./zfsbackup-go receive -F -d music-pool@FULL--3d file://backup/ music-pool

Restore if you had incremental (do the above command to restore full, then incremental)

./zfsbackup-go receive -d -F -i music-pool@FULL--3d music-pool@FULL_inc001--3d file://backup/ music-pool