After a scrub of my 4-disk RAID5 mdadm array I got these log entries:
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340608-204340616
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340616-204340624
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340624-204340632
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340632-204340640
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340640-204340648
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340648-204340656
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340656-204340664
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340664-204340672
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340672-204340680
Dec 03 07:20:53 srv10 kernel: md1: mismatch sector in range 204340680-204340688
How can I find the file(s) affected on the ext4 filesystem?
Tried to use the method described in this thread but didn't manage to "translate" it to my setup. Use 4K-sector disks and ext4.
Superblock on the disks (same on all 4):
mdadm -E /dev/sdh3
/dev/sdh3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 05346821:52973e66:6ebd7f2d:ef64dcca
Name : localhost:R5_01
Creation Time : Wed Dec 16 17:53:50 2020
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 1827202440 sectors (871.28 GiB 935.53 GB)
Array Size : 2740803648 KiB (2.55 TiB 2.81 TB)
Used Dev Size : 1827202432 sectors (871.28 GiB 935.53 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=8 sectors
State : clean
Device UUID : 0d31c56e:a07be732:8258d07a:b05813bf
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Dec 3 12:49:22 2022
Bad Block Log : 512 entries available at offset 16 sectors
Checksum : ac57370b - correct
Events : 4651
Layout : left-symmetric
Chunk Size : 64K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
Partitions of the 4 disks (RAID5 on part. 3)
Total size: 931.51GB
Logical- / physical sector: 4096 / 4096
Total sectors: 244190646
First usable sector: 6
Last usable sector: 244190640
Partitions on /dev/sdc
ID PartNo DevName Label Start-sector End-sector Sectors Size Type
-- ------ --------- ------ ------------ ---------- --------- --------- ----------
1 1 /dev/sdc1 P1-EFI 2048 28671 26624 104.0 MiB EFI System
2 2 /dev/sdc2 P5-R10 28672 15757311 15728640 60.0 GiB Linux RAID
3 3 /dev/sdc3 P9-R5 15757312 244190640 228433329 871.4 GiB Linux RAID
EXT4 properties:
#dumpe2fs -h /dev/md1
Filesystem volume name: R5_01
Last mounted on: /mnt/R5_DATA
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem state: clean
Inode count: 42825728
Block count: 685200912
Free blocks: 207789491
Free inodes: 42129576
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64
Reserved GDT blocks: 1024
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 2048
Inode blocks per group: 128
RAID stride: 16
RAID stripe width: 48
Flex block group size: 16
First inode: 11
Inode size: 256
Required extra isize: 32
Desired extra isize: 32
Journal inode: 8
I also got some EXT4 error that I fixed with e2fsck:
Dec 02 19:35:21 srv10 kernel: EXT4-fs (md1): error count since last fsck: 7004
Dec 02 19:35:21 srv10 kernel: EXT4-fs (md1): initial error at time 1663363366: ext4_xattr_block_get:534: inode 4789231
Dec 02 19:35:21 srv10 kernel: EXT4-fs (md1): last error at time 1670003362: ext4_xattr_block_list:707: inode 4789224
Dec 02 20:35:11 srv10 kernel: EXT4-fs error: 26 callbacks suppressed
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_get:534: inode #4789231: comm smbd: corrupted xattr block 76627790
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_list:707: inode #4789231: comm smbd: corrupted xattr block 76627790
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_get:534: inode #4789228: comm smbd: corrupted xattr block 76627787
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_list:707: inode #4789228: comm smbd: corrupted xattr block 76627787
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_get:534: inode #4789231: comm smbd: corrupted xattr block 76627790
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_list:707: inode #4789231: comm smbd: corrupted xattr block 76627790
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_get:534: inode #4789228: comm smbd: corrupted xattr block 76627787
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_list:707: inode #4789228: comm smbd: corrupted xattr block 76627787
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_get:534: inode #4789221: comm smbd: corrupted xattr block 76627780
Dec 02 20:35:11 srv10 kernel: EXT4-fs error (device md1): ext4_xattr_block_list:707: inode #4789221: comm smbd: corrupted xattr block 76627780
When/if I manage to pinpoint the file(s) I assume I should do a echo repair > /sys/block/md1/md/sync_action and then do a new ext4 check.
It's not very likely these errors is coursed by any bad-block.....that should md be able to fix automatically, right?
SMART looks good for all 4 drives.
And then I need to take a look at ZFS!
