I am new to tape backup systems and recently bought a used IBM TS3100 3573-L2U library with a ULT3580-TD4 LTO4 SAS drive as well as some fujifilm tapes. I have followed this guide/HOWTO and am using the lto4 file from here with updated manufacturer and model flags to match my hardware.
I am running into an issue where backups of vm disks fail and mpt3sas offlines the device. In my troubleshooting, I tried writing a 100GB file to the tape a couple times to simulate doing disk image backups. The writes seem to fail after ~250GB have been written to the tape with this error from dd and similar from tar:
dd: error writing '/dev/tape': Input/output error
dd: closing output file '/dev/tape': Input/output error
After which I get:
root# mt status
/dev/tape: No such device or address
root# mtx status
cannot open SCSI device '/dev/changer' - No such device or address
root# ls -lh /dev/changer /dev/tape
lrwxrwxrwx 1 root root 4 Jul 12 19:37 /dev/changer -> sg11
lrwxrwxrwx 1 root root 4 Jul 12 19:37 /dev/tape -> nst0
root# uname -a
Linux vmhost 5.8.0-45-generic #51~20.04.1-Ubuntu SMP Tue Feb 23 13:46:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
relevant dmesg output
I was able to fix this a couple days ago by rebooting both the library and the server. However, I really don't want to have to reboot the server or even run into this scenario in the first place.
Question 1: How can I fix this without rebooting? I tried reloading mpt3sas but modprobe -r fails because the it's in use. In the library's web interface, the drive is reporting ready (my kernel does not support force reload).
Question 2: How can I prevent this from happening in the future?
