I will launch and administrate a Linux storage server: 4U, HBA LSI 9500-16i Tri-Mode, hybrid. It wiill possess 12 positions for the HDDs/SSDs in the back, 24 positions in front. There is an expander that allows to combine NVMe/SAS. My question is how to efficiently maintain the structure of the HDDs throughout the lifetime of the server.
At the beginning, less than half of the positions will be occupied and two RAID6 arrays will be operated. However, as the time will go, I am sure that some drives will fail and I will have to purchase new ones with the same capacity. In addition, I am sure that one more array will be started after some time and more drives will be installed. We are talking about all SAS server HDDs.
Is it ok to mix different models of HDDs in the same array (as long as they have the same capacity/block size/cache/rpm)? I assume that after two or three years, it will be difficult to get the same model of a HDD to replace some that will be failing.
On one hand, I always believed that some variety of the HDD models in the RAID is welcomed as it decreases the probability that two drives will fail simultaneously/during rebuild. At the same time, the development of the product lines usually makes it difficult to get the same drive after a few years. So this is what I was doing until now, combining different models that seemed to me compatible.
However, on the other hand, in other storage server I administrate (the same HBA and configuration as mentioned above), I recently encountered some unexplained S.M.A.R.T. UDMA_CRC errors on all drives simultaneously (10 drives affected within a few seconds from each other). There are SATA server HDDs. The technician of the HW supplier (supposed to have experience with these HBAs) concluded that the reason is too large variety of the attached HDDs, too big variety of the spin-up times, latency, etc. This happened after three years of a flawless operation spiced by an occasional exchange of some failing HDD once or twice a year.
So what is a decent strategy for such storage server? What is an acceptable variety of the HDDs? Or am I missing anything important here? Thank you for any comment.