I have a PC with AMD Ryzen 9 3950X, ASUS ROG X570 and mixed 2x16GB DDR-3200 + 2x16GB DDR-3600 RAM.
Ubuntu 20.04 LTS
6 x SATA HDDs via MB
In the beginning I used the HBA only for 5 x SAS HDDs, and it all worked as expected with speeds 150-200MBs.
IBM M1015 with LSI SAS2008 flashed to IT mode (latest FW)
HP SAS2 Expander with 8 ports and latest FW (2 ports for HBA link) connected via one port to HBA (no second SFF-8087 cable)
5x SAS HDDs via Expander
4x SATA HDDs via Expander
4x SATA HDDs via 2nd HBA port
All SATA HDDs via HBA/Expander are in LVM JBOD like mode and act as one large disk array.
In the first test, I had all connected via Expander, but there were issues with negotiation speeds...one HDD was linked only with 1.5Gbps. So I moved 4 x SATA HDD to the second HBA port.
Now, all HDDs are linked at least with 3Gbps and there should be no issues with bandwidth.
Everything is default...I tried different IO scheduler like bfq but ended up back to mq-deadline because there was no difference.
All HDDs have write back cache on, so there should be no issues as well.
I observe HDDs with iostat for usage...there is none before copying any data.
I copy 100GB files
I do simple data transfer from internal SATA drives to JBOD array...there are times when I get stable 150-180MBs...first few seconds are over 200MBs due caching.
Then something happens and speeds drops to 15-50MBs without any reason nor errors in logs.
I have no idea how to debug it or where may be issue. I have seen some guys claiming SAS and SATA should be not mixed but it is related to RAID.
Either Expander is faulty or mixing SAS and SATA doesn't work.
Any suggestion, please?
