I'm a Camera OP / Video editor, but I manage our post infrastructure for our team of 2 people at small production company.
We're having an issue with slow transfer speeds over the network from one of our cameras. This camera records to an SD card (Kingston Canvas React 256 GB) a DNG sequence. So after a shoot, on the SD card there are folders for every clip containing tens of thousands of 1-2 MB .dng files.
We're ingesting footage on a Windows 10 PC (Alienware Aurora R10, R9-5900x 32GB RAM RTX 3080) with DaVinci Resolve and it's Clone tool, which verifies the files after copying with checksumming. If we're using the explorer to copy the files to the network share, it's the same speed, so i don't think the problem is with DaVinci resolve. (There are software with expensive licenses (for example YoYotta) which claim to copy faster, but I haven't tried those). It's the same by the way on my 2015 Macbook Air with a Thunderbolt 2 to Ethernet adapter.
Our server is an old Dell T320 with 16 GB RAM and 5 x 3TB WD Red Disks (not the SMR ones) connected to an H200 Raid controller flashed to IT mode. It runs XigmaNas and the disks are in a raidz1 pool. This is then shared via SMB. (I know this server is very slow and low capacity for what we're doing, we've been planning on upgrading for a almost a year now to 12 disks and and an ssd for L2ARC and 10Gbps network, but we haven't been able to find a week free when there aren't any projects planned and I can do the upgrade.)
We have an embedded PC running PfSense for our router and firewall and we're using an unmanaged 8 port Zyxel switch to connect the computers. It can handle a full gigabit connection in every benchmark and test that I've tried.
So the problem is the following. When I start the transfer it starts off at aroung 70-80 Mb/s, it quickly drops to 10 MB/s in like half a minute, and the slowly to around 2-3 MB/s in a couple of hours. If it's done transferring one folder, the next one starts at full speed again. So if the card is filled with a bunch of folders (so smaller clips) it's not that bad, the transfer is done by a couple of hours. But when we fill the card with 1-2 clips, so 1-2, 100-150 GB folers containing 1-2 MB files it takes literally more than a day.
When we're using our other cameras, which are recording in 4K-6K, so much bigger files (for longer clips it's 100-800 GB files) it transfers at full speed, so usually maxed at 112 MB/s for the full duration of the transfer.
The transfer of the small files is of course a bit slower with the pool above 80-90% capacity (these are the numbers that I wrote above), but when it's just under 50% the slowest it gets is 5 MB/s which isn't that far from 1-2 MB/s. So it's still slow and I don't think it's the pool being full, because the large files are still transferring at full speed. I don't remember what it was like when the pool was empty.
When we're working with the videos in DaVinci Resolve it's slow at indexing the whole folder that the files are in (5-10 minutes for 256 GB), but once it's done it reads the files at full gigabit speeds. So it can be done, that's why I'm guessing it's a network overhead problem.
So, what am I missing? Where seems to be the problem? I've been googling and reading forums for almost a year intermittently, but can't seem to find the exact issue anywhere and I can't really experiment on the server because it's being used every day.
If it's the architecture of the whole setup and We'd have to move to an SSD based server, I'd rather come up with another solution, because We don't really have the funds right now for such a large upgrade. I was thinking about writing a script which makes a zip or a tar of the folders, transfers them over the network and then uncompress them. But then i'd have to do the checksumming and so on, and DaVinci Resolve does it quite elegantly and you can schedule transfer and such. And it's a GUI. So i'd be a bandaid solution. But I'm looking for any and all solutions.
Thanks you guys in advance.
Bence Szentiványi
