Score:0

AzCopy on Linux: copying between mounted folders fails

al flag

Unsure if this should be in Stack Overflow instead, but here goes:

I have an Azure File share mounted as /source and an SMB/CIFS share mounted as /dest on an Azure VM running Ubuntu 20.04. Between the Azure File Share and VM and the NAS is an S2S VPN tunnel. There are no Windows computers in the mix.

I have several thousand 120-130 MB raw binary files organized in folders and subfolders that I need to copy from the Az FS to the NAS, but the AzCopy job fails after a few thousand files with the puzzling error message:

> 2023/07/24 08:56:02 ERR: [P#7-T#7336] DOWNLOADFAILED:
> https://redacted.file.core.windows.net/folder/subfolder/subfolder/subfolder/subfolder/sourcefile?se=2023-09-01T14%3A21%3A12Z&sig=-REDACTED-&sp=rwdlacupiytfx&spr=https&srt=sco&ss=bfqt&st=2023-06-30T06%3A21%3A12Z&sv=2022-11-02
> : 000 : request size greater than pacer target. When Enqueuing chunk.
> X-Ms-Request-Id:

That's the entirety of the logged error, and it repeats for every single file that has status DOWNLOADFAILED. I have no idea what it means and I can't find any reference to this or a similar error message anywhere.

AzCopy is v10.19.0, and this is the command I run on the Ubuntu 20.04 Azure VM:

./azcopy cp "sourcefolder/?SAStoken" "destinationfolder/" --check-md5=NoCheck --recursive=true --check-length=true --overwrite=false --preserve-last-modified-time=true --cap-mbps=60

Does anyone have any idea what the error message is trying to tell me? There are lots of moving parts in the setup (VM, Az file share, S2S VPN, NAS), so I have a hard time figuring out even where to begin looking.

Please note that several thousand files were successfully copied before this error started showing up in the log, and if I rerun the job to copy skipped/failed files, all files generate this error message.

Thanks.

Score:0
in flag

This is a bug related to --cap-mbps. My workaround is to limit the entire network interface with wondershaper, which is not ideal.

I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.