Is there a way to speed up the uploading/downloading of many small files using onedrive-abraunegg?

IamPedro-Dominguez

2/21/23, 6:30 PM

I recently switched to Linux from Windows and I've been having issues synchronizing my OneDrive. Although the onedrive-abraunegg program is really cool and I'm really happy using it, it has a small problem. It takes time to upload/download many small files.

I'm currently in college, and one class is making me use npm/angular. The issue is that angular needs a lot of dependencies(?) that creates tons of small files.

Right now I have a directory that contains 40,000 small files (300 MB) and though it's been almost 3 hours it still keeps synchronizing. Each file takes 00:00:01 to upload, but there are so many of them it takes a lot of time.

I want to clarify that I've got a good internet connection. I've uploaded and downloaded movies to my cloud using abraunegg and the synchronization worked fine and fast.

Is there a way for me to keep using abraunegg or do I have to look for alternatives?

0 + 0

sync

20.04

Nmath

2/21/23, 6:35 PM

Lots of small files take a lot longer to file transfer than the same capacity of a few large files. I doubt you can improve on this. Is there a way the files can be archived before transfer (like a zip file)? This would speed up the transfer.

IamPedro-Dominguez

2/21/23, 9:21 PM

That's a good aproach if it ever comes to it I'll have to zip those folders. But my issue is more about convenience rather than backing up my data. It's pleasant having all your info in the cloud and being able to access it in your phone, tablet, etc. by just pressing a button or typing a command.

Score:1

Ubuntu

abraunegg

2/21/23, 7:21 PM

Many thanks for using the client just refer to it as 'onedrive' or 'onedrive' by abraunegg.

I'm currently in college and one class is making me use npm/angular. The issue is that angular needs a lot of dependencies(?) that creates tons of small files.

Small files is the OneDrive API Achilles' heel - it does not handle these well due to how the API requires each file regardless of size to be sent in a new connection to OneDrive. The only way to speed this up is to introduce parallel uploads (which OneDrive limits to 4) which will be looked at as part of https://github.com/abraunegg/onedrive/issues/232

The best suggestion I have is that you exclude all these dependencies from syncing to OneDrive, and only sync your code | important stuff - the data you create - to OneDrive.

0 + 0

IamPedro-Dominguez

2/21/23, 9:22 PM

I didn't expect an answer from the owner of the client itself, that's really cool. I'll take your advice and exclude unnecessary files from uploading.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Is there a way to speed up the uploading/downloading of many small files using onedrive-abraunegg?

TH: มีวิธีเพิ่มความเร็วในการอัปโหลด/ดาวน์โหลดไฟล์ขนาดเล็กจำนวนมากโดยใช้ onedrive-abraunegg หรือไม่

RO: Există o modalitate de a accelera încărcarea/descărcarea multor fișiere mici folosind onedrive-abraunegg?

RU: Есть ли способ ускорить загрузку/выгрузку большого количества небольших файлов с помощью onedrive-abraunegg?

VI: Có cách nào để tăng tốc độ tải lên/tải xuống nhiều tệp nhỏ bằng onedrive-abraunegg không?

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.