Score:0

SFTP - files are only transferred in short bursts for one client - routing issue or SFTP/SSH configuration?

im flag

I have a server running Ubuntu 21.04.1 LTS with daily incoming file transfers via SFTP. For the most part, these work fine, but there is one particular client where the transfer speeds are unreasonably slow.

Using a python script to monitor the growth of the incoming files, I noticed that the transfers come in bursts, rather than continuously. For example:

2023-01-10 10:57:50 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:51 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:51 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:52 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:52 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:53 --> 900.94 MiB (0.00 B/s)
2023-01-10 10:57:53 --> 901.47 MiB (1.06 MiB/s)
2023-01-10 10:57:54 --> 902.11 MiB (1.28 MiB/s)
2023-01-10 10:57:54 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:55 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:55 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:56 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:56 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:57 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:57 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:58 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:58 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:59 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:57:59 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:58:00 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:58:00 --> 902.11 MiB (0.00 B/s)
2023-01-10 10:58:01 --> 903.24 MiB (2.26 MiB/s)
2023-01-10 10:58:01 --> 903.26 MiB (38.04 KiB/s)
2023-01-10 10:58:02 --> 903.26 MiB (0.00 B/s)

As you can see, there are almost 10 seconds of nothing between two bursts of data transfer. For other clients, the transfers are pretty much continuous:

2023-01-10 11:21:14 --> 30.00 MiB (703.31 KiB/s)
2023-01-10 11:21:14 --> 30.41 MiB (831.19 KiB/s)
2023-01-10 11:21:15 --> 30.81 MiB (831.19 KiB/s)
2023-01-10 11:21:15 --> 31.34 MiB (1.06 MiB/s)
2023-01-10 11:21:16 --> 31.75 MiB (831.19 KiB/s)
2023-01-10 11:21:16 --> 32.28 MiB (1.06 MiB/s)
2023-01-10 11:21:17 --> 32.72 MiB (895.12 KiB/s)
2023-01-10 11:21:17 --> 33.16 MiB (895.12 KiB/s)
2023-01-10 11:21:18 --> 33.72 MiB (1.12 MiB/s)
2023-01-10 11:21:18 --> 34.15 MiB (895.12 KiB/s)
2023-01-10 11:21:19 --> 34.72 MiB (1.12 MiB/s)

My questions:

  1. Is this a normal pattern?
  2. Could this pattern give us any indication as to where the bottleneck lies?
  3. Are there any settings that can be changed on our server to counter this bottleneck?
  4. Provided I had access to the client machine (which I don't have at the moment), how could I further troubleshoot the connection other than running traceroute?
Score:1
ru flag

No, that pattern isn't normal. (Although using a python script to monitor the growth of the incoming files is not a good gauge. You should monitor the network bandwidth towards your server, filtered by source address.)

We can only speculate but the most probable cause is some bandwidth policing at their firewall. A true bottleneck would more likely show itself by a low overall transmission rate, not bursts like that.

If you have access to the client machine you can do some download/upload testing, to see whether yours is the only site impacted.

DataWiz avatar
im flag
Thanks for your input - which tool would you use to monitor the traffic in the suggested manner?
Zac67 avatar
ru flag
Given the short periods of pause and burst, I'd say you'd need direct port monitoring/packet capturing on the server port or somewhere along the path firewall<->server. Anything like NetFlow or sFlow doesn't provide the required resolution.
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.