Join Log in
Score:3
garethhumphriesgkc avatar Server

Only one TCP socket (via nc) able send data to the same host/port at once

in flag
garethhumphriesgkc

Simple repro - in one window watch processes in top, in the other run: nc -lkp 10000 > /dev/null & ( head -50000000 /dev/urandom | nc -N 127.0.0.1 10000 ) & ( head -50000000 /dev/urandom | nc -N 127.0.0.1 10000 )

Observe that only one head and nc process are actively using CPU.

Attach strace to the head that isn't active - see it's stalled on a write, e.g.:

strace: Process 589084 attached
write(1, "\264\347\270\26\27\24'BRb^\353\302\36@\216\17V\210*n\252`\353\330\351\276\2\250\330\350\217"..., 4096^Cstrace: Process 589084 detached
 <detached ...>

Set up two listeners on different ports - e.g. 10000 and 10001, and both go at full speed.

This is a simple example, but I can reproduce it with other inputs and outputs - e.g. zcatting large files and sending them to production services over the network. It's not to do with the input, and it's not to do with the listening socket.

So - why can I can only have one tcp connection to any given host/port actively sending data?

There is an independent data source (feel free to experiment if you don't believe me), and an independent process opening its own tcp connection (netstat will show them both open) - the only thing in common is the destination (which doesn't have to be an nc listening on lo - happens to anything).

Given the destination can definitely have multiple incoming sockets receiving data at once, and the source can definitely send data down multiple network sockets at once, I'm struggling to figure out where the contention is coming from, causing only one pipe to be active at once.

488
0 + 0
Ron Maupin avatar
us flag
Ron Maupin
A port can only be claimed by a single process. You need to let the application use ephemeral ports for the source (port `0`), where the source ports are "randomly" chosen for each process. A TCP connection is identified by source and destination addresses and source and destination ports. If you use the same values for all four, then it is, by definition, the same connection, and trying to run a new process on the same connection will throw an error.
1
Reply
garethhumphriesgkc avatar
in flag
garethhumphriesgkc
There are different source ports being used by each connection. That's how nc operates by default, the second instance wouldn't open if the source port was already in use, and you can see in the netstat output that they're different. I assure you, they are two different TCP connections - I encourage you to try it yourself and see.
1
Reply
Score:8
A.B avatar Server
cl flag
A.B

Disclaimer: there are many nc variants. Assuming from -k that it's the OpenBSD variant. Each nc variant has its own perks.

nc is the wrong tool for this job: nc -lkp 10000 will manage one connection at the same time, because it doesn't fork, and despite using poll(2) never uses accept(2) on a 2nd incoming connection until the first's handling is finished: it's a guarantee that the 2nd connection will be left unhandled. Should there be a few more, they'll start staying in SYN-SENT state because nc -lkp 10000 uses 1 as listen(2) backlog: this is not a random choice. This can be checked by using strace on the running nc -lkp 10000 process.

The documentation about -k option tells the same:

-k

When a connection is completed, listen for another one. Requires -l.
[...]

the way it's written doesn't suggest that two connection will be accepted at the same time: only one behind the previous.

Replace the listening netcat with socat (-d -d for additional information):

socat -d -d tcp4-listen:10000,reuseaddr,fork /dev/null

The fork option ensures easy parallel processing: one process per connection.

0 + 0
garethhumphriesgkc avatar
in flag
garethhumphriesgkc
Thanks for the tip - socat behaves as expected. Now I've got to work why java doesn't - that sounds like a stackoverflow post though.
0
Reply
A.B avatar
cl flag
A.B
Try more. the first ones are "autoconnected" by the OS. Normally that's up to the backlog but there might be a few more. Here the 4th (ie the 3rd unprocessed) gets the first SYN-SENT (with `ss -tn dst == 127.0.0.1:10000`)
0
Reply
garethhumphriesgkc avatar
in flag
garethhumphriesgkc
Sorry A.B - I deleted my comment after I saw you'd amended you answer to address it, but before seeing your response. For those coming later, I just mentioned that subsequent connections were fully established, not in SYN_SENT.
0
Reply
A.B avatar
cl flag
A.B
I'm talking about pending connections: those nc waiting to be processed. Have at least 4 nc clients running at the same time (before the 1st of these 4 ends) to see this. Exact behavior might depend on kernel version and its settings, just try with more. Your example in OP only runs two clients so this behavior won't be seen.
0
Reply
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.