Short answer: NAT Connection tracking
One thing to remember is that the vast majority of Routers on the IPv4 internet is NAT Routers.
Most NAT implementations does smart tracking, When you send UDP from a internal client to somewhere you will have a Destination Port and a Source Port.
If traffic comes in with the ports reversed, then that traffic will be routed back to your client, and allowed in most firewalls.
NAT/Firewalls with tracking detects these packets as related and forwards them back.
Example based on comments with server on port 5000 UDP
- Client sends a packet for
server:5000, source client:5001
- First NAT router will see a packet with source ip and port of
client:5001, and destination server:5000.
- Router sends this on it's way, and will have
NATip:NATport
- Server receives this and creates a response to
NATip:NATport that has the source of server:5000
- NAT receives this and has source
server:5000 destination NATip:NATport, which matches the packet that was sent out (but has source and destination reversed)
- NAT sends this to
client:5001 still with source server:5000
The source ip+port and destination ip+port creates a combination that can be tracked. (there is more details, but this is the basics)
Some more reading But I should dig up better documentation on this and not just refer to anecdotal evidence from what I have seen experienced.
