Asked this in network-engineering in stack exchange and was redirected here.
I have a couple of servers with the below config
server1:
eno1: 127.15.0.1/16 scope global
eno2: 5.0.0.1/24
server2:
lo: 127.0.0.1/16 (it had /8. I changed the subnet mask using 'ip addr del 127.0.0.1/8 dev lo; ip addr add 127.0.0.1/16 dev lo')
eno2: 5.0.0.2/24
eno1 of server1 is connected to a completely different L2 network and is completely isolated.
eno2 interfaces of both servers are connected to the same L2 network. Now i have to access 127.15.0.1 from server2.
Server1 was deployed long back and i dont have permissions to change any sort of config. I dont know why some one used 127.x.x.x subnet with scope global. Not sure if it is a valid config but i have to live with it. I have complete control on the server2 and i can change anything.
Both servers are linux based.
connectivity between 5.0.0.1 <-> 5.0.0.2 is good.
My first try was to add a route in server2 as below
ip r add 127.15.0.1/32 via 5.0.0.1
pinged 127.15.0.1 from server2. I see the ping requests and replies in tcpdump on server2, but ping command is showing 100% loss.
I disabled rp_filters
sysctl.cnf:
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.lo.rp_filter=0
net.ipv4.conf.eno2.rp_filter=0
rebooted after updating sysctl.conf
And i flushed out the iptables. (iptables -F)
Same result. I thougt may be server2 doesnt like using 127.x.x.x series. So i added the below rule on server 2
iptables -t nat -A OUTPUT -d 5.0.0.1 -j DNAT --to-destination 127.15.0.1
this rule is supposed to replace destination ip to 127.15.0.1 if packet is destined to 5.0.0.1.
pinged 5.0.0.1 from server2. Iptables replaced the destination ip with 127.15.0.1(confirmed this on server1 tcpdump). Server1 replied but the replies are dropped again.
I ran out of ideas at this point. I took down server1 for maintenance and replaced 127.15.0.1/26 with 192.168.1.1/16. Connectivity worked fine in this case (with and without iptables). Now the question is, is the issue because of using 127.x.x.x? If yes, is there a way out of it?? If no, what else can I try?
Note: This config was working before. We recently lost server2 (which had old Linux) and I am building it from scratch. Also windows doesn't allow using 127.x.x.x to interfaces other than loopback. Not sure why Linux allows it on non lo interfaces. may be there is a rationale!
To summarize I have these questions:
- Windows outright rejects the config when we try to configure 127.x.x.x but Linux allows it and that too with global scope. Is there a use case for this?
- In this case server2 sends out requests destined to 127.x.x.x and server1 is actually sending replies back. If 127.x.x.x is only host-internal, why are they even sending packets on the link??
