I have several programs that interact with a rest api, each one with a different call rate. My goal is to run all of them on the same vps using supervisord. But if I do it without any control mechanism the remote api gives me lots of rate violation errors. I'm running an Amazon Linux 2 instance with two interfaces and two elastic IPs attached.
What I'm struggling to do is routing the calls from some programs throughout eth0 and some throughout eth1. The idea is making the api see two different IPs coming in so the rate limits don't trigger.
Before any attempt, here's the stock config:
iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ip route show
default via 172.31.32.1 dev eth0
default via 172.31.32.1 dev eth1 metric 10001
169.254.169.254 dev eth0
172.31.32.0/20 dev eth0 proto kernel scope link src 172.31.43.78
172.31.32.0/20 dev eth1 proto kernel scope link src 172.31.47.106
ip rule show
0: from all lookup local
32765: from 172.31.47.106 lookup 10001
32766: from all lookup main
32767: from all lookup default
ip route show table 10001
default via 172.31.32.1 dev eth1
172.31.32.0/20 dev eth1 proto kernel scope link src 172.31.47.106
ip route show table local
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
broadcast 172.31.32.0 dev eth0 proto kernel scope link src 172.31.43.78
broadcast 172.31.32.0 dev eth1 proto kernel scope link src 172.31.47.106
local 172.31.43.78 dev eth0 proto kernel scope host src 172.31.43.78
local 172.31.47.106 dev eth1 proto kernel scope host src 172.31.47.106
broadcast 172.31.47.255 dev eth0 proto kernel scope link src 172.31.43.78
broadcast 172.31.47.255 dev eth1 proto kernel scope link src 172.31.47.106
ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 0a:17:21:84:33:f8 brd ff:ff:ff:ff:ff:ff
inet 172.31.43.78/20 brd 172.31.47.255 scope global dynamic eth0
valid_lft 3423sec preferred_lft 3423sec
inet6 fe80::817:21ff:fe84:33f8/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc pfifo_fast state UP group default qlen 1000
link/ether 0a:08:10:90:fa:f8 brd ff:ff:ff:ff:ff:ff
inet 172.31.47.106/20 brd 172.31.47.255 scope global dynamic eth1
valid_lft 2474sec preferred_lft 2474sec
inet6 fe80::808:10ff:fe90:faf8/64 scope link
valid_lft forever preferred_lft forever
I have tried the following, but with no success. bmon shows no activity on eth1 when programs started by the user test do their calls. I've set them with chown test: ...
and chmod +s ...
, besides adding user=test
to the supervisor config files.
# adduser --system --no-create-home test
# iptables -A OUTPUT -m owner --uid-owner test -j MARK --set-mark 993
# ip rule add fwmark 993 table 993
# ip route add default via 172.31.32.1 dev eth1 table 993
# ip route add 172.31.32.0/20 dev eth1 proto kernel scope link src 172.31.47.106 table 993
# ip route flush cache
What am I missing?
Thanks in advance!