Score:0

AWS NIC configuration

us flag

I'm trying to setup a EC2 instance for a specific use case. In that i will need 2 ENI with publicly exposed IP address. (Both should be pingable)

I've done following steps so far:

  • Attached 2 Network Interface from a same subnet
  • Run instance
  • Created 2 Elastic IP address, attached with each ENI card of instance.

I can see following result on EC2 Instance Network interfaces section.

PublicIP

But I could not able to ping 3.104.193.180.

Also my ip address gives following result . . .

ubuntu@ip-172-31-37-139:~$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000
    link/ether 06:ce:dd:28:5b:b8 brd ff:ff:ff:ff:ff:ff
    inet 172.31.37.139/20 brd 172.31.47.255 scope global dynamic eth0
       valid_lft 3460sec preferred_lft 3460sec
    inet6 fe80::4ce:ddff:fe28:5bb8/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000
    link/ether 06:a6:9a:4a:98:c6 brd ff:ff:ff:ff:ff:ff
    inet 172.31.41.29/20 brd 172.31.47.255 scope global dynamic eth1
       valid_lft 3460sec preferred_lft 3460sec
    inet6 fe80::4a6:9aff:fe4a:98c6/64 scope link
       valid_lft forever preferred_lft forever

So my questions are . . .

  • What should i do to get another IP exposed/pingable ?
  • How should I know my both elastic IP from within the server?
Score:0
my flag

To obtain the public IP address associated with a given interface, first, get the MAC address from the interface, then retrieve the info you're looking for from the Instance Metadata Service (IMDS). So in your example above, eth1 has MAC address 06:a6:9a:4a:98:c6, so you can get the public IPv4 address with:

curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/02:1d:90:af:65:a3/public-ipv4s

There's more info in the AWS documentation

Regarding your connectivity issue, as a first step, double check the VPC routing, ACL, and security group configuration associated with each of your ENIs. These details are easy to misconfigure and can result in dropped traffic if you miss something. Go check those details first, and then come back and read the rest of this post. There's another more subtle issue that comes up specifically when you have multiple ENIs associated with an instance, and it could be your problem.

AWS VPC implements strict anti-spoofing protections. This is a good thing, as it protects AWS and its customers against participating in various forms of network attack in the event that an instance is compromised for some reason. However, it is something you need to take into account when attaching multiple ENIs to instances.

The issue is that basic routing behavior is driven only by the destination of an outgoing packet. This means that the response to an inbound packet may not be transmitted via the same interface on which the original packet was received. But to a VPC, this is considered a "spoofed" packet, in that the source address in the response packet does not match any of the private IP addresses associated with the ENI.

Consider the following interface configuration and routing table:

admin@ip-10-0-0-115:~$ ip -4 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    altname enp0s5
    inet 10.0.0.115/24 brd 10.0.0.255 scope global dynamic ens5
       valid_lft 2801sec preferred_lft 2801sec
3: ens6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    altname enp0s6
    inet 10.0.0.8/24 brd 10.0.0.255 scope global dynamic ens6
       valid_lft 2889sec preferred_lft 2889sec


admin@ip-10-0-0-115:~$ ip ro
default via 10.0.0.1 dev ens5
10.0.0.0/24 dev ens5 proto kernel scope link src 10.0.0.115
10.0.0.0/24 dev ens6 proto kernel scope link src 10.0.0.8

In this case, 10.0.0.8 is an address assigned to ens6. A inbound packet to this IP address will be received via ens6 and processed as expected. But an outbound response to that packet will be routed according to the above routing table and will egress via ens5 and be dropped by the VPC.

You can test this like this:

admin@ip-10-0-0-115:~$ ip ro get 8.8.8.8 from 10.0.0.8
8.8.8.8 from 10.0.0.8 via 10.0.0.1 dev ens5 uid 1000
    cache

Note that the device is ens5, even though 10.0.0.8 is assigned to ens6! The VPC would drop this traffic!

In order to ensure your packets are delivered by the VPC, you need to implement policy routing. Broadly speaking, policy routing refers to the situation where your system uses additional information beyond just the destination to make routing decisions. In this case, you need to also take the source IP address into account. What we need to do here is ensure that any outgoing packet with a source address of 10.0.0.8 leaves via ens6.

Policy routing in Linux is typically configured using the ip(8) command. To make the instance with the above routing table work, you'll want to create a secondary routing table specific to ens6. Manipulation of secondary tables works just like manipulating the 'main' table, except you specify a table ID. So, in this case we can add a route to the local network and a default route via the gateway to table 10000 like this:

admin@ip-10-0-0-115:~$ sudo ip ro add 10.0.0.0/24 dev ens6 table 10000
admin@ip-10-0-0-115:~$ sudo ip ro add default via 10.0.0.1 table 10000
admin@ip-10-0-0-115:~$ ip ro show table 10000
default via 10.0.0.1 dev ens6
10.0.0.0/24 dev ens6 scope link

And create a rule to route outgoing traffic from 10.0.0.8 according to this table:

admin@ip-10-0-0-115:~$ sudo ip rule add from 10.0.0.8/32 table 10000 pref 10000
admin@ip-10-0-0-115:~$ ip rule
0:      from all lookup local
10000:  from 10.0.0.8 lookup 10000
32766:  from all lookup main
32767:  from all lookup default

Notice the presence of rule 10000 showing that traffic from 10.0.0.8 will be routed via table 10000.

We can confirm this with ip ro get again:

admin@ip-10-0-0-115:~$ ip ro get 8.8.8.8 from 10.0.0.8
8.8.8.8 from 10.0.0.8 via 10.0.0.1 dev ens6 table 10000 uid 1000
    cache

Notice that it's being routed according to table 10000, but more importantly that it's going to be sent via device ens6!

Amazon Linux sets up policy routing rules like this automatically when you have multiple ENIs attached to your instance. I don't know if Ubuntu does, so you may need to do a bit of research on that front and possibly implement your own automation for your particular situation.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.