Score:4

2 Different Public Address for High Availability

pf flag

this is my first time asking a question here, and I was wondering is it possible to have two different ISP public IPs connected to a single system for high availability?

for example, if our ISP 1 became offline the 2nd ISP will be available, just like google and youtube have different Public addresses, I am just at a loss where to config this.

we are currently using fortinet 300D.

Nikita Kipriyanov avatar
za flag
It's possbile, but you'll have to fiddle with DNS, updating dynamically. And DNS servers must be outside. Many DNS hosters provide API for that. Many caveats including: non-trivial NAT and routing setup (I don't know how to set up this with Fortinet, but it is tricky, for example, in Linux); issues with dead internet detection (it could happen the link is OK and operator gateway is available, but *some* part of internet through that operator is not, while that part could work through other operator), and so on. I'd consider this not a "reasonable for business" solution. For home, maybe...
Davidw avatar
in flag
This is probably best done using dynamic routing protocols rather than DNS.
pf flag
it's like I just can't find the right word to search. or do you have any idea how google and youtube have different public IP pools? that they don't have down time? but I'll check out your suggestions.
Nikita Kipriyanov avatar
za flag
BGP and PI addresses, that's how big enterprises do HA in the Internet.
in flag
^Nikita has the right approach here. In more layman terms: you have your own IP addresses that belongs to you, not your ISPs. You tell your ISP's that the IP addresses are reachable on your own network, and they'll route your traffic to you. This is the core part of the Internet; it will re-route traffic as needed. Google has multiple IP addresses so they can have multiple servers, but each address in turn can have multiple incoming routes. At Google's scale, the real question is not if something is broken, but how much.
Paolo avatar
ua flag
High availability for what service? Web site? Vpn endpoint? Mail host?
pf flag
for a webserver, the thing is our ISP has been intermittent from sometimes, and I like to use our secondary IP address diff ISP as redundancy.
Doug avatar
in flag
You can definitely do that, with the limitations from the top of my answer that clients will still get failures. It will just be easier/faster for you to work around the outage. You setup IP addresses (or more likely NAT rules) so that you have external addresses on both ISP networks assigned to your server. Publish one or both addresses in DNS with a short TTL. When there is a failure remove that IP address from the DNS records, leaving only the working address. Some but not all configurations will require two ports at your server (e.g. ISP1:80->:80 and ISP2:80->:8080).
Score:7
in flag

Yes, but how you implement this will affect the user experience when one of the systems fails.

At it's simplest you can enter 2 address A records in your external facing DNS and users will be sent to both addresses (known as round-robin DNS load balancing). This is not particularly good, as it means that when one of the addresses is unavailable approximately half of the user connections will fail. It also is inefficient as clients of one ISP my be sent through the other ISP, or through the ISP with the less desirable route. Client application and DNS caching can delay clients getting the address to the working system, so failing clients tend to fail for fairly long periods without intervention to restart applications and flush DNS caches. If you keep the DNS TTL short and don't mind brief outages you can manually disable an address when the service is not available at that address, however the user experience is still of a brief failure.

To smarten this up you need to have an external system check that your service is available, and automatically update the DNS records to point users to the working system(s). Further improvements have the DNS system directly connected to the back-end monitoring to direct users to the less loaded system. Although automated, there is still a user experience in which some users will still see a failure.

None of this is specific to your firewall, which simply presents two external interfaces to the two ISPs. Note it is not possible to route traffic for ISP1 through ISP2 or vice versa as internet routing will simply drop this traffic. You cannot "cross connect" two ISPs and expect anything to work.

Major enterprises will generally not depend on DNS round-robin alone. Instead they will move to their own network (or partner networks) and have the ISPs route to their network in a system known as peering. The corporate network may have many peers consisting of multiple ISPs distributed on a regional to global scale. By exchanging routing information clients are routed from their ISP through the ISP(s) that are currently working and onto the corporate network. This can still result in brief outages while networks are unreachable, however these systems provide excellent redundancy for the corporate network to be reachable even during link outages.

Other more complex sophisticated solutions are possible, but outside the scope of a StackExchange answer. As examples:

  • Place a load balancer on a highly reliable system (Azure, AWS, etc.) and have it forward traffic to the monitored address that is "up".
  • Use a VPN based peer (sometimes referred to as a tunnel broker) to get an external IP independant of your ISPs, and allow the VPN tunnel to come across both ISPs
  • move the entire system to a high availability location
Score:0
ua flag

Buy a virtual machine from a provider that suits your needs (cloudflare level maybe?), set a virtual firewall there and establish multiple vpn from the on-premise site/sites to be bonded and used for virtual routing (thinking of MPTCP or similar).

The internet connection of the on-premise system will be redundant, with as many link as you like (different providers, multiple technologies) and a vpn to the virtual firewall for each of these links.

You will publish the desired service from the on-premise system through the hosted virtual firewall.

Depending on availability requirements, you can add an on-premise wan link, bandwidth to the virtual firewall or choose a more reliable provider for the VM hosting.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.