WAN Balancing & Failover with same ISP, duplicate gateway issue

[Mike_ITDept]

Hi,

I suspect that I already know the answer to this question, however some clarification would be great!

I am running two Arista firewalls at two locations. They each have 2 internet connections attached to them, a primary line with gigabit bandwidth and a secondary line with 80mb bandwidth, provided from the same ISP. They are both setup using WAN Failover and WAN Balancing with the appropriate settings relative to the bandwidth available.

Here is one of the routing tables for the WAN Balancer:

= IPv4 Table balance =
default
nexthop via ***.**.255.80 dev ppp0 weight 92
nexthop via ***.**.255.85 dev ppp1 weight 8

The IP addresses listed are the ISP gateways. In this case they are different, however unlike the connections main IP addresses which are static, these gateways are dynamic, which can result in both nexthop gateways being identical!

When both gateways are identical, issues occur! For example:

• The firewall does not know which connection to send traffic to, it may ignore one connection completely, or send upstream via one and downstream via another, breaking things like VoIP.
• The firewall loses track of it's physical ports, when WAN2 is unplugged it thinks WAN1 is unplugged.
• The firewall is unable to accurately report the usage of each WAN, it may show that WAN1 has zero usage yet I can demonstrate that it is carrying traffic.

This doesn't happen every time, I can restart the firewall and it will behave normally even with the duplicate gateways and WAN balancing will work as expected with zero issue.

Is anyone else using multiple WANs that are from the same ISP and have clashing gateways? Is there a way to make this work, or do I need to get my other lines from a different ISP?

[jcoffin]

Yes, unfortunately the way routing is done to the ISP concentrator in the street, the WANs with the same ISP are in a local network so the WAN Failover / Balancer will not work correctly.

[oj88]

Not the answer you're looking for but just an aside.... It is best practice to use different ISPs for primary and backup internet connections. The way you are setup now, you're only getting physical redundancy on the last mile. But if that ISP somehow develops an internal network problem or if that one gateway goes down for whatever reason, neither your primary or backup links will work.

If you connect via different ISPs, you won't run into the issues you're having and what you'll have is a proper redundant connection.

Just my 2c.

[donhwyo]

Different isp would be better. If for what ever reason you cant. You could stick an openwrt on one modem. Then the gateway could be different to the untangle.

[Mike_ITDept]

Thank you all for the insightful information. I am now looking to move our backup lines to a different ISP as this certainly seems like the ideal way forward on all fronts.