EVPN/VXLAN Fabric Designer

The underlay is a plain routed network. Its only job is to get every VTEP loopback to every other one, with equal-cost paths through the spines. It knows nothing about tenants, VLANs or VNIs.

The overlay is a mesh of tunnels between those VTEPs, carrying tenant frames inside VXLAN. Every leaf can reach every other leaf directly as far as the overlay is concerned, regardless of how many spines the packet crossed underneath.

Most fabric faults are one of these being reasoned about as though it were the other.

What gets checked

• A leaf-to-leaf link. In a Clos fabric leaves reach each other through the spines only. A direct link gives some traffic a shorter path and breaks the equal-cost assumption the whole design rests on.
• A leaf missing a spine. It looks identical in a diagram, has less uplink capacity than its neighbours, and loses a larger share of it when a spine fails.
• Duplicate VTEP addresses, where traffic for one leaf arrives at another.
• One VLAN mapped to two VNIs. Hosts in the same VLAN on different leaves land in different segments and cannot reach each other, with nothing logged to say so. This is the quietest serious fault in a fabric.
• A VNI used as both L2 and L3. They share one 24-bit number space.
• An underlay MTU that cannot carry the tenant packet. VXLAN adds 50 bytes, so an underlay left at 1500 fails on full-sized tenant traffic — and it fails in a way that looks like an application problem.

Anycast gateways are the other half. The same address and the same MAC on every leaf means a host keeps its gateway wherever it moves. A mismatched MAC is worse than a mismatched address, because a moved host keeps the old one in its ARP cache and cannot reach its gateway until the entry ages out.