VXLAN/EVPN Outline
EVPN/VXLAN Config Structure/ Skeleton
0
Enable EVPN with the following command
service routing protocols model multi-agent
1. Add the Addresses to the devices / configure ports.
The first step is to add the addresses to the appropriate locations. Each devices will need;
- Address on uplinks/downlinks
- Loopback0 address for EVPN overlay
MLAG Devices will need additional addresses;
- 1 x VLAN SVI (normally 4094) for MLAG Heartbeat
- 1 x VLAN SVI (use 4093) for iBGP peering over the MLAG peer-link
Additionally, you need to ensure the MTU is set to 9214 on ports, and they are set as ’no switchport’. Make the descriptor something that identifies where that link goes to.
E.g Leaf1
int e1
no switchport
des Spine1_e1
mtu 9214
ip add 10.10.1.1/30
More on Addressing:
2. Configure MLAG & iBGP Peerings
For me, this is the hardest bit to remember, so here’s a checklist to help.
MLAG peers need an SVI to establish an iBGP neighborship. This is so that traffic can continue to flow if a Leaf -> Spine Uplink is lost on a leaf. The vlan needs to be a part of the trunk group added to the peer-link
MLAG Peers share a loopback address. Each leaf of an MLAG pair gets a unique loopback, but also a shared loopback for the logical VTEP
Set a shared mac address using “ip virtual-router mac-address …”
ip virtual-router mac-address 00:1c:73:00:00:99In the BGP Underlay config, set “Next-hop self” for the mlag peer
Click below for config examples:
3. Create the BGP Neighborship (Underlay)
Now that the addresses have been added, we can build the BGP Neigborship between the leafs and the spines. Key things to remember in this step are:
- set the router-id. Generally this is the loopback0 address
- set the maximum-routes to something large (12000) for each neighbor on both leaf and spine
- Set the maximum paths to the number of spines you have in thetopology.
- set the bgp distance (20 200 200)
- bgp log-neighbor-changes
- advertise the loopback address, which is required for overlay
- disable ipv4 default unicast AND
- create an ipv4 address family, and activate the neighbors underneaththat
It is super-duper important that the Leaf switches can see the loopback of all the spine switches at the end of this step!
Click below for config examples:
4. Configure the EVPN Overlay
Once you’ve verified that full neighborship is formed between the spines and the leaf’s, you can configure the overlay. Steps include:
- Form BGP neigborship using the Loopback0, utilising Peer groups
- Specify update-sour lo0
- use ebgp-multihop to enable peering of the loopbacks
- Enable send-community so we can send RD’s & RT’s
- Set maximum routes to 0
- create the address-family evpn mp-bgp mode
- on Spines,set the next-hop to be unchanged for leaf neighbors so they can act as a route server
The good part about this bit is that you can copy & paste once you’ve got the peer-group config sorted for the Leaf switches
Click below for config examples:
5. Create the networks to advertise, and map them to VNI’s
Now that the underlay & overlay are built and verified, we now need to create something to advertise. This is the good bit about EVPN/VXLAN. In this step we;
- Config VXLAN Interface
- Bind to lo0 - This is the mosrt important part of this process, don’t forget it.
- Declare udp port 4789
- Config/create the VLANS, and use VARP to configure the Gateway addresses.
#int vlan 30
#ip address virtual 10.100.30.1/24
- Config/create the VRF’s
- enable routing for each vrf
- Map the VRF’s and vlans to a VNI under the VXLAN Interface
Click below for int vxlan config examples:
6. Create the MAC-VRF and enable routing for VRF’s under the BGP Config
This is the last step in the process. Here, we are creating the MAC-VRF to allow us to route mac addresses across the fabric. Additionally, we are advertising the VRF to allow multi-tenancy and segmentation if required. We create these under the BGP Confing. Important things to note:
- the Prompt will change to mac-vrf when configured under BGP. Make sure you see this before configuring
(config-router-bgp)#vlan 10
(config-macvrf-10)#
- Design a RD naming schema that works for you. Suggestions include ASN:VNI for L2, and LO0:VRFID for Layer 3
- Create and RT naming schema that works for you (as above). I like to use vni:vni for both Mac-VRF and l3 vrf
- IN an MLAG pair, the RD will be shared for each for a vlan. They will need to be unique for L3
- For ease of use, set the RT to ‘both’ for import/export. Of course, if you needed to create an extranet you can also do that. (the EOS version I used complained about using ‘both’ and made me specify import/export)
- You can configure a route-map for adress filtering here if required
- You can also use ‘RD Auto’ as well. This is another advantage of using eBGP in the underlay instaled of OSPF
- ensure to use the ‘redistribute learned’ and ‘redistribute connected’ so that the MAc adresses and Routes are learnt and distributed through the fabric
I ran into a ‘funniness’ here, where the EOS version I was using complained about the “route-target both” command deprectated under the VRF. It wanted me to use
route-target import evpn 20010:20010
route-target export evpn 20010:20010
Though it did still add that command to the configuration. It didn’t give me this warning for the MAC-VRF component.
This is easier to explain by looking at configuration examples below;