Category: Networking

Home Lab, Networking, VCF

Configuring Centralized Connectivity Networking with BGP in VCF 9.0

sargonkhizeran

VCF 9.0 introduced a new way to setup and configuring networking. The two models that can be configured depending on your business’s needs. There is Centralized Connectivity & Distributed Connectivity. Not going into full detail with the difference of those two, please read about it here, Setting up Network Connectivity.

Some of the plumbing of the network, it’s all Ubiquiti Unifi system, ESX hosts have dual 10GB interfaces each. My choice of routing is BGP, read more about my experience here; BGP Peering NSX Tier-0 with Ubiquiti UDM Pro. This will be key as I want my virtual networking to traverse the physical (North/South)

My blog on BGP Peering is pending an update. I will work on an update, however should get the peering to work.

Deployment

From the vSphere client, highlight the vCenter instance, in the right pane navigate to Networks >> Network Connectivity.

You will be presented with a Gateway Type, ours will be Centralized Connectivity

Networking Prerequisites

This is where we configure our Edges, select our size and give a unique name for our Edge cluster object.

Clicking ‘Add’ will bring up configuration for the first Edge node. In this section it will be mainly management configuration for the Edge as well as Edge TEPs.

The bottom was cut off, here is the remainder

Once we save, we have an option of Adding or Cloning the configuration.

Fill out the unique settings for the clone

There is an option to have passwords auto generate or you can specify ‘admin’ & ‘root’ passwords.

Example of filling it out

The final step before Deploying will be the ‘Workload Domain Connectivity’, this will contain BGP related configuration and Edge gateway uplink configuration.

The next series of screenshots are configuring 2 Gateway uplinks for each edge appliance.

I have 2 physical networks created (vlan 3 & vlan4) as interfaces specifically for overlay and North-South communication.

The final step is Deploy. Take a look at some of the photos post-install.

The final step is to perform an Inventory Sync in VCF Operations. This let’s Operations manage monitoring and password Lifecycle.

Under Fleet Management >> Passwords, Edge appliances are not there.

From VCF Operations >> Inventory, select the Management Domain and perform a Sync Inventory

After some time, the additional accounts will appear under Fleet Management/Passwords

Cloud Management, Home Lab, Networking, NSX

BGP Peering NSX Tier-0 with Ubiquiti UDM Pro

sargonkhizeran

After several months, attempts, and VCF rebuilds to get BGP peering with VMware Cloud Foundation, NSX, and my Ubiquiti Unifi network, it finally happened. I wanted to share my experience. This required a lot of help from internal rock star TAMs in our organization, external blogs, and Ubiquiti support. By no means considered an expert in networking, especially when it comes to routing protocols.

Running UDM Pro, firmware 4.1.13, and Network app version 9.0.114. A recent release of UDM allowed the functionality of BGP (via the GUI). I found others in the community who got BGP working on earlier versions of UDM Pro. Check out Chris Dook’s blog.

Not to get too deep into the VMware stack, I deployed an NSX Edge Cluster consisting of 2 nodes out of SDDC Manager. When viewing the Networking Topology in NSX, the following IPs you can see are the Edge interfaces, which will need to be defined in your configuration file

My order of deployment was off a little as I was troubleshooting. I was in the middle of my Edge Cluster deployment, but if I could do it again, I would make sure my configuration on the Ubiquiti is done.

During the Edge Cluster deployment, you can go into NSX and do a few things, it might be stuck on a failed task if BGP is not working, luckily I was able to Restart Task and complete it. I must say the SDDC Manager appliance was resilient, considering I had to reboot my UDM Pro several times in a span of 2 days.

Configured 2 BGP Neighbors, one for each network containing their respective interface subnets, you can see I have one neighbor for the .3 and .4 subnets.

I saved the following in a Notepad file and gave it a name. It was <name>.conf, initially I was using the default frr.conf, but the UDM appliance already has that file, so it was suggested by support to not use the same name.

!
router bgp 65000
bgp router-id 192.168.100.254
!
neighbor 192.168.3.2 remote-as 65001
neighbor 192.168.3.3 remote-as 65001
neighbor 192.168.4.2 remote-as 65001
neighbor 192.168.4.3 remote-as 65001
!
address-family ipv4 unicast
  redistribute connected
  redistribute static
  redistribute kernel
!
  neighbor 192.168.3.2 activate
  neighbor 192.168.3.3 activate
  neighbor 192.168.4.2 activate
  neighbor 192.168.4.3 activate
!

  neighbor 192.168.3.2 soft-reconfiguration inbound
  neighbor 192.168.3.3 soft-reconfiguration inbound
  neighbor 192.168.4.2 soft-reconfiguration inbound
  neighbor 192.168.4.3 soft-reconfiguration inbound
!
exit-address-family
!
!

From the Unifi Network UI, go to Settings >> Routing >> BGP and you can create the entry in there, upload the configuration. Once that is completed, connect to the UDM CLI.

The following commands will need to be added and saved to the running config.

root@UDMPro:~# vtysh

Hello, this is FRRouting (version 8.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.

frr#configure terminal
frr(config)# ip prefix-list ALL-ROUTES seq 5 permit 0.0.0.0/0 le 32
frr(config)# route-map EXPORT-ALL permit 10
frr(config-route-map)# match ip address prefix-list ALL-ROUTES
frr(config-route-map)# exit
frr(config)# router bgp 65000
frr(config-router)# address-family ipv4 unicast
frr(config-router-af)# neighbor 192.168.3.2 route-map EXPORT-ALL out
frr(config-router-af)# neighbor 192.168.3.3 route-map EXPORT-ALL out
frr(config-router-af)# neighbor 192.168.4.3 route-map EXPORT-ALL out
frr(config-router-af)# neighbor 192.168.4.2 route-map EXPORT-ALL out
frr(config-router-af)# exit
frr# write memory

For the instructions above, please ensure to type exit until you’re at the frr# prompt and then ‘write memory’

So, what happens next? Let’s verify our routes are being advertised and learned from NSX. There are other ways to validate, however, this is what I was chasing down, and it ultimately resolved my Edge Cluster deployment.

From one of the NSX edges (in admin mode) you can access the Tier0 and run ‘get route bgp’ and we can see which paths are connected via bgp

This is all in my personal lab, please do not rely on these to deploy to a production environment, use caution, and consult with a partner or any professional services.