NSX

Upgrading VMware NSX-T to NSX 4.0.1

In preparation for vSphere 8 upgrades, I’m in the process of upgrading many of the solutions in the homelab before upgrading to the big 8.

I’m currently running NSX-T 3.2.1 with NSX Manager appliances. I have NSX deployed out to a cluster with a couple of Edge appliances in a cluster configuration.

For those that might’ve missed the word, it was announced early-2022 that NSX-T 3.x would be no longer and that the naming would be shifting to NSX with versions 4.x going forward. You can read more about this here.

The first step was to ensure I have a recent ‘Successful’ backup from within NSX-T manager itself.

When you go out to Customer Connect Downloads, you will want to download the NSX 4.0.1.1 *.mub Upgrade file.

Once the file is downloaded, I chose to upload it from my local system where I was using my browser to access the NSX interface.

Once the file uploads, the next step will be to click ‘Prepare for Upgrade’

The following process will take some time, you might even be prompted with a session timeout, In my instance I came up with the error “Repository synchronization operation was interrupted. Please click on resolve to retry. Repository synchronization failed. a ‘Retry’ ran and it completed the check successfully.

Once this process completed, it took me to step 2, the manager console will reload.

Click on the drop-down and select ‘All Pre-Checks’

After reviewing the results of the Upgrade, I reviewed the alarms and felt I wanted to move forward with the upgrade. The Edges are alarming due to Memory Consumption and the Manager alarms were relating to the NSX ‘audit’ account.

I selected to run the upgrade in ‘Serial’ and selected ‘After each group completes’ for Pause upgrade condition. click ‘Start’

The Edge upgrades completed successfully, I will click Next for the Hosts. There is also an option to ‘Run Post Checks’

The post checks ran fine and the next step is start the Hosts for upgrades.

The hosts upgrade completed successfully, I even ran the post upgrade check and it succeeded. The only gotcha for my cluster was I had to manually move some VMs onto other hosts and power some VMs down to conserve on resources, so once that was done, hosts entered maintenance mode.

The final step which was to upgrade the NSX Managers, click ‘Start’

So the upgrade failed immediately as the ‘audit’ account came back to bite me, there was a strange behaviour where when I update the password, the account was still showing a Password Expired status, I ‘Deactivated’ and ‘Activated’ the account and it showed Active. Once that was completed, the message also stated to take action on the ‘Alarm’ in NSX, so I went back and Acknowledged and Resolved alarms and did not leave any in Open status.

The upgrade will allow you to continue once you navigate back to System>>Upgrade. Go back to Step 1 and run the “Pre Check” for NSX Manager Only, before proceeding to the final upgrade step.

The upgrade completed successfully. You will notice the top left-corner banner will now only read ‘NSX’.

Home Lab

NSX-T 3.2.1 – Error: Failed to fetch System details.

Running VMware NSX-T 3.2.1 on a clean install, no previous upgrade. Home lab is not sitting around doing much other than whatever idle tasks might be going on with the appliance. I decided to log in and BAM! The system section was not loading

Error code: 513002

Under System>>Configuration the following error would appear

I was able to navigate to check the basic health of my managers, and checked ‘View Details’ for each one; everything was up, all green, and space utilization looked good.

From the VM console logged in as admin, I tried ‘restart services controller ‘and no success.

While troubleshooting and looking for KBs and forums on the web, when running ‘get managers’ from admin console, one of my managers was in Standby, but then there was some intermittent activity where all went into Standby for a brief moment.

Not just this happening but my Appliances GUI was also now switching between ‘Healthy’ & Degraded.

Next step was to reboot one appliance at a time, which did not seem to fix the issue.

After taking a break and happen to be doing something else in the home lab, I had to shut down and restart the vCenter, well I happen to check later on in the day and NSX was able to load with no problems.

I’m going to keep this up in the event the issue comes around, I plan on updating.