For ages I have been pondering over ‘something’ which seems very trivial and the fact I can’t get any solid info on it is driving me nuts: Why does every pNIC also have a virtual MAC address?
So here with have the VMkernel ports, a VSS and pNICS:
Just as you would expect, every physical NIC (pNIC) on a host has a MAC address. BUT when you type
‘esxcfg-info -n‘ in the shell, you will notice that every vmnic also has a Virtual MAC Address!
\==+Physical Nic :
|—-Name…………………………………………..vmnic3
|—-PCI Segment…………………………………….0
|—-PCI Bus………………………………………..2
|—-PCI Slot……………………………………….0
|—-PCI function……………………………………1
|—-MAC Address…………………………………….XX:XX:XX:32:06:1f
|—-Virtual MAC Address……………………………..00:50:56:52:06:1f
|—-FPT Shareable………………………………..true
Now as you can see the Virtual MAC Address starts off with a VMware MAC and then finishes at the end with the Physical MAC.
Now the question is why does each vmnic on a host need a Virtual MAC Address?! I have asked this question everywhere, and I really mean everywhere: from Reddit to Slack to Twitter and even Experts Exchange.
vExperts and VCDXs admitted to not knowing the exact answer and most people are even wondering why this trivial question keeps me awake at night. To be fair they do have a point but it comes down to this key fact:
Someone asked me why they had a Virtual MAC Address and the fact I don’t know myself and the fact that I can’t find a solid answer to his question…. well, just bugs me a lot!
I just couldn’t find any answer anywhere. I posed the question to my new friend Graham Barker (who has just signed up for twitter). He did some digging and found something very interesting:
When you run the esxcfg-info -n command you will be given info on the shadow vmnics too. Now for those who don’t know, shadow vmnics are used as part of the the VDS Health Check Feature. They check for VLAN/MTU mismatches across your network. It was introduced in the 5.1 version of the VDS.
As you can see I did a simple post on it last year, for anyone who wants a bit more info!
Now lets have a look:
|—-Client Name………………………….Shadow of vmnic4
|—-MAC Addr…………………………….00:50:56:52:5e:1e
\==+Physical Nic :
|—-Name…………………………………………..vmnic4
|—-PCI Segment…………………………………….0
|—-PCI Bus………………………………………..4
|—-PCI Slot……………………………………….0
|—-PCI function……………………………………0
|—-MAC Address…………………………………….XX:Xx:XX:XX:5e:1e
|—-Virtual MAC Address……………………………..00:50:56:52:5e:1e
As you can see the MAC Address of the shadow vmnic matches the Virtual MAC address of the pNIC!
So my first thought was well that’s it, the Virtual MAC Address is assigned so that it can be used for the VDS Health Check. So the health check doesn’t impact day to day operations in some way.
But I couldn’t resist the urge to dig just that little bit deeper! I still have a couple of ESXi 4.1 hosts that I wanted to cross check against!
For what follows keep this in mind: The Health check modules were introduced with vSphere 5.1+ and are installed as part of ESXi 5.1+ regardless of whether you have a VDS or not.
The ESXi 4.1 hosts all have Virtual MAC Addresses for their pNICS too! So the virtual MACS have been introduced before the VDS Health Check existed!
Now my team mate @ShadyMalatawey mentioned that there was a good chance that the Virtual MAC Address for the pNICS were introduced in 4.1 but never went live till 5.1, kind of like in the way NSX has features in versions that have not been enabled yet but are planned for in future releases. Now that makes sense.
I was chatting to the guy who runs http://www.govmlab.com/ who has a lot of old school knowledge and we discuss topics now and again. He mentioned the following:
“As per my knowledge, each pNIC has a Virtual MAC (last 3 bytes should be from the actual pNIC MAC) this is used for heartbeat protocols like beacon probing”
Now I thought that this was pretty interesting.
He also said there was no way from him to confirm this as he just knew it from a while ago and that there wasn’t anything written about or existing documentation that he knew of.
Our newest VCDX @Apollokre1d mentioned that the best course of action would be to raise a support request with VMware Support. To be fair, my experience with VMware Support has been a hit and miss story but since I really wanted to know i decided to ask them anyway and get an answer from the horses mouth!
Their first reply was just a general copy and past about how VMs get their Virtual MACs which is not what I was asking at all. So I asked again and this time they pulled some info from an internal article which made for an interesting read.
The info was:
There are 5 ‘types’ of MAC addresses that can potentially exist on an ESX host.
1) The first and simplest is the MAC assigned to a Virtual Machine. I’ll ignore these for this conversation.
2) The manufacturer assigned MAC address of a physical NIC.
*) On a Classic ESX system this address is used only for PXE booting the machine, afterwhich it is never used for traffic again.
*) On ESXi the MAC address of the PXE booted NIC, or the first physical NIC, by PCI slot, is stolen and given to the VMkernel TCP/IP interface vmk0.
3) A VMkernel TCP/IP interface MAC address is a generated MAC address based on a hash of the system UUID and the name of the interface (vmkX). These will always be in the form: 00:50:56:7X:XX:XX where XX are hash bits.
4) A Service Console interface MAC address is a generated MAC address based on a hash of the systemUUID and the name of the interface (vswifX). These will always be in the form: 00:50:56:4X:XX:XX where XX are hash bits.
5) A Generated MAC address is then assigned to each Physical NIC for use in beaconing and all traffic that comes from the virtual switch itself. These are generated in a special way. They will always be in the form: 00:50:56:5X:YY:ZZ where X is 0x50 | (physicalMac[3] & 0x0f), YY and ZZ are the last 2 bytes of the physical card’s MAC address: example: Physical NIC MAC : 00:15:17:3a:ca:05 Produces virtual MAC address: 00:50:56:5a:ca:05.
To be fair, while I knew the answers listed in point 1 to 4, number 5 struck gold and is also very similar to what Mr GoVMLab has said.
So long story short:
Each pNIC on an ESXi host gets a Virtual MAC Address assigned to it which you can look up using ‘esxcfg-info -n‘
This is then used by the virtual Switch itself for beaconing/heart-beating and is also used for the VDS Health Check feature.
So if anyone asks you why they have Virtual MAc Address….I have done all the digging for you!
Great post! This is why my network admin sends me notices of “network abuse” in which are reported unknown MAC discovered on the LAN.
But now come the next question: How can I disable the generation of a virtual MAC on a specific vmKernel NIC ?
The Network policy does not allow me to use MAC different from those provided with the Physical NICS.
Thank you