Well hello, there party people!
I recently came across an issue where the customer had just configured alerting to look for Thin Provisioned VMs, and to my and their dismay it came back with over 300 VMs across 2 sites in staging and production that were like this.
Now they have Dell Compellent SANs and these Thin Provision at the storage layer too, so it was basically thin on thin!
Now the key points here were:
- Let’s figure out why this happened
- Let’s fix the issue for the currently deployed VMs and for future deployments
Now we did some digging and it appears the Devs and their automation scripting were the cause of the issue. In their scripts, they had it specifically deploying to Thick Lazy, but they had realised that the script would fail at this point. So they decided to deal with it later and comment it out!
So we dealt with that and educated them about this kind of thing. I also changed the templates to all be Thick Lazy, so if this kind of issue happens again, it will default to Thick Lazy instead of Thin Provisioned.
So with the assistance of the company Power Shell guru and the vExperts in the #powercli slack channel, we came up with a script that will move VMs that you list in a csv file, to a temp datastore and move them back to their original datastore while changing the disks to Thick Lazy.
#This ensures you can only connect to one vCenter Server at a time Set-PowerCLIConfiguration -DefaultVIServerMode single -Scope Session -confirm:$false #Connects to the vCenter Server Connect-VIServer <YOURVCENTERSERVER> #Specifies the source csv file with the list of VMs $csvfile = "c:\temp\YOURCSVFILE.csv" #Temp datstores to be used 1 per cluster, add more if needed $tempdatastores = @{ "Production Cluster 1" = Get-Datastore -Name "TEMPDS1"; "Production Cluster 2" = Get-Datastore -Name "TEMPDS2"; } #For every VM in the csv file it will check to see if the VM is in either of the clusters specified above #and if it is not it will error and output to screen, and then continue #It will do $MoveVM1 which moves the VM to the temp datastore #It will then do $MoveVM2 to move the VM back to it's original location while converting it to Thick Lazy #After completing the process for one VM it will move to the next #You will notice that there is a commented out Read-Host. #You can uncomment this if you want it to pause after the first move and you can control when it does the move back ForEach ($row in Import-CSV -path $csvfile) { $VM = Get-VM -Name $row.vm $currentcluster = $VM | Get-Cluster if ($tempdatastores.keys -notcontains $currentcluster.name){ Write-Host ($currentcluster.name + " Cluster name is not valid! ") -ForegroundColor DarkRed continue } $Thistempdatastore = $tempdatastores.($currentcluster.name) $currentdatastore = $VM | Get-Datastore $MoveVM1 = Move-VM $VM -Datastore $Thistempdatastore -RunAsync #-whatif while($MoveVM1.state -eq "Running"){ Start-Sleep -s 5 $MoveVM1 = Get-Task -ID $MoveVM1.id } #Read-Host "Press any key to Move $VM back" $MoveVM2 = Move-VM $VM -Datastore $currentdatastore -DiskStorageFormat Thick -RunAsync #-whatif while($MoveVM2.state -eq "Running"){ Start-Sleep -s 5 $MoveVM2 = Get-Task -ID $MoveVM2.id } } #This disconnects the connect to vCenter Disconnect-VIServer <YOURVCENTERSERVER> -confirm:$false
Now let me show you the layout of the csv file, as you can see below it is very basic. Just one column labelled vm and then a list of VMs below it.
Also, one thing to note, when moving around such a large number of VMs, the Dell Compellent (or any Thin Provisioning Array) will see this as all new writes. The issue here is that this will cause an increase in disk usage. Now if you have enough space to handle this, then you are good to do it all in one go and I already assume you are already manually doing SCSI UNMAPs in some way or are using VMFS6 with the correct settings.
If you want to play it safe I would do it in stages and then make sure you run the SCSI UNMAP commands on the temp datastores from one of the hosts on to ensure space is reclaimed correctly. There are plenty of articles and KBs on how to do that already.
There is no warranty given, so please know what you are doing before running this script!
Leave a Reply