Most of the time, a restart would get you going again. But today, a simple restart won’t cut it. I had to try a few different ways to finally get the login page again.

First of all, if SSH isn’t an option, you will need to get access to the console, which mostly you can through the ESXi server management portal.

Press Alt+F1 to get to the login screen and use root user to log in. I hope you still remember your root password. If not, check this one out. Once logged in, launch shell, by

shell.set --enabled true
shell

Then, run the following to check the services status.

service-control --status --all

If you see any stopped services, run the following to start all services.

service-control --start --all

Then wait for another 15 minutes before checking the web portal again.

While you are in the console, you should also check and make sure none of the logical drives should be full.

df -h

A few other resources to look at if the above doesn’t help.

• Reindex the database – https://www.running-system.com/vmware-vcsa-error-503-service-unavailable/
• VMware KB – https://kb.vmware.com/s/article/67818
• Due to the vSphere web client – https://chrisdsibug26.wordpress.com/503-service-unavailable-error-when-connecting-to-the-vcenter-server-using-the-vsphere-web-client/
• Due to the corrupted certificate store – https://kb.vmware.com/s/article/80020

/Updated on Oct. 23, 2022/

To fix the issue due to the corrupted certificate store,

Download the crl-fix.sh script attached to the VMware KB, and transfer the file to the vCenter Server via the tool WinSCP. You may need to follow this first.

Then run the script.

First, grant the script execution permission.

chmod +x crl-fix.sh

Then, run the script.

./crl-fix.sh

However, if you got the error message saying no such file or directory, run this first and try again.

sed -i -e 's/\r$//' crl-fix.sh

It could take a quite long time to see any results from the script. When it is finished, it should stop and restart the vmafdd service again.

Then, restart all the services of the VCSA.

First of all, enable SSH service so you can tune in through SSH to perform the commands.

To get the list of storage devices installed on the host, run

esxcli storage core device list

Find the one that has Is SSD: true and mark down the device ID for the next step.

esxcli storage core device smart get -d=deviceID

The output will look something like this. Mine says all N/A so there could be something still missing.

You can archive the same results for all disks in once by running a script called smartinfo.sh in the following folder:

/usr/lib/vmware/vm-support/bin

If you are satisfied with the information provided by this builtin command, great. Otherwise, you may want to try a modified copy of smartmontools for VMware, thanks to this vExpert.

Here is a step of how to get the tool installed on VMware host.

1. Download smartctl-6.6-4321.x86_64.vib.
2. Upload it to the /tmp folder of the ESXi host.
3. SSH to the host.
4. Set the ViB acceptance level to CommunitySupported.
5. Then install the package.

esxcli software acceptance set --level=CommunitySupported
esxcli software vib install -v /tmp/smartctl-6.6-4321.x86_64.vib

The tool is installed at /opt/smartmontools folder and works just like the Linux version.

/opt/smartmontools/smartctl -d sat --all /dev/disks/[DISK]

Windows Explorer

Supporting VMDK seems to be a feature built right into the Windows system. If you are using a system that runs Windows 7 or above, you may be able to map the image file right through Windows Explorer.

Navigate to the fold that hosts VMDK file in Windows Explorer, right-click the file and choose Map Virtual Disk.

Select the drive letter you want to map to, and click OK button on the Map Virtual Disk window.

To disconnect, right-click the mapped drive and choose Disconnect Virtual Disk.

It works beautifully when it works. When it’s not, it’s time to seek alternative options.

VMware Workstation

VMware Workstation is VMware’s desktop solution for virtualization. It has the “Map Virtual Disks” feature built in that opens VMDK in a wider format. If the method of using Windows Explorer doesn’t work out, VMware Workstation is next one in the line to try.

Go to File menu and select Map Virtual Disks… from the drop-down list to start the process.

vSphere Disk Development Kit

The  “VMware-mount” command line is the one you can use to mount a VMDK disk without a GUI interface. The tool is part of the vSphere Disk Development Kit, so you need to download and install the kit to use the tool.

To mount a VMDK file (saved at C:\temp) in read-only mode use the following command:

vmware-mount.exe X: “C:\Temp\TestVM.vmdk”

If the VMDK file contains more than one partition you can use the parameter /v:x to mount the other volumes:

vmware-mount.exe /v:2 X: “C:\Temp\TestVM.vmdk”

If you need a writeable access, just use the parameter “/m:w“. This is extremely helpful if you need to replace a broken system file.

vmware-mount.exe /m:w X: “C:\Temp\TestVM.vmdk”

If you want to list all the mounted virtual drives use /L:

vmware-mount.exe /L

The command line is easy to use and powerful, but I am finding it’s not so easy to get the Development Kit installed on my computer for some reason.

OSFMount

OSFMount is a 3rd party free image mounting tool that mounts not only ISO format images but also VMDK files in Windows with a drive letter. It works on pretty much all versions of Windows.

It’s fairly easy to use the tool. Click the Mount new… button at the bottom of the window and follow the instructions.

If you have other alternative options, feel free to share. It’s never enough to have a reliable tool that does the job well in hand.

/Update on July 8, 2020/

As some of the commenters pointed out, the open-source archiving tool 7-Zip does happen to be the easiest way to open the VMDK file for you. By all means, give 7-Zip a try first and if it failed, try something mentioned above.

Concerning about the performance impact that might be? Check out the VM Encryption Performance Whitepaper by VMware to find out.

In short, according to the Executive Summary in the whitepaper, We show that while VM encryption can lead to bottlenecks in I/O throughput and latency for ultra-high-performance devices (like a high-end NVMedrive) that can support hundreds of thousands of IOPS, for most regular types of storage, like enterprise class SSD or VMware vSAN, the impact on I/O performance is very minimal.

RVTools is a windows .NET 2.0 application which uses the VI SDK to display information about your virtual machines and ESX hosts. Interacting with VirtualCenter 2.5, ESX Server 3.5, ESX Server 3i, VirtualCenter 4.x, ESX Server 4.x, VirtualCenter 5.0, VirtualCenter Appliance, ESX Server 5.0, VirtualCenter 5.1, ESX Server 5.1,VirtualCenter 5.5, ESX Server 5.5, VirtualCenter 6.0 and ESX Server 6.0. RVTools is able to list information about VMs, CPU, Memory, Disks, Partitions, Network, Floppy drives, CD drives, Snapshots, VMware tools, Resource pools, Clusters, ESX hosts, HBAs, Nics, Switches, Ports, Distributed Switches, Distributed Ports, Service consoles, VM Kernels, Datastores, Multipath info and health checks. With RVTools you can disconnect the cd-rom or floppy drives from the virtual machines and RVTools is able to update the VMware Tools installed inside each virtual machine to the latest version.

This application supports ESX Server 3.5, VirtualCenter 2.5, ESX Server 3i, ESX Server 4i, VirtualCenter 4.x, ESX 4.x, VirtualCenter 5.0, VirtualCenter Appliance, ESX 5.0, VirtualCenter 5.1, ESX 5.1, VirtualCenter 5.5, ESX 5.5, VirtualCenter 6.0 and ESX 6.0.

All these VMs have network adapter set to use as Bridged mode, pretending connecting directly to the physical network. But when I went into Virtual Network Editor from Edit menu, I didn’t see a Bridged type of connection available in the list. And it kept telling me it’s unavailable when I tried to manually add it.

What I did that actually fixed this issue is to simply press that Restore Default button at the bottom left corner in Virtual Network Editor. It took about a minute to reset all network settings. And when it’s finished, the VMnet0 set as Bridged type miraculously returned.

First of all, I need to find an easy and reliable way to access VM’s BIOS to change the boot sequence.

I can press F2 during the normal VM boot process but I have to be quick and I have to be inside the VM window to be able to do that. And I found that adding the following line into the .VMX file works way better.

bios.bootDelay = "10000"

What it does is to delay the boot process for 10 seconds before the actual operating system kicks in, which gives me plenty of time to go into the VM and press F2 key to launch BIOS window.

Secondly, I need to find a way that allows the VM to boot off the bootable USB device, since neither VMware Player nor Workstation offers an option to boot from USB bootable device.

And that’s when I find this Plop’s BootManager awesome. Download the problem and unzip the content plpbt.iso to a place that’s convenient. Then, set this iso file as the content of CD/DVD drive, change the boot sequence to boot from CD-Rom. Then, fire up. Here is what you will see.

Plug in the bootable USB device, and select USB from the list, bingo.

From the times I’ve used this option, it’s been great.

Tools that require to finish the work:

• At least 1G USB flash drive
• Virtual CD mounting tool, i.e. Daemont Tools
• WinImage to build the image to USB storage.

Steps to make the bootable ESX Server 3i USB flash drive:

1. Download ESX Server 3i Installable ISO
2. Extract INSTALL.TGZ from the root directory of the ISO image using the Virtual CD mounting tool.
3. Extract /usr/lib/vmware/installer/VMware-VMvisor-big-3.5.0-xxxxx.i386.dd.bz2 from INSTALL.TGZ
4. Extract VMware-VMvisor-big-3.5.0-xxxxx.i386.dd from VMware-VMvisor-big-3.5.0-xxxxx.i386.dd.bz2
5. Attach the USB flash drive and make sure you no longer need the data on it
6. Use WinImage to transfer VMware-VMvisor-big-3.5.0-xxxxx.i386.dd to the USB flash drive
   1. Disk->Restore Virtual Hard Disk image on physical drive…
   2. Select the USB flash drive (Warning: If you select the wrong disk you will lose data!)
   3. Select the image file VMware-VMvisor-big-3.5.0-xxxxx.i386.dd
   4. Confirm the warning message
   5. Wait for the transfer to complete
7. Unplug the USB flash drive (Warning: If you forget to unplug the flash drive from the PC you might lose the data on your hard drives the next time you boot!)
8. Attach the USB flash drive to the machine you want to boot (Warning: If ESX Server 3i recognizes local drives, you might lose the data on it, so make sure you don´t need it anymore or unplug all hard drives!)
9. Turn the machine on and make sure the USB flash drive is selected as boot device
10. Enjoy.

Notes:

I failed to so on my Vista machine but had no problem on XP.  And the machine you want to boot from must have the option to make the USB flash drive as the boot device.

You can also watch the video to see how to do it in action.