Workaround: “Unable to Change Virtual Machine Power State: Cannot Find a Valid Peer Process to Connect to”

My Problem

Attempting to start a virtual machine in VMware Workstation 15 Pro (15.0.3) on a RedHat based Linux workstation caused the following error: “Unable to Change Virtual Machine Power State: Cannot Find a Valid Peer Process to Connect to”

I was able to start other virtual machines in the VM library, however.

My Workaround

Note that this is simply a workaround. I don’t yet know the ultimate cause, but I’m documenting how I workaround it until I or someone else can figure out the ultimate cause of this problem.

First, check to see if the virtual machine is actually running, in spite of there being no visual indicators within VMware Workstation: vmrun list

You’ll probably see that the virtual machine is running. If you don’t, then this workaround isn’t likely to help you. Attempt to shut the running virtual machine down softly: vmrun stop /path/to/virtual_machine.vmx soft

After that, you should be able to start the machine again, until the next time it crashes for unknown reasons. More news as I discover it.

Dumping Grounds (Turn Back Now):

I’ll dump some of my notes here and they’ll be updated periodically as I find out more info about this issue. You’re completely safe to ignore everything past this point. Abandon all hope, ye who proceed.

I had recently upgraded from Fedora 29 to Fedora 30, and was experiencing some minor instability with my main workstation. I’m not sure if that was the ultimate cause of this issue, but I’m suspicious since I never had this issue until after the upgrade.

My first act was to go to the Help menu, select the “Support” menu and then “Collect Support Data‚Ķ” I chose to collect data for the specific VM that was having this issue. This took quite a while, by my standards. About 20 minutes. It basically creates a giant zipped dump of pertinent files across your physical machine that pertain to VMware and that specific virtual machine. It’s not super easy to parse and know what to look for.

I searched through /var/log/vmware/ for any clues in any of the log files found therein. Grepping for all files that had the pertinent virtual machine’s name, and looking for surrounding context didn’t turn anything up.

I attempted to start the vmware-workstation-server service but that failed. I don’t think that’s the issue since the virtual machine isn’t a shared VM.

I tried vmrun list and saw that the Windows VM was actually listed as running. I stopped it soft: vmrun stop /path/to/my/virtual_machine.vmx soft and was then able to start the virtual machine. I’m not sure what’s causing the crash, and what’s causing the crash of VMware Workstation Pro, and why when I start it back up it doesn’t appear to know that the VM it was previously working with is actually running.

Solved: VMware Workstation 15 Fails to Compile Kernel Modules with “Failed to build vmmon” and “Failed to build vmnet.”

My Problem:

After updating Fedora 29, VMware Workstation Pro 15 needed to have some kernel modules compiled. However, attempting to install them earned me warning signs on the “Virtual Machine Monitor” and “Virtual Network Device” compilation process, and of course starting the services failed. Logs stated “Failed to build vmmon” and “Failed to build vmnet.”

My Solution:

Digest this SuperUser thread: https://superuser.com/questions/1328401/vmware-workstation-vmmon-broken-on-ubuntu-18-04/1330890

Ultimately, you need to clone this github repo: https://github.com/mkubecek/vmware-host-modules.git. Checkout the proper branch that corresponds to your product and version (for example, I used the Workstation 15.0.3 branch). make then make install within that repo, and finally you’ll want to create a symlink for libz.so.1.

Using two separate answers in the above SuperUser thread, then modifying it for my own purposes, I came up with this:

#!/bin/bash
VMWARE_VERSION=workstation-15.0.3 #This needs to be the actual name of the appropriate branch in mkubecek's GitHub repo for your purposes
TMP_FOLDER=/tmp/patch-vmware
rm -fdr $TMP_FOLDER
mkdir -p $TMP_FOLDER
cd $TMP_FOLDER
git clone https://github.com/mkubecek/vmware-host-modules.git #Use `git branch -a` to find all available branches and find the one that's appropriate for you
cd $TMP_FOLDER/vmware-host-modules
git checkout $VMWARE_VERSION
git fetch
make
sudo make install
sudo rm /usr/lib/vmware/lib/libz.so.1/libz.so.1
sudo ln -s /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/vmware/lib/libz.so.1/libz.so.1
systemctl restart vmware && vmware &

The Long Story:

After a massive cascade of updates on my Fedora 29 workstation that I had been delaying, VMware Workstation Pro 15.0.3 (build-12422535, and Kernal 5.0.3-200.fc29.x86_64 for whatever it’s worth) was unable to launch, instead requiring some kmods to be compiled and loaded:

“Before you can run VMware, several modules must be compiled and loaded into the running kernel.”

Dutifully clicking install earned me these lovely caution signs:

“Virtual Machine Monitor and Virtual Network Device have probably not done what you wanted.”

The services were unable to start, so I checked out the logs:

“Unable to start services. See log file /tmp/vmare-root/vmware-17464.log for details.”

Checking out the logs, I see a build command that failed:

2019-03-28T13:51:03.779-07:00| host-17464| I125: Invoking modinfo on "vmnet".
2019-03-28T13:51:03.781-07:00| host-17464| I125: "/sbin/modinfo" exited with status 256.
2019-03-28T13:51:03.833-07:00| host-17464| I125: Setting destination path for vmmon to "/lib/modules/5.0.3-200.fc29.x86_64/misc/vmmon.ko".
2019-03-28T13:51:03.833-07:00| host-17464| I125: Extracting the vmmon source from "/usr/lib/vmware/modules/source/vmmon.tar".
2019-03-28T13:51:03.839-07:00| host-17464| I125: Successfully extracted the vmmon source.
2019-03-28T13:51:03.839-07:00| host-17464| I125: Building module with command "/usr/bin/make -j12 -C /tmp/modconfig-PG76zy/vmmon-only auto-build HEADER_DIR=/lib/modules/5.0.3-200.fc29.x86_64/build/include CC=/usr/bin/gcc IS_GCC_3=no"
2019-03-28T13:51:05.179-07:00| host-17464| W115: Failed to build vmmon.  Failed to execute the build command.
2019-03-28T13:51:05.181-07:00| host-17464| I125: Setting destination path for vmnet to "/lib/modules/5.0.3-200.fc29.x86_64/misc/vmnet.ko".
2019-03-28T13:51:05.181-07:00| host-17464| I125: Extracting the vmnet source from "/usr/lib/vmware/modules/source/vmnet.tar".
2019-03-28T13:51:05.185-07:00| host-17464| I125: Successfully extracted the vmnet source.
2019-03-28T13:51:05.185-07:00| host-17464| I125: Building module with command "/usr/bin/make -j12 -C /tmp/modconfig-PG76zy/vmnet-only auto-build HEADER_DIR=/lib/modules/5.0.3-200.fc29.x86_64/build/include CC=/usr/bin/gcc IS_GCC_3=no"
2019-03-28T13:51:06.597-07:00| host-17464| W115: Failed to build vmnet.  Failed to execute the build command.

I see two other log files:

total 44
-rw-------. 1 root root 16669 Mar 28 13:51 vmware-17464.log
-rw-------. 1 root root 16555 Mar 28 13:50 vmware-apploader-17464.log
-rw-r-----. 1 root root  2792 Mar 28 13:51 vmware-authdlauncher-20116.log

The apploader log file didn’t appear to have anything of note in it. The authdlauncher… I wasn’t so sure:

2019-03-28T13:51:10.246-07:00| authdlauncher| I125: SOCKET 1 (12) creating new listening socket on port 902
2019-03-28T13:51:10.246-07:00| authdlauncher| W115: SOCKET Could not bind socket, error 98: Address already in use
2019-03-28T13:51:10.246-07:00| authdlauncher| I125: SOCKET Could not create IPv6 listener socket, error 11: Socket bind address already in use
2019-03-28T13:51:10.246-07:00| authdlauncher| I125: SOCKET 2 (12) creating new listening socket on port 902
2019-03-28T13:51:10.246-07:00| authdlauncher| W115: SOCKET Could not bind socket, error 98: Address already in use
2019-03-28T13:51:10.246-07:00| authdlauncher| I125: SOCKET Could not create IPv4 listener socket, error 11: Socket bind address already in use
2019-03-28T13:51:10.246-07:00| authdlauncher| I125: failed to listen on port 902, error 11: Resource temporarily unavailable... Exiting.

Hmm, indeed something is listening on port 902 that looks like VMware:

# sudo lsof -i -P -n | grep 902
vmware-au  1556    root   12u  IPv6  28200      0t0  TCP *:902 (LISTEN)
vmware-au  1556    root   13u  IPv4  28201      0t0  TCP *:902 (LISTEN)

I have my doubts that this is the ultimate problem, but I figured I’d HUP those suckers and see what happens. Of course, that didn’t do anything worthwhile. The processes listening on port 902 are left up and running after the failed installation, not idling there as the cause of the failure.

After tooling around with untaring the kmod and making it by hand, I saw enough errors that made me think there was a serious lack of… something on my install. This just didn’t seem right. As a result, I gave up and Googled, which brought me to this: https://kb.vmware.com/s/article/58533?lang=en_US

The short story to that long thread is that you should uninstall VMware Workstation, make sure that you have the proper prerequisite packages installed, then re-install VMware Workstation. In my case, that did absolutely nothing and I still had the same issue.

A bit more searching and I found this thread from 2017 that seems to have about a year of activity on it: https://communities.vmware.com/thread/568089. Apparently this is a fairly common issue that doesn’t have a very elegant solution. Basically, VMware Workstation’s latest version doesn’t support the kernel that I updated to, and I had to patch it. I blindly followed repomon‘s Apr 4, 2018 7:16 PM post, even though it was for VMware Workstation 12. Of course, that didn’t work well because it ended up compiling for an older version of vmmon and vmnet than what I had previously.

Some more Googling brought me to this SuperUser post: https://superuser.com/questions/1328401/vmware-workstation-vmmon-broken-on-ubuntu-18-04/1330890. Using that as inspiration, I realized that GitHub user mkubecek appears to be keeping up to date with the latest versions of VMware Workstation and Player products and creating appropriate patches to help work through this issue.

The specific script / solution I came to is described in the “My Solution” section above.