VMware Server 2.0.1 on linux 2.6.29
Monday, June 29, 2009 at 12:49PM
Tux in Guide, Linux, Linux, Tips and Tricks, fedora, how-to, vmware

Good morning earthbound darlings! I ran into a problem recently after upgrading my kernel to 2.6.29. I run VMware server so that I can access an ancient Windows XP image that I turn to if ever I need Internet Explorer or some other arcane Windows program that simply will not run under Linux.

After my upgrade to 2.6.29 I found, to my horror, that VMware once again didn't work. I say "once again" because VMware often will not survive an upgrade to a higher kernel version. In the past, I always relied on the so-called "vmware-any-any" patch that a VMware guru would release which would fix the problem 9 times out of ten.

The VMware-any-any patch seems to have splintered a bit since I last used it, and there are different versions of it floating around the net with different names.

I figured I would just wait until VMware released a new version of their (free) server product, as that invariably fixes this type of kernel incompatibility problem as well.

The problem is, it's taking too long. And I needed to use my XP image. So I decided to do some Googling. Wouldn't you know it, there was an answer on the vmware boards that addressed a similar problem for VMware Workstation. I figured there couldn't be *that* much difference between the two products, and it turns out, there isn't. Well, not where fixing this problem is concerned anyway.

So here's the skinny:

If you're running VMware Server 2.0.1 on a 2.6.29 Linux kernel and the /usr/bin/vmware-config.pl command keeps crapping out on you with the following tex among all the other output:

/tmp/vmware-config0/vmmon-only/linux/driver.c:xxxx: error: too many arguments to function ‘smp_call_function’
make[2]: *** [/tmp/vmware-config0/vmmon-only/linux/driver.o] Error 1
make[1]: *** [_module_/tmp/vmware-config0/vmmon-only] Error 2

Then go to the following url and download the two scripts listed there. One is a patch, the other is a shell script that will automate the patch process.

As you may have noticed, the post is taken from VMware's own community forum, so there should be relatively low risk running them. I've also taken a look at the patch file contained therein, and can find nothing dodgy or creepy about the patch that is being applied to your VMware server modules.

Put both patches in the same directory and run the script by entering

sh sh vmware-6.5.2-modules-2.6.29-fix.sh

If you had Worsktation, things would probably go wonderfully and you would be done. With Server, unfortunately, running the above script throws an error message (don't worry, running the script doesn't change anything until the script knows it won't get any errors). The error message relates to the patch that it tries to apply to the "vmblock-only" code, which is not present in VMware server.

The solution is a no-brainer: Edit the patch file you downloaded, namely "vmware-6.5.2-modules-2.6.29-fix.patch" and remove the first 12 lines, the lines that relate to the vmblock-only code. This means you will be deleting all the lines starting from the line that reads:

diff -urN source-OLD/vmblock-only/Makefile source-NEW/vmblock-only/Makefile

and deleting all the way up to, but NOT INCLUDING, the line that reads:

diff -urN source-OLD/vmci-only/Makefile source-NEW/vmci-only/Makefile

After deleting the lines, save the file, and then re-run the script again. In my case, the patch applied seamlessly and a subsequent run of

/usr/bin/vmware-config.pl

was able to recompile my VMware perfectly for the running kernel. I tested it, and much to my relief, I can now access my Windows XP image again. Just for old time's sake.

Don't forget, when VMware eventually does release a new server version (greater than 2.0.1), chances are it will work without the above hack.

Hope this helps someone else. I carried this out on a Fedora 10 install. Although other distro's should work equally well. The important thing here is the kernel version (2.6.29) and VMware version (2.0.1).

Until next time, earthlings.

Tux lives

 

Article originally appeared on dutchtechies (http://dutchtechies.com/).
See website for complete article licensing information.