Hello,
I am pretty new to VMware/ESXi but I am taking ESXi 5.1 for a spin (free trial license) to see if it will become the hypervisor of choice for my home server. So far so good, until I wanted to get around the 2TB VMFS limit by passing my Adaptec 7805 card through directly to an operating system.
I cannot, for the life of me, get it to work right. I made sure I am using the compatible 1.2.1-29900 drivers on both the host, and the guest operating systems, but I am having some strange issues when actually trying to do the pass-through. I don't know if I am encountering a bug or just doing something wrong or unsupported with my license?
As expected, after enabling IOMMU on the host's BIOS, the Adaptec card (as well as my NICs) showed-up under Advanced Configuration as being eligible for pass-through/DirectPath. So, I of course enabled it on the PCIe Adaptec 7805.
I am able to get the device to be passed-through in the VM settings; and when I install the operating system Ubuntu shows the PCI card under "lspci":
inferno@amahi:~$ lspci | grep Adaptec
0b:00.0 RAID bus controller: Adaptec Device 028c (rev 01)
...and, if I try a Windows 2008 R2 VM, it will up as a "RAID Controller" with the yellow exclamation mark, as expected.
[...]
The headaches begin when I attempt to install drivers, on either type VM. NOTE, I am not attempting to use both VMs at the same time, I've tried these one at a time with the opposite VM completely deleted.
In Ubuntu 12.04, using the stock/default (v1.1.x) aacraid drivers does not cause me any problems, but it never sees my RAID10 array that's built in the BIOS of the controller card. I was assuming this might be because they don't match the version of drivers running on the host, so I've tried a myriad of things and I get pretty much the same results as soon as I tinker with the aacraid drivers. Here is what I've attempted:
- installing the 1.2.1-29900 Ubuntu 12.04 binary drivers from Adaptec
- compiling the 1.2.1-29900 dkms drivers myself and installing them
- installing the 1.2.1-33000 Ubuntu 12.04 binary drivers from Adaptec
- upgrading the host drivers to b33000 (as well as Ubuntu), however these are not listed as compatible on the VMWare site.
What happens after I install the aacraid drivers is pretty strange (and this happens with any of the four scenarios mentioned)... after an extended period of time after the first reboot following aacraid driver installation, I just get a black screen on the console. The GUI never will load, but eventually I get a text-based Ubuntu login prompt. This only lasts 5-10 minutes, then the console (but not the VM!) completely dies. Completely black screen, no mouse cursor, typing doesn't do anything yet the play button icon is depressed. The VM is not completely hung -- just the console, as I am able to log into the VM through SSH and poke around, but if I run "arcconf" (Adaptec controller mgmt utility for CLI), it will not detect a controller. dmesg / fdisk / udev all do not see the RAID array as /dev/sdb (or any other device name, for that matter).
In Windows 2008 R2, it also will not work. If I manually try to update the yellow exclamation mark "RAID Controller" driver to the appropriate drivers from Adaptec (same version running on the host), the entire VM instantly crashes (play button pops out, stop button depresses, and I get an error notification in the host's Event log that states:
"VMware ESX unrecoverable error: (vcpu-0) PCIPassthru: 01:00.0 tried to modify MSI-X vectors number 32-32, but maximum supported vector number is 31"
I actually have no idea what that means, and doing some preliminary Google searches on it have led me astray. Any help with this issue would be greatly appreciated; my goal is to get PassThrough working to the Ubuntu VM so it can become my file server with full access to the RAID10 array without having to worry about the 2TB VMFS limitation and for increased performance...
Here are my system specs for reference:
- ESXi 5.1 w/Update 1
- AMD Opteron 6300 16-core CPU
- 64GB ECC RAM (<-- I have read the free ESXi will only support 32GB RAM, but is that the case with the trial version as well? are my issues related?)
- SuperMicro H8SGL-F Motherboard
- Adaptec 7805 SAS RAID Controller Card
- 128GB SSD (attached to on-board SATA controller, for datastore/VMs)
Any help, tips, or suggestions would be greatly appreciated!