Tuesday, November 18, 2008

ESX intermittently disconnects from VirtualCenter

Error:
Sometimes an ESX host could intermittently disconnects from VirtualCenter. There are many things could cause it, but here are two common, but haven't paying much attention places that we need to look for.

Solution:
1) Use "df -h" Check to see if / directory in ESX Service Console is filling up. After / directory is clean up and wait for a while before reconnecting ESX to VirtualCenter.

2) When seeing a message like below in /var/log/vmware/hostd.log, it is time to upgrade the memory in service console, and change its default soft and hard limit. If the memory checker reach its hard limit, bad things could happen to ESX.

/var/log/vmware/hostd.log
"[2008-11-14 22:05:13.319 'Memory checker' 20937648 warning] Current value 138020 exceeds soft limit 122880."
"Unable to fork: Cannot allocate memory"

SC memory: With VirtualCenter 2.5 and above, it is easy to upgrade the ESX service console memory to 800MB, but it requires a reboot. See KB1003313  and KB1003501 in VMware's website.

Soft/Hard limit: Edit the config.xml under /etc/vmware/hostd and add the following into the <config> section:

<hostdWarnMemInMB>200</hostdWarnMemInMB>
<hostdStopMemInMB>250</hostdStopMemInMB>

Do a restart on hostd process by:
service mgmt-vmware restart 

No comments: