QmailtoasterMain Page | About | Help | FAQ | Special pages | Log in

Printable version | Disclaimers | Privacy policy

VMware

From Qmailtoaster

This page is under construction

Contents

VMware Hosting Products

VMware offers a variety of hosting products. Our experience has been primarily with Server version 2, which runs on a variety of operating systems. We have primarily used CentOS version 5 as the host OS.

We expect that ESXi, which runs on bare hardware (no other OS required), would perform at least as well as Server does on CentOS, and probably a bit better, especially on more recent hardware that supports virtualization technologies.

Our understanding is that VMware Server and ESXi are much the same, with the exception of the underlying kernel and OS. As such, many of the performance tuning considerations, especially of the guests, are the same in both environments, and also have a good deal of commonality with VMware desktop products (Workstation, Player, Fusion).

Please refer to VMware's Server FAQs and ESXi FAQS for more details.

Our approach here is to treat everything as being compatible with current VMware hosting products, and to specify any differences where we're aware of them. While there will continue to be various versions of VMware hosting products, it's difficult if not impossible for us to take into account every version of every product. In other words, please understand that YMMV depending on your environment. If you experience something different than what's contained here, please update this page with your experience.

If you want to use QmailToaster using VMWare ESXi with pre-built images you can follow these steps:

  1. You need a computer runing something like ESXi ( free ) or Virtual Box (free )
  2. In ESXi, File, Delpoy OVF Template.
  3. Select the OVF file you are using., select next many times.
  4. Name your VM, eg: Qmail
  5. Select destination drive.
  6. Select thin, ( you may not have this setting )
  7. Leave VM Network as default
  8. Finish deployment of OVF

When deployed:

  1. Select your New VM in the ESXi menu, edit any setting to suite,( RAM CPU etc)
  2. Power on VM, Select console, login & use the qmail server.
  3. Or login to : http://youripaddress/qcontrol

If you grabbed the VM from my site http://techyguru.com You should also grab the PDF, it has all the passwords etc

If you still need the vmdk format( Vmworkstation & player ), I can make it avaliable also. Please contact Madmac <sysadmin at tricubemedia dot com>

VMware Tuning

We think it's important to understand the reasons why various tuning options have been chosen, and why they affect performance. That's why we've provided an explanation here of the various settings we recommend (and use). However, you may prefer to cut to the chase.

Host

CPU

Memory

Storage

1. The disk system is probably the largest bottleneck in any system. A good disk system is even more critical in a virtual system. There are several points to keep in mind when setting up a reliable disk system.

Drive types
Some people think they will get 2x the performance from a SATA3 that they do from SATA2. Not so, sorry... For the most part, the speed difference is in getting the data from the buffer to the machine. The speed to get the data from the platter to the buffer is only slightly better. That being said, you would still be better with the SATA3.

2. RAID Like I mentioned earlier, disks are cheap. Running any server without some form of disk redundancy is just asking for trouble. There are many sources of information about RAID. There could be whole wiki's dedicated to RAID performance, setup, recovery, etc. I'm going to concentrate on the two most common, RAID-1 and RAID-5 and keep it basic.

3. Off host storage

Getting the disks off your host can be a great benefit to a virtual system. It unloads the mundane RAID tasks onto a separate system all together so your host can concentrate on the guests and not have to worry about the disk housekeeping. It also give you the benefit of running any virtual guest from any attached host. This is great if you need to take a host down for maintenance.

CAUTION: DO NOT try to run a guest from 2 different hosts at the same time!!! Go read that last line AGAIN! You WILL corrupt your guest beyond reasonable repair (no maybe about it!) You MUST shut down the guest on one host before starting up on another. Also, pay attention to the autostarts so this can't happen accidentally.

4. Recommended settings

   tmpfs /opt/tmp tmpfs size=3G 0 0
Used in conjunction with tmpDirectory = "/opt/tmp" in the /etc/vmware/config, this sets up the temporary file system for the memory mapped file created for each guest. This will put those files into memory. The size=3G is the amount of memory you want to allow the VM system to use for the temp files, the more the better. It can be expressed as a size or percentage. Some recommend the setting size=100%. If you browse the folder, you won't see the files as they are hidden and even showing hidden won't show them. There will be a couple small files in there so that will tell you it's working.
   noatime,data=writeback,noacl,nodiratime
There are some things linux looks at when it accesses files or folders that we don't care about for the drives holding the virtual machines. Why waste time on what we don't care about.
noatime -Tells the host to not bother changing or looking at the last access times attribute of the files.
noacl -Simply put, bypasses asking the server for permission to use the file and looks directly at the file permissions.
nodiratime -Same as noatime but for directories. (not needed if you use noatime)
data=writeback This needs to be researched more.

Network

VMware Server Config

The config file for the host is stored in etc/vmware/config. This is a simple text file with all of the parameters for the VMware Server host.

There are MANY settings in the config file. We are only presenting a few of our favorite items. PLEASE do not empty your config file and replace it with ours as ours is only a subset of the whole file. Also, backup your original config file before playing in here. To make any of these settings take effect, you must restart the vmware server service or reboot the host.

mainMem.partialLazyRestore = "TRUE"
- If you use snapshots, this will allow them to be restored in background without having to shut down the guest.
mainMem.partialLazySave = "TRUE"
- Allows Snapshots to be created in background without shutting down the guest.
mainMem.useNamedFile = "FALSE"
- VMware server creates a .vmem file and without this setting, will put it in the same folder as your guest. This setting, in conjunction with the tmpDirectory = "/opt/tmp" will place the files into the /opt/tmp folder which is in memory.
MemTrimRate=0
- NOT GLOBAL
prefvmx.minVmMemPct = "100"
- Keeps all guest memory in RAM. You can select a lower % but it can cause swapping.
prefvmx.useRecommendedLockedMemSize = "TRUE"
- Windows only???
sched.mem.pshare.enable = "FALSE"
- Windows only???
tmpDirectory = "/opt/tmp"
- Puts all VM teamp files in here instead of in the same folder as the guest. This folder needs to be created as tempfs with the etc/fstab before you can user it.

Guest

CPU

SMP

Unless you need the resources that additional threading provides, you do not need to enable more than one CPU in the guest. Running more than one CPU in a guest brings in overhead that isn't needed and can actually slow the machine down. I know it's hard, but in the virtual world you really have to change the "bigger is better" thought process. Think of it this way; Your host is a quad core CPU running 1.2ghz per core. That's 4.8ghz of processing power at VM's disposal. VM does a very good job of balancing that load among the guests. If you only have 1 guest, it has the whole processor (minus the small host overhead) to use for the guest. Do you really think giving the guest 2 CPUs will do any better? Like I said, if you have an application that uses multiple CPU threads then you MIGHT see an improvement but we're talking about QMT here.

Timekeeping

The CPU is perhaps the most complicated part of the virtual system. In that, the CPU clock gets top billing for complexity. When you think about it, the OS is used to talking to a chip that is happily ticking away at a rate controlled by hardware. Nothing will change that rate of tick. In the virtual system, that "chip" is now a layer of virtualization or simply, another program. If your unvirtualized OS gets bogged down with heavy a load, the real CPU is still ticking away at the same rate but the programs run slower. If you bog down the host of a virtual system, the guest's virtual CPU (which is software now) is subjected to the same bog so the virtual CPU ticks slower. The virtualization layer usually takes care of this by adding extra ticks or using some other compensation method. If your guest kernel can deal with this, you're OK. If it can't, the most obvious symptom will be the guest clock drifting sometimes minutes or hours up or down. It can actually get so bad that the guest can cause the host to bog and lock up just because the host and guest kernels are not cooperating very well. Sounds like real life at times.

There are more than a handful of kernels available and the best setting depends on which kernel you're using. Refer to the VMware Knowledge Base - Timekeeping best practices for Linux guests page for the setting(s) you should use. The setting you use is added to the "kernel" line in your /boot/grub/grub.conf file. Of course you'll need to reboot the guest for the setting to take effect.

Also, when you subsequently update your kernel, it's a good idea before rebooting to check to be sure that the update retained your kernel settings on the new kernel line in the /boot/grub/grub.conf file.

In addition to adjusting kernel settings for timekeeping, it is recommended to always use ntp as well to be sure the clock stays as accurate as possible. Refer to the link above or the Nutshell section below for recommended ntp guest configuration settings.

When using ntp in a guest, you should disable VMware Tools periodic timekeeping as well. Details are in the link above.

Memory

There is a slight additional overhead on the virtual machine monitor (VMM) for 32-bit guests with more than 896MB of memory. Allocate no more than 896MB of memory for your QMT 32-bit guest for best performance. It's not clear at this point whether or not this overhead is diminished by hardware assisted virtualization.

I don't recall where that came from. In the real world, a 32-bit guest running Server v2.0.0 and VMware-Tools hit a ceiling of a little over 640MB, above which cpu utilization went up drastically, like by a factor of 10. Overall performance was pretty much unusable ('top' alone took 10% of the cpu). I'll be continuing to attempt to diagnose this problem. I'm guessing that it is in the Memory Management component of VMware-Tools, because a similar guest with no Tools did not experience this problem.

Here's what the Best Practices for VMware vSpehere-4.0 has to say:

For best performance, consider the following regarding enabling VMI support:

Storage

Raw Device Mapping

Raw devices are useful in some situations. They do not necessarily provide a performance boost over VMFS filesystems according to some documentation I've read (I wish I could remember where), although they appear to perform significantly better in some situations. I'll write more on this as I obtain hard evidence.

Raw device mapping exists in Server v1 (and ESX 2.5+ I think). Server v2 can run Raw Devices, but the ability to create/modify them is not yet in the Server v2 gui interface, as of v2.0.2. Rumor has it that this omission was simply due to developer workload, so there's reason to believe it will be included in the gui of some future version.

The following steps describe how to create a Raw Device Mapping in a guest machine using the CLI. The procedure is rewritten (copied largely) from Using Raw Disks with VMware Server 2, although we won't be using Server v1 to create anything for us. Why am I documenting this? Because there should be more than one place on the 'net that says how to do this, plus I don't want to forget.

Here is a sample .vmdk file:

# MyRawDisk.vmdk (WDC WD1002FBYS-02A6B0)
# Disk DescriptorFile
version=1
CID=e8aa779f
parentCID=ffffffff
createType="fullDevice"

# Extent description
RW 1953525168 FLAT "/dev/disk/by-id/scsi-SATA_WDC_WD1002FBYS-_WD-WMATV5034836" 0

# The Disk Data Base
ddb.adapterType = "lsilogic"
ddb.geometry.cylinders = "121601"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.geometry.biosCylinders = "121601"
ddb.geometry.biosHeads = "255"
ddb.geometry.biosSectors = "63"
ddb.virtualHWVersion = "4"

If you use virtual hardware version 7, you'll also have these lines:

ddb.virtualHWVersion = "7"
ddb.encoding = "UTF-8"
ddb.toolsVersion = "0"

Network

VMware Guest config

A few topics ago we showed you at the host level is the /etc/vmware/config file that controls the global settings for all guests and the host itself. There is also a file for each individual guest as well. You'll find that file in the same folder with the rest of the guest files. It will have the extension .VMX. Again, here we'll show you some of our favorite settings and an brief explanation about each. Don't forget to copy off your original in case you totally hose this one.

autostop = "softpoweroff"
- When you shut down the host, "softpoweroff" will tell the OS of the guest to shut down. "poweroff" yanks the power cord. Some OSs don't mind have power removed abruptly, most don't care for it.
mainMem.useNamedFile = "FALSE"
- This is the same as the host config setting of the same name. It can be listed globally there or per guest in the .VMX.
MemAllowAutoScaleDown = "FALSE"
- ??
memTrimRate = "0"
- Memory Trim rate allows the host to reclaim memory that isn't being used by the guest. Setting this to 0 tells the host to not reallocate the memory.
mem.ShareScanTotal = 0 GLOBAL??, EXPLAIN
- Specifies the total system-wide rate at which memory should be scanned for transparent page sharing opportunities. The rate is specified as the number of pages to scan per second. Defaults to 200 pages/sec.
mem.ShareScanVM = 0 GLOBAL??, EXPLAIN
- Controls the rate at which the system scans memory to identify opportunities for sharing memory. Units are pages per second.
mem.ShareScanThreshold = 4096
-
sched.mem.maxmemctl = 0
-
sched.mem.pshare.enable = "FALSE"
-
workingDir = "/opt/tmp"
- Specifies where the swap file for the VM will go. This only takes effect if swapping cannot be avoided by any means. This shouldn't be needed in optimal situations.

In a Nutshell

This section wraps up everything above, and shows the various settings according to where they go.

Host

Kernel
apm=power-off
elevator=deadline
nohz=off
dev.rtc.max-user-freq = 1024
vm.dirty_background_ratio = 5
vm.dirty_expire_centisecs = 1000
vm.dirty_ratio = 10
vm.overcommit_memory = 1
vm.swappiness = 0
tmpfs /opt/tmp tmpfs size=3G 0 0
noatime,data=writeback,noacl,nodiratime
VMware
mainMem.partialLazyRestore = "TRUE" (i386 only?)
mainMem.partialLazySave = "TRUE" (i386 only?)
mainMem.useNamedFile = "FALSE"
MemAllowAutoScaleDown = "FALSE"
MemTrimRate = "0"
prefvmx.minVmMemPct = "100"
prefvmx.useRecommendedLockedMemSize = "TRUE"
sched.mem.pshare.enable = "FALSE"
tmpDirectory = "/opt/tmp"

Notes: You can use # to start a comment line to document your work. Also, there is no particular order to any lines in the /etc/vmware/config or .vmx files.

Guest

Kernel
elevator=noop
noapic
nolapic
nosmp (single cpu only)
timing setting(s) (varies by kernel)
vm.swappiness = 0
VMware
autostop = "softpoweroff"
mainMem.useNamedFile = "FALSE"
MemAllowAutoScaleDown = "FALSE"
memTrimRate = "0"
mem.ShareScanThreshold = 4096
mem.ShareScanTotal = 0
mem.ShareScanVM = 0
sched.mem.maxmemctl = 0
sched.mem.pshare.enable = "FALSE"
workingDir = "/opt/tmp"

Notes: You can use # to start a comment line to document your work. Also, there is no particular order to any lines in the /etc/vmware/config or .VMX files.

ntp
tinker panic 0
restrict 127.0.0.1
restrict default kod nomodify notrap
server 0.vmware.pool.ntp.org
server 1.vmware.pool.ntp.org
server 2.vmware.pool.ntp.org
driftfile /var/lib/ntp/drift
# server 127.127.1.0
# fudge 127.127.1.0 stratum 10

Note: The directive tinker panic 0 must be at the top of the ntp.conf file.

Note: the last 2 directives should be commented out if they exist.

0.vmware.pool.ntp.org
1.vmware.pool.ntp.org


Security

VMware security: Protecting a VMware environment (registration required) contains a nice overview of VMware security. It recommends the following settings be used in the guest vmx file to harden security:

isolation.tools.connectable.disable = "TRUE"
isolation.tools.copy.enable = "FALSE"
isolation.tools.diskshrink.disable = "TRUE"
isolation.tools.diskwiper.disable = "TRUE"
isolation.tools.log.disable = "TRUE"
isolation.tools.paste.enable = "FALSE"
isolation.tools.setInfo.disable = "TRUE"
isolation.tools.setguioptions.enable = "FALSE"
log.rotatesize = 100000
log.keepold = 10
tools.setinfo.sizeLimit = 1048576

Of course, if you don't run VM Tools (which is entirely feasible with QMT), the tools related parameters are ineffective.


Snapshots

One of the greatest assets of Virtual machines is the ability to take snapshots. The problem is that most tuning sites tell you not to run with snapshots but they don't explain what they mean. Therefore, people tend to shy away from them thinking there might be some detriment to their server. The statement to not run with snapshots is true but don't NOT use them. I'll explain why.

A snapshot is exactly what it sounds like. It's a picture of your machine at that time.

The way snapshots work is when you take a snap, it stops writes to the VMDK and starts writing all changes to a *snapshot.vmdk file. That is ALL changes get written forward with NO deletes. What I mean is that if you are running a snap and copy in 2 meg file the snapshot file grows by 2 meg. Now, if you delete that file, the snapshot file DOES NOT shrink by 2 megs, it merely marks the file deleted. It continues to use the original VMDK but only for reads. Now you see where the rub is, that file GROWS!! The sooner you get out of the snapshot the better.

There are a few terms related to snapshots that can be confusing.

  1. Take Snapshot - This one is easy, it creates a snapshot of the current machine
  2. Remove Snapshot - This rolls all changes into the original VMDK. and deletes the snapshot file.
  3. Revert to Snapshot - This sends the machine back to the original state it was at when you took the snapshot and deletes the snapshot file.

You don't run with a snapshot in the background. However, before you do any updates, you take a snapshot then perform your updates. If the updates run ok then you remove the snapshot and you're back to normal. If the updates blowup, you revert the snapshot and you're back to the way you were before the updates.

So now you see, using snapshots before installing service packs, OS updates, QMT updates, etc. is a good idea.

Now the cautions:


Backups

Any discussion of backups regarding a QMT install would be remiss without including mention of the QMail Toaster Plus backup and restore utilities. If all you need to backup is the mail, configs and database of QMT, then this is your answer. It will require less storage space and with a small amount of work, you can even use it to recover a specific file or files.

If your goal is to backup the entire virtual machine, then the best way is to leverage the snapshot system. The critical files we need to backup are the VMDK/s and the VMX. The rest are only logs and temporary files. If you read the section on snapshots above, then you saw that when you snap a machine, it puts the VMDK files in 'read only' mode. That's what we're depending on here because you can't copy them while they are in read/write. This script will take a snapshot of your machine then copy the VMDKs and the VMX files to the backup location. While this is happening, your machine is happily running with all file activity going to a special snapshot file. When the copy is done, it then removes the snapshot and compresses the copy. You don't even need to shut the guest down! You can schedule this with a cron job and even set the retention time (How many backups to keep.). The drawback is that if you need to recover only a file or two, you would have to mount the VMDK with another machine and retrieve the files. It's not hard when you understand it but the first time can be unnerving.

This backup script isn't a QMT piece and isn't maintained by the QMT team. Rather than copying it here, I'll link to the page which includes the instructions for it here.


References

Documentation


Online Resources

Retrieved from "http://wiki.qmailtoaster.com/index.php/VMware"

This page has been accessed 27,958 times. This page was last modified on 30 January 2012, at 09:01. Content is available under GNU Free Documentation License 1.2.


Find

Browse
Main page
Community portal
Current events
Recent changes
Random page
Help
Edit
View source
Editing help
This page
Discuss this page
New section
Printable version
Context
Page history
What links here
Related changes
My pages
Log in / create account
Special pages
New pages
File list
Statistics
More...