Duncansapien’s Techblog

Overview

VMware ESX3i is now a free product, comparable to the ESX software.

Quick Facts

• One can evaluate both ESX and ESX3i inside Workstation 6.5 - see "Running VMware ESX 3.5 and ESXi in Workstation" (great video blog)

• One can convert Workstation 6.x files to ESX3i format and upload them to the ESXi server from within the VMware Converter Software - see Carl Tyler’s Blog for an excellent screen-capture based guide on all this.

Overview

A 60-day evaluation of VMware Infrastructure 3.5 was downloaded, including VirtualCentre 2.5. This article describes the installation of VI3.5 and VirtualCentre software.

External Links

1. VMware Infrastructure 3 Documentation by VMware themselves, includes excellent PDF guides to installing and using ESX server. This should be the starting point for all queries on VI3.
2. This blog has some extremely helpful summaries on VMware - advanced level.

Physical Hardware Used

• SAN: OpenFiler - on a 2.8GHZ Fujitsu Siemens PC, 512Mb RAM and 75Gb EIDE single disk.
• ESX Server 3.5: 2 x HP DL380G5 servers, 2×74Gb disk in RAID1 (mirror) config, 12Gb RAM
• VirtualCenter 2.5: Installed inside an XP VM running inside VMware Workstation 6.0.
• Admininistrative PC on which Virtual Infrastructure Client 2.5 could be run, if required, or it could run from the same VM as Virtual Centre. Mentioned for completeness, not required.

Network Topology

We have one, slow network for management and another fast, data network for datastore (the VM disk files).

Management LAN: A NetGear 10/100 Hub to which both ESX servers, the OpenFiler SAN, Administrative PC and Virtual Center VM (via bridged networking) are connected on a live subnet (137.195.80.0) protected from other subnets by the University Firewall - which is why I can pop the IP details here safely (he hoped:o). Using a switch rather than a hub would have been preferable but the traffic volume is so low that it doesn’t matter in this test environment.

Data LAN: A low-cost NetGear Gigabit Switch to which both ESX servers and the OpenFiler SAN are connected, on a private subnet of 192.168.19.0.

Where the Downloads were stored:

Only for personal reference, downloads from VMware, which require registration, were saved to N:admininstallvmwarevmware_trial_29062008 where N: maps to Novell server \somfs2:apps.

Getting it all working

1. Virtual Centre 2.5
First of all, a VM was established on which to run the Virtual Centre 2.5 software. The VM was based upon XPSP2, assigned 768Mb RAM and has a 6Gb virtual HDD. This machine was named vc1.som.hw.ac.uk.

Important Amendment:

In the evaluation edition of VI3.5, VirtualCenter 2.5 uses Microsoft SQL Server 2005 as the underlying database. It has been found that the Virtual Infrastructure Client often cannot login to the Virtual Centre Service, when it has been installed on a Virtual Machine. Whilst there is a known bug with VMware Workstation 6 and SQL Server 2005 when using SCSI virtual hard disks, this occured for me using IDE disks. Changing the disk type to pre-allocated did not resolve matters.

Virtual Centre was thus installed directly on the VMware Workstation 6.0 PC and an IP alias of vc1.som.hw.ac.uk was setup to point to this physical PC.

In the notes that follow, when reference is made to the Virtual Machine of the Virtual Center software, please bear in mind it is now a physical machine.

IMPORTANT: Once registering on the VMware site, a Virtual centre (VC) download is made available named VC-1.4.1-36208.zip (60Mb). This is v1.4 and will NOT work with the ESX server version 3.5 also made available. It’s confusing, but the other VC download named VMware-VIMSetup-2.5.0-84782.zip (480Mb) is what’s needed, this is v2.5 of VC as the name suggests.

Installation steps follow those found on the VC 2.5 User Manual (accept the defaults). Local login names to the VM hosting Virtual centre 2.5 and IP numbers are omitted here.

2. ESX Server
A CD was created of VMware ESX Server 3.5 and installed into the first of the two HP GL380 servers. The management LAN interface was configured with an IP on the ‘80′ subnet and defaults were accepted for everything else. We now have server ESX1 online with local (server-based) storage only, which we’ll add to later with NFS storage.

3. Accessing Virtual Centre 2.5

• Once installed, we need to access Virtual Centre. This is accomplished using VMware Infrastructure Client 2.5 (abbrev. VIClient 2.5) which was run from within the VM on which VC was installed.
• The next step is to create a DataCenter - this is wizard-based and is just a container for ESX servers, etc. A Datacenter named ‘SML’ was created.
• Following this, one adds an ESX server to the datacenter. Right-click on the ‘SML’ Datacenter and ‘Add Host…". A wizard asks for the server name (ESX1) and credentials (root, and root password). After a few seconds, server ESX1 appears listed in the DataCenter.

We wish to add NFS-based storage to ESX1, but haven’t yet setup the DATA NIC nor the NFS share on OpenFiler, so now we attend to these tasks.

4. Configuring the Data Network on ESX1 Host

Screencaps below were made with FastStone Image Viewer.

• Still within VIClient 2.5, click on server object ESX1 and then on the tab named Configure.
  
• Under Configure, locate the Hardware section named Network Adapters.
  We see vmnic0 is using the 137.195.80 subnet, the management LAN.
  We see vmnic1 is unconfigured. This is our DATA LAN.
  
• Under Configure, locate the Hardware section named Networking.
  Top-right of this section is a blue hyperlink entitled "Add Networking…", click this.
  The ‘Add Network’ wizard starts. Of the three types available (Virtual Machine, VMKernel and Service Console) choose VMkernel as this "handles traffic for iSCSI and NFS".
  
• In the next screen, we create a new virtual switch to which the unused (as yet) physical nic named vmnic1 will connect. The defaults were for a switch named vmnic1, 1000Mbps and full duplex. (Yes, the switch name matches that of the NIC here). Networking is fully discussed in the ESX Server Configuration Guide (PDF).
  
• Next, we see a window in which we must complete Port Group Properties.
  I accepted the default Network label of VMkernel, left the VLAN ID empty, ticked the box marked "use this port group for VMotion". A private IP of 192.168.19.1 was assigned with subnet 255.255.255.0
• The next screen is a summary screen labeled "ready to complete", just click Finish.
  
• A popup appears saying there is no default gateway set, and do we wish to configure this now? As this is a closed LAN on which only the ESX servers and OpenFiler SAN will connect, there isn’t a need for a gateway so choose "no".
• The final screen shows we now have two virtual switches (networks). One for management, vSwitch0, and one for data, vSwitch1.
  

Adding the NFS Filestore to our ESX Servers

In a separate entry the configuration of an NFS share on an OpenFiler SAN is described.

We hopefully have an OpenFiler share named esx-nfs-share mounted on host 192.168.19.3 in location /mnt/esx/esxvol1/esx-nfs-share/. The root credentials on both ESX server and OpenFiler machine are identical. Now we wish to add the NFS file store.

1. On the Virtual Center (VC) computer, start VMware Infrastructure Client 2.5.
2. Browse to server esx1 and click on the Configuration tab and Storage link.
   
3. Observe the "Add Storage…" button, click on this. A wizard starts, select "Network File System" for the storage type.
   
4. In the next screen under Server enter 192.168.19.3 and for folder enter /mnt/esx/esxvol1/esx-nfs-share and under DataStore Name enter nfs_openfiler.
5. The NFS share should (and did) appear under the storage link.
   

We now have an OpenFiler NFS storage SAN attached to our ESX server.

Adding another VMware 3.5 Server: ESX2

The steps to adding ESX2 to Virtual Centre 2.5 and adding NFS storage are exactly as for ESX1 as described here.

Overview

VI3 has just been installed, now we need to test it by installing a virtual server (hereafter VS).

Starting points

1. VI3 has just been installed on one HP GL380 server (80.200)
2. VirtualCentre software and the FlexLM licence server are running on a VM (80.145)
3. ISO images of CD1 & CD2, Windows Server 2003 (180-day trial) downloaded to a shared folder of the above VM (C:install of the host PC, Duncan’s HP workstation).

Creating a first Windows Server 2003 VM: First Steps

1. Using VMware Virtual Infrastructure Client (hereafter VI-Client) from Duncan’s PC, connect to the ESX server (80.200) using username root and password (same as SML admin password). This is behind the Uni Firewall so hacking from outside isn’t going to work, folks.
2. Create VM named W2K03-a
   Assign 1 CPU, 2GB RAM, and a local 8Gb disk (we are not testing iSCSI, NAS or SAN yet).
3. A choice of virtual SCSI controller type exists for this server: BusLogic (VI3 default) or LSI (the default for VI3 W2003 server installations). Leave it at LSI - this is discussed here.
4. Does W2003 Server come with the LSI drivers in-built? Apparently yes - "Microsoft provides the port driver and LSI Logic provides the miniport drivers, which are called LSI_SCSI.SYS, LSI_FC.SYS and LSI_SAS.SYS." (source).

Creating a first Windows Server 2003 VM: Networking

1. We’ll create this VM on a private network, but also want to ‘activate’ it as we’re using a demo licence. So create two virtual LAN cards. One should be Bridged and use real IP 137.195.80.81, the other should be VMNET1 and use 192.168.19.5.
2. VMNET1 is host-only, internal to the VMware environment. This will permit connection to other VMs, such as Linux-based iSCSI target VMs which we don’t want on the real network. Note that in this setup, DHCP also works on VMNET1 but supplies IPs starting at 128, so we can safely manually assign IPs lower than that.
3. Using VI-Client, select the above VM - W2K03-a - and choose the Console tab.
4. Power on the VM and press F2 (BIOS). Change to Boot order to be CD/HDD (F10 save and exit, but not yet, do the next step first)
   Press CTRL + ALT to release the curser from the VM Console, as with all VMware products.
5. Insert the W2003 Trial CD into the ESX Server. Under the Summary tab, you need to ensure that the VM’s CD is set to use the ESX Server’s physical CD drive.
6. Restart the VM (from the VI-Client) and the W2003 Server setup should begin.
7. Use network settings -to be verified -
   IP: 192.168.19.5
   Subnet: 255.255.255.0
   Gateway: 137.195.80.90 - the real SML gateway
   DNS: Use the HW real DNS entries
8. Opt not to join a domain (as we haven’t any) but workgroup named ‘SML’.
9. Use the SML default admin password as the local admin password.

Starting up the new Windows Server 2003 VM for the first time

1. The VM installs W2003 and a login screen appears.
2. Use CTRL+ALT+INSERT instead of CTRL+ALT+DEL!
3. Login as Administrator. Windows asks for the second installation CD (drive D:). Insert this into the GSX server and proceed.
4. A licence type of ‘Per device or per user‘ was chosen and the computer name ‘w2k03-a’ entered.
5. A ‘Post setup security updates’ screen appears with options
   1) Update this server
   2) Configure automatic updates
6. The first option failed, probably because of the private network, so assign a real IP number of 80.127 and install VMware tools.
7. A Vmware tools popup appears after installation, advising to set
   Display Properties - Settings -Advanced -Troubleshoot - set ‘Hardware Acceleration’ to FULL.
8. A restart is advised.

Post-Installation, after the VMware Tools have been installed

1. Login (CTRL ALT INS) as Administrator.
2. The "Manage Your Server" Window appears.
3. A popup quickly appears from the system tray advising "14 days left to register your server". Follow the prompts to just activate Windows (not Register with Microsoft). A ping test to the SML gateway shows IP is working, as does a C:> tracert www.ed.ac.uk show that DNS is working.
4. Windows Activation proceeded without a problem.

Cloning this Virtual Server

Now that we have a basic W2003 Server working, it is a good idea to clone it so that subsequent servers can be quickly made. The steps are more fully described here.

1. On VMware Server PC 02, copy the VM files to C:Virtual machinesW2k03-a_sysprep
2. Start the copied VM in VMware Server and login.
3. Insert the W2003 CD2 and locate SupportToolsDeploy.cab
4. Right-click on each file and Extract… to c:sysprep (which you need to create).
5. Run the setupmgr.exe program and choose to create a new answerfile, type of setup: ’sysprep setup’ and licence agreement: ‘fully automated’. Complete all the information sections (computer name: generate, regional settings, screen resolution, product key etc). This is then saved to syspref.inf
6. Run Sysprep.exe and tick ‘Don’t reset grace period for activation’ then ‘reseal’. The VM then shuts down.

Interestingly, the VMware Server PC02 also shut down! On restarting, the VMware Server PC began the mini-setup … did I mix-up the real machine or VM above when installing sysprep? Nope, there is no c:sysprep on the VM Server PC, it was performed correctly.

No damage appears to have been done other than losing the network settings and profiles of each user, which were minimal. VMware PC02 is back in action after 5 minutes.

Alternatives to sysprep: Newsid (downloaded to N:adminutilssysinternalsnewsid)- see Techrepublic article.

Using the Sysprep Clone: w2k03-b

Let’s see if we can use the newly sysprep’d clone.

1. Copy the sysprep clone folder above to c:virtual machinesw2k03-b
2. Edit the .vmx file so that the display name of this VM is w2k03-b
3. Start the VM. Nope - no sysprep or mini-setup starts. No matter, change the following:
   Computer Name: w2k03-b
   IP: 192.168.19.6, subnet: 255.255.255.0, Gateway: empty, DNS: empty
   IP: 137.195.80.68, subnet: 255.255.255.9, Gateway: 137.195.80.90, DNS: as HWU
4. Newsid was downloaded to C:tools of this new VM and run, opting to generate a random sid and also rename machine to w2k03-b. The VM then restarted.

Issues

• Ping was not working between w2k03-a and w2k03-b, plus only one VM could ping the real gateway in HWU.
• Reason: On startup of w2k03-b, I had answered ‘no’ to the ‘config file has changed location, recreate?’ which meant that MAC addresses were not changed. These were manually changed in the.vmx file for w2k03-b and the server restarted. Also, the folder holding this VM was renamed for force a recreation of settings within the .vmx file including the VMware UUID - Universally Unique Identifier; see Setting the MAC address Manually).

Wallpaper: System Info

A small utiliity which displays system info as wallpaper was downloaded to each VM (c:toolswhoami) from http://www.kewlit.com/whoami/ and configured to run at startup.

a

Overview

• A trial of the VI3 software was obtained from HP, along with 2 x GL380 file servers each having 12Gb RAM and 2x 72Gb SAS HDDs (RAID 1 - Mirroring).
• All software and documentation are held under N:admindocsAdmin & TechnicalProjects2007 Oct - HP Virtualisation
• The background to this project is written up here
• An invaluable resource in all this is the VI3 Installation Guide (PDF)

Virtual Centre

The Virtual Centre product is software to control the whole business. It was installed inside a VMware Workstation 6.0 VM itself as follows:

• VM Host: Duncan’s Test HP workstation
• VM Guest: XPSP2, 6Gb IDE HD, 1Gb RAM, McAfee Enterprise 8.0i SP15, Novell Client 4.83, Windows patches to current date.
• VM Guest Network: IP 137.195.80.145 (bridged, sml-vm-xp01)
• VM Guest Location: Buffalo USB HDD, path: to-be-inserted
• VM Credentials: Login to SOMFS2 as network admin, login locally as SML PC Admin.

Installation

1. Virtual Centre v2.0.2-62327 hereafter abbreviated VC - was downloaded to C:temp of the VM, unzipped, and autorun.exe was run.
2. Choose VC Management Server 2.0 from the list that appears (top of list).
   Install to c:program filesvmwarevmware virtual center 2.0 as ‘typical’.
3. Choose the MSDE database server for installation ( Oracle/SQL Server also available).
4. "Install a local VMware Licence Server and configure VC Server to use it".
   "Use the browse button to locate the licence to be served".
   [Copy the .lic files from network to C:program filesvmwarelicences - after creating this folder - and choose VCMS.lic]
5. Enter the connection information for the web service:
   IP: 80.145 (not needed, here just as a reminder)
   https port: 443
   http port: 80
   Diagnostics Port: 8083
   VirtualCenter Port: 902
   Heartbeat Port (UDP): 902
6. Accept the XP Firewall unblocking of what follows.
7. VC Apache Tomcat: Will run on port 8086 (and set to run as a service).
8. Restart the VM

What do we find?

Under start menu - programs -vmware -vmware licence server - vmware licence server tools one sees the FlexLM licence server (as used in our Admin PC to licence Trados, etc).

Under start menu - programs -vmware -vmware web access - web access one finds a hyperlink to the local Apache Tomcat home page. Opening this, the Vmware Virtual Infrastucture Web Access page appears with a login form (Login Name/Password).

In the system tray is a new icon which expands to SQL Server Service Manager. It has two dropdown boxes entitled Server and Services, both of which are empty.

Creating the MSDE Database

The documentation (p60 of the PDF guide) suggests one must create the database for VC Server to use.

1. Open the Control Panels - Administrative Tools - Data Sources (ODBC)
2. Select the System DSN tab.
   One finds under name "VMware VirtualCenter" with driver "SQL Server" already here.
3. Highlight the above and click on the ‘Configure…’ button. Under ‘what name do you want to use to refer to the data source’ one finds ‘VMware VirtualCenter’. Click ‘Next’.
4. The authentication tab/window follows: Windows NT authentication is selected, with ‘Login ID’ that of the SML PC Admin account.

As the SQL Server Service Manager still doesn’t see this data source, restart the VM. No change, so leave this just now.

Go back to ( start menu - programs -vmware -vmware web access - web access ) Vmware Virtual Infrastucture Web Access page and login as SML PC admin account credentials. Success! But this doesn’t show very much so, we need some more software.

Virtual Infrastructure Client 2.0

The above wasn’t installed, so return to the autorun.exe and choose to install this on the VM.
"Warning 25800 - requires .NET v1.1 with Service Pack 1. For increased stability and performance please upgrade". The ‘finish’ button then said the installation was complete and a IE window opened prompting one to download .NET SP1. This was downloaded to N:admindocsAdmin & TechnicalProjects2007 Oct - HP VirtualisationdotnetSP1 then installed, and the VM restarted.

The Virtual Centre Client icon is available on the desktop, and requests a host, username and password. At a bit of a loss as what to do next, so continue with ESX Server.

Later addition: Doh! The IP information and login credentials required are those of the ESX Server machine installd below.

Installing ESX Server

One of the two HP GL380 server was chosen (2 x 72Gb SAS HDD, RAID 1 - mirroring, 12Gb RAM). The ESX Server software upgrade-esx-3.0.2-61618.iso was burned to CD and the server booted with CD in the drive.

The IP number used was 80.200, DNS registered at present as somwebcam.som (doesn’t matter for testing).

Details are written on paper beside the server, to be written-up here later, but a default partitioning of the local disk was chosen and root user password set (same as SML PC local admin password). The single disk is actually a RAID 1 affair with 65Gb available for formatting with VMFS.

Once installed, Virtual Infrastucture Client (hereafter VI Client) was opened on the VirtualCentre PC and the above IP used to connect, along with username root and root password (same as SML PC local admin password). This is pretty self-explanatory, allowing one to view or create VMs and assign who has permissions to access them.

Virtual Infrastucture Client was also installed on Duncan’s work PC - it’s just client software so can be installed anywhere.

The First Virtual Server: Windows Server 2003

A 180-day trial copy of Windows Server 2003 was obtained and downloaded to N:admininstallWindows Server 2003R2 Trial (start date of trial is Nov 18th 2007). Two CD images are available in .img format, but are apparently in ISO format. Renaming the file with an ISO extension permitted burning with Roxio Easy CD Creator.

Using the VI Client, a VM was created on the ESX server for the above server software. One locates the ESX server in the left-hand pane, right-click and choose "New Virtual Machine". A 12Gb HDD was chosen and 1Gb RAM allocated (just a small test system).

Licencing

Insert the W2003 Server CD1 into the ESX server then, with the VI Client software (on the VirtualCenter VM), right-click on the name of the VM created above and choose ‘Power On’. A licence warning/error is returned "There are not enough licenses installed to perform the operation", advising to click on the ESX Server ‘Configuration’ tab and select ‘Licenced Features’.

An ‘edit’ link is beside the Licence Features, with options to use a licence on the host, licence server or to upload local file. Select the last option and browse to C:program filesvmwarelicence and upload VI_singlehost.lic.This doesn’t help the matter. An identical problem is documented on the VMware Support forum.

FlexLM:

On the VirtualCenter VM, dig into the FlexLM licencing matters.
One creates a new system environment variable named LM_LICENCE_FILE with value 27000@137.195.80.145 and the FlexLM licence server looks for and parses all .lic files in the C:Program FilesVMwareVMware Licence ServerLicences folder (into which VCMS.lic, VI_Centralized.lic and VI_SingleHost.lic have been manually copied). Using the LMTOOLS diagnostics (icon on the desktop of the Virtualcentre VM), one can verify that the licences have been read by FlexLM.

However … the W2003 Server VM still does not start for the same licence error.

Try the following:

• Move all .lic files into a backup subdirectory, just keeping VI_Centralized.lic and restarting the FlexLM licence Server.
• Setting the ESX Server Firewall to minimal settings via
  root> esxcfg-firewall –allowIncoming –allowOutgoing
  root> service mgmt-vmware restart
• Using this great Troubleshooting VI3 Guide (PDF) - page 27 of this says to change the "ESX Server Licence Type" from "unlicenced" to "ESX Server Production" in the VI Client (done).

Success! The ‘Power On’ of the W2003 VM now starts without error.