PMX spam filter server and recieve emails for multiple domains in your exchange server

Recently, we receive alert from ISP about mail latency. Check Exchange server logs and everything looks fine. Compare logs. The process time between PMX and Exchange is quite short. In this case, either spam filter or routing is the source of the problem. Double check and no one has made any changes to the router.

Check the PMX server. The load and iostat is high. Looks like the bottleneck is on the local disk. Increase memory does not really help. The only other option is to take the database out and migrate it to another server. However, we don't have enough space in the rack since we plan to retire one more rack. We get an appliance from Sophos ES4000. Very easy to setup. Once installed, no more performace issue.

We only config inbound email pt to the appliance. Outbound email will bypass the appliance. Basically make sure routing is setup correctly (point to your exchange server). If you are config to receive emails for multiple domain, make sure include those domain in the PMX config. No more sendmail.mc, pmx.conf modification and recompile. Other than routing setup on the appliance, make sure you have MX record setup correctly on your DNS server and ISP. Also, make sure your recipient policy in your Exchange server include the other domains you are going to receive emails in your Exchange server.

For example, your company has a domain called first.com. Your Exchange server is config to receive email sent to first.com. However, you register another domain for your other product called second.com. You can receive emails sent to second.com on your Exchange server. First, contact your isp to add a MX record (may take 1 to 24 hrs to replicate). Then go to your DNS server (most likely ur DC and add the MX record) . Basically, point the MX record for second.com to your exchange server for first.com (for example, mail.first.com). Add second.com to the accepting mail domain in your PMX appliance. Lastly, goto your Exchange server default recipient policy, add SMTP for second.com. Force update the recipient policy so you can receive emails immediately for your second.com.

Timeout on HBA

If you see Event ID 129 for your HBA as well as path connection drops by failover software,

The description for Event ID ( 129 ) in Source ( elxstor ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: \Device\RaidPort1.

Check your HBA driver, switch firmware and failover software. Also, make sure the SAN controller firmware is in support matrix. Sometimes, you may need to check the HBA settings to make sure those are supported by vendor. There are chance you may need to reapply firmware on HBA. Compare switch logs to see if GBIC requires replacement.

Storport vs SCSIport

If you use some older HBA, you will see there are two sets of drivers; storport and SCSIport. Bascially, Storport provides better I/O performance and widely used for new HBA.

For old HBA, you can try to use Storport. However, SCSIport is more stable.

Here is the Microsoft article
http://download.microsoft.com/download/5/6/6/5664b85a-ad06-45ec-979e-ec4887d715eb/Storport.doc

Security Configuration Wizard for Windows 2003 SP1

Security Configuration Wizard for Windows 2003 SP1 is a very handy tool to config security on Windows 2003 server with at least SP1 installed. I find out a lot of ppl don't know about this tool. Search the internet and find an introductory article about SCW.

The tool helps you config firewall, audting and services......

http://www.windowsecurity.com/articles/Security-Configuration-Wizard-Windows-Server-2003-SP1.html

Registry fix for Windows 2000 server

It happened more than a month ago. We had a Windows 2000 server which would be retired. It crashed in the weekend. Most of the important roles were migrated. However, our diesel backup power generator connected to this domain controller. So, I have to get it back up and running because my manager cannot locate the monitoring software for the backup power generator.

I tried all the startup mode such as safe mode and it refused to boot with a bluescreen. Checked the backup logs over the weekend. Seems like the disk was bad. Eventually, I try the registry fix for Windows 2000. There is a warning some of the data may be lost when using the tool. However, we don't have any critical info. So we goto the following site to download the tool.

http://support.microsoft.com/kb/830570

We get it back up in 30 min.

Iphone with Exchange

We got 3 iphone from States for testing. Currently, it is not available in Canada yet. So, have them unlocked and try to have it talk to the Exchange. One of the users install itune 7.4......result is iphone being locked. So, we are warned by the cellphone tech to use itune 7.3 or below. Setup Exchange side following Wilson's blog

http://blog.monkeykit.com/2007/08/31/how-to-setup-imap4-exchange-2003-email-to-sync-with-your-iphone/

I have to contact ISP to open up only the secure IMAP port. I got the receiving part working. However, sending is still an issue. Eventually, I am using my phone carrier smtp server. One thing I found out: iphone sucks a lot more bandwidth than Blackberry. Hopefully, it will support OMA soon. I still have trouble sending if I use webmail.mycompany.com as the outgoing smtp. The other thing I don't like is power consumption. I will go back to my blackberry for sure. Will play with the new toy for one more week then return it to my manager. I always drop cellphone on the ground by accident. So, iphone is definitely not my device.

Sharepoint

Port Server becomes more and more popular. Well, it is easy to setup and save a lot of work. Our company uses the free Microsoft SharePoint Service. Suppose it should save a lot of time. However, be very cautious with update. Everytime we install major service pack on our SharePoint Server, some of the function fail. The reason is the third party addon. It resets some of the permission and causes some of the pages won't display.

I check permission of each portals and they are set correctly. Eventually, we have to uninstall and reinstall the addon back. Everything is back to normal. Test first before install Service Pack for SharePoint. This is second time already. Future plan....put an identical virtual machine in VM to test ServicePack first.

Cannot run exe file in 2003 server

Try to run driver update on the SQL servers connected to the SAN this weekend. However, I keep receiving error message "Windows cannot access the specified device, path, or file. You may not have the appropriate permission to access the item."

It is caused by one of the Windows security update. You need to unblock it. Basically goto Properties of the exe file. Then click Unblock. You will have no trouble after that. Same thing applies to file such as chm file download from internet and email. Check MS article KB902225.

VLAN with NIC teaming on Blade server

We have 2 c-class blade servers with 4 NIC. On the c-class enclosure, there are only two ethernet switch. Currently, NIC teaming is config on the blade server. It is running ESX 3.0. One of the VM will be put on DMZ. The easiest way is to buy two more ethernet switch and install them on the c-class blade enclosure. Then config another NIC teaming just for DMZ.

Another way is to enable VLAN on the VMs. Then enable VLAN tagging on the switch. On the blade server is easy. Just open switch management GUI, enable VLAN tagging on the NIC ports where the ESX servers connect to and make sure the crossVLAN ports are configure on the same VLAN as the NIC. For the external switch, (in our case, our blade enclosure connects to a Nortel switch) our Nortel 8300 can definitely handle it. Need to contact Nortel to find out though.

Fixing VCB on my EVA4000

My datastore is residing on the EVA4000. I find out VCB is not working and complains about path not found and SCSI error. So I go through the guide and find out the LUN number has to be the same for both ESX server and VCB proxy (which is my backup and Virtual Centre Server).

To assign the same number on EVA4000 for the ESX host and VCS is easy.

1) Open up CommandView.
2) Goto Virtual disk properties
3) Present the luns using Assign Lun function. Then assign the same lun number to ESX hosts and VCS.
4) Don't forget to save the changes.
5) Go back to the VCB proxy (again, it is my VCS server and backup server). Goto MPIO and use NLB on the luns that holds the vmfs datastore.
6) This will turn off multipath and use dedicated path unless the path fails.

How to create vmfs datastore in ESX 3.0 and config NFS

Create VMFS datastore
First, create a Lun on SAN then present it to both ESX server (our setup has 2 ESX Server HA and DRS). After that, go back to VI client and follows steps below

Before creating a new datastore on a Fibre Channel device, rescan a Fibre Channel
adapter to discover any newly added LUNs. For more information, see “Performing a
Rescan” on page 131.
When you create a datastore on a Fibre Channel storage device, the Add Storage
wizard guides you through the configuration.

To create a datastore on a Fibre Channel device
1 Log into the VMware VI Client, and select a server from the inventory panel.
2 Click the Configuration tab, and click Storage (SCSI, SAN, and NFS) under
hardware.
3 Click the Add Storage link.
The Select Storage Type page appears.
4 Select the Disk/LUN storage type, and click Next.
The Select Disk/LUN page appears.
5 Select the Fibre Channel device you want to use for your datastore, and click Next.
The Current Disk Layout page appears.
6 Look over the current disk layout, and click Next.
The Disk/LUN–Properties page appears.
7 Enter a datastore name.
The datastore name appears in the VI Client and must be unique within the current
Virtual Infrastructure instance.
8 Click Next.
The Disk/LUN–Formatting page appears.
9 If needed, adjust the file system values and capacity you use for the datastore.
By default, the entire free space available on the storage device is offered to you.
10 Click Next.
The Ready to Complete page appears.
11 Review the datastore information, and click Finish.
This process creates the datastore on a Fibre Channel disk for the ESX Server host.
12 Perform a rescan.
See “Performing a Rescan” on page 131.
For advanced configuration, such as using multipathing, masking, and zoning, refer to
the SAN Configuration Guide.

*** if lun is equal to or smaller than 256 GB, leave block size at 1MB!!! ***

Before present luns to Virtual Center Server (my VCS is also the backup server), make sure your VCB is setup correctly in the past with the following steps.

If you do not perform this configuration step, data corruption for virtualmachines using RDM can occur.

Disabling Automatic Drive-Letter Assignment
All versions of Windows, except Windows 2003 Enterprise Edition and Windows 2003 Datacenter Edition, automatically assign drive letters to each visible new technology file system (NTFS) and file allocation table (FAT) volume. For Consolidated Backup, change this default behavior so that volumes are not automatically mounted on the proxy. To prevent Windows from automatically assigning drive letters to RDM

1 Shut down the Windows proxy.

2 Disconnect the Windows proxy from the SAN or mask all the LUNs containing VMFS volumes or RDM for virtual machines.

3 Boot the proxy and log into an account with administrator privileges.

4 Open a command‐line interface.

5 Run the diskpart utility by typing:

diskpart

The diskpart utility starts up and prints its own command prompt.

6 Disable automatic drive‐letter assignment to newly seen volumes by typing at the diskpart command prompt:

automount disable

7 Clean out entries of previously mounted volumes in the registry by typing at thediskpart command prompt:

automount scrub

(CAUTION If you do not perform this configuration step, data corruption for virtual machines using RDM can occur. VMware, Inc. 35Chapter 2 VMware Consolidated Backup)

8 Exit the diskpart utility by typing:

exit

9 Shut down Windows.

10 Reconnect the Windows proxy to the SAN, or unmask all previously masked LUNs containing either VMFS volumes or RDM.

11 Boot the proxy.

*** PPL always get confuse about umask lun and zoning. Unmask a lun is to present a lun to the host. Zoning is to control which servers can connect to the controller of a SAN or Tape driver. ***

Migrating VM from VMware server to ESX 3.0

It actually takes a long time for me to sort out what to do. Finally, I had my BES server migrated to ESX from VMWare Server.

First step, please check if your HDD in the VM is SCSI or IDE. If it is SCSI, it saves you a lot of steps. If it is IDE, consult VMWare technical support first before migrating since I find an knowledge base article in the VM website (ID 1881). It requires you to convert from IDE to SCSI virtual disk first (the article applies to ESX 2.5, not sure if it applies to ESX 3.0, again, contact VM for more info.).

Second step, export a virtual disk from VMware server to ESX3.x. Since I don't have NFS running on the VMware server, I need to move the VM to another server with NFS running. Go to the VMWare Server. Remove any snapshot running on the VM. After that, power down the VM. Copy the whole folder for the VM from VMWare server to my NFS Share in the Windows NFS server. telnet to the ESX server. mount the NFS share to the ESX server. then run the following command on the ESX server.

vmkfstools -i /src_folder/src.vmdk -d 2gbsparse /dest_folder/dest.vmdk

(where src_folder is my mount pt for the NFS share. dest_folder is a folder I create on the vmfs datastore. However, the /dest_folder can be any directory that the ESX host can acess).

This exports the source VMDK files into a series of COW (Copy-On-Write) files that are of maximum 2GB size. Once they're sitting in /dest_folder, you will need to re-assemble them into a monolithic flat VMDK file for use with ESX. Run the following command to re-assemble them.:

vmkfstools -i /dest_folder/dest.vmdk /vmfs/volumes//new_vm_disk.vmdk

Now, follow the steps in ESX 3.0 server configuration guide to import the VM

In the VI Client, create a new virtual machine using the Custom configuration option.

When configuring a disk, select Use an existing virtual disk option and attach theWorkstation or GSX Server disk you imported. If you experience bluescreen for Windows server, check the SCSI controller type you define for the VM.

Introduction

This is my first time to write a technical blog. After 2 yrs of SAN support, I accepted a job as network administrator last yr for a mid size non-profit organization. It's a new challenge for me. Compared to supporting SAN, it is totally different. In the past, we always joke about network administrator not reading manual. One of the senior tech like to use the word RTFM. Now, I am in their position and understand the challenge of the job. Being a network administrator, you need to know a bit of everything; from security, SAN, switching, servers and desktop issue. Since my organization has only one dedicated support technician who is supporting CRM as well, I have to help him out a lot of times. Good thing is, I don't need to deal with SQL which I have no knowledge of.

I will write down the main technical issues I face and how I resolve it. Feel free to email me if you have any technical knowledge to share with me. Enjoy surfing