VMM tricks: PRO alerts sent twice

So This one comes from Microsoft TechNet forums http://social.technet.microsoft.com/Forums/en-US/virtualmachinemgrproscintegration/thread/dcf580e1-1366-4eed-967b-57b1062eba51/

 

Here you are the situation

You have PRO integration configured between SCOM 2007 R2 and VMM 2008 R2 SP1

PRO alert is being generated twice actually. There are 2 alerts in OM with the exact same time they were raised, the only difference is the alert ID.

Solution:

a) Administration->Settings->DB grooming in OM console.

b) Run Set-VMMServer -OpsMgrServer "" (empty string) from VMM powershell, this will reset OM-VMM integration.

[RHS] Error 5023 from ResourceControl for resource Cluster Disk

This week I got a call from one of my team mates (Nashaat Sorial) telling me that he is facing a problem with his Hyper-V cluster..

He is running 3 Hyper-V hosts with Windows 2008 R2 SP1 running multiple applications. One of the hosts keep failing and He can not Live migrate any VM to the other hosts until restart.

We went through all well known workarounds for such cases like

1- if you have AV please disable

2- Also check http://technet.microsoft.com/en-us/library/cc773533%28WS.10%29.aspx

3- Please if you have teaming..Break it and check again

4- Please check http://support.microsoft.com/kb/981618

5- Make sure that you enable the "file and printer sharing" as well as the "client for MS networks" on the heartbeat networks cards on all nodes.  http://support.microsoft.com/kb/2008795

So we have to start reading more in the cluster logs. You can generate the cluster logs as per

http://blogs.msdn.com/b/clustering/archive/2008/09/24/8962934.aspx

We got some interesting data and errors in the logs just like:

In Win2008 the Physical Disk resource type private property that stores the disk signature change from "Signature" and it is now "DiskSignature" in Win2008.  The lack of the DiskSignature property not being populated was resulting in the resource failing to come online

ERROR_CLUSTER_GROUP_MOVING(5908)’ because of ”Virtual Machine Configuration R-Web2003′ is owned by node 2, not 1.’

mm.. So it looks like something from the Hardware level. Searching HP blades errors come with more useful information

http://cb-net.co.uk/index.php?option=com_content&view=article&id=84:hp-bl460-asr-hpqilo2-issues&catid=3:windows-server-2003&Itemid=3

 

he HP Integrated Management Log shows ‘ASR Detected by System ROM‘ along with the following events in the System event log on an affected machine:

Event Type:    Warning
Event Source:    hpqilo2
Event Category:    None
Event ID:    57
Failed GET SENSOR READING, sensor 16
Event Type:    Warning
Event Source:    hpqilo2
Event Category:    None
Event ID:    57
NetFN 0×4, command 0x2D timed out

The solution for this was to perform the following:

• Install the latest ILO Firmware Update v. 1.81
• Install the HP iLO Management Channel Interface Driver v. 1.15.0.0
• Install the HP ProLiant iLO2 Management Controller Driver v. 1.12.0.0

All of these drivers can be downloaded form the following location, selecting your operating system:

http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=3288156?=en&cc=us&prodTypeId=3709945&prodSeriesId=1842750&taskId=135

VMM 2012: VM properties crash the console

When I start testing for VMM 2012 I face small problem. After Adding my host to VMM and start testing when I select VM properties crash the console

This happens for VMs running on Hyper-V Windows 2008 R2 SP1..this host used to be managed by VMM 2008 R2 SP1..then I removed the agent and installed VMM 2012 Agent.

VMM server is running on VM..When I right click on the VM and select properties it crash..but when I press properties from the top Pan I freeze…

I submit this error on Connect website and Richard Rundle from Microsoft was kind enough to give me help in that

Resolution

This is a known issue with the Beta. You can run the following script on the VMM Server to prevent the crash:

Get-SCVirtualMachine | where {$_.HostType -eq "VMHost"} | Get-SCVirtualNetworkAdapter | where {$_.logicalnetwork -eq $null} | Set-SCVirtualNetworkAdapter –NoLogicalNetwork

 

Now everything is working fine

Remove failed DC from AD manually… Never been easier

You perform metadata cleanup on a domain controller in the domain of the domain controller that you forcibly removed. Metadata cleanup removes data from AD DS that identifies a domain controller to the replication system. Metadata cleanup also removes File Replication Service (FRS) and Distributed File System (DFS) Replication connections and attempts to transfer or seize any operations master (also known as flexible single master operations or FSMO) roles that the retired domain controller holds.. Removing failed DC manual was hard process that need some level of professionalism as I used to do it with Ntdsutil command-line tool.

Please check “How to remove data in Active Directory after an unsuccessful domain controller demotion”

http://support.microsoft.com/kb/216498

How to remove orphaned domains from Active Directory

http://support.microsoft.com/default.aspx?scid=kb;en-us;230306

Clean up server metadata

http://technet.microsoft.com/en-us/library/cc736378%28WS.10%29.aspx

I used to use it since Windows 2000, 2003. But I was suprized to discover that Windows 2008, 2008 R2 has new GUI. Really easy and efficient one.

http://technet.microsoft.com/en-us/library/cc816907%28WS.10%29.aspx

Clean up server metadata by using GUI tools

When you use Remote Server Administration Tools (RSAT) or the Active Directory Users and Computers console (Dsa.msc) that is included with Windows Server 2008 or Windows Server 2008 R2 to delete a domain controller computer account from the Domain Controllers organizational unit (OU), the cleanup of server metadata is performed automatically. Previously, you had to perform a separate metadata cleanup procedure.

You can also use the Active Directory Sites and Services console (Dssite.msc) to delete a domain controller’s computer account, which also completes metadata cleanup automatically. However, Active Directory Sites and Services removes the metadata automatically only when you first delete the NTDS Settings object below the computer account in Dssite.msc.

As long as you are using the Windows Server 2008, Windows Server 2008 R2, or RSAT versions of Dsa.msc or Dssite.msc, you can clean up metadata automatically for domain controllers running earlier versions of Windows operating systems.

Membership in Domain Admins, or equivalent, is the minimum required to complete these procedures. Review details about using the appropriate accounts and group memberships at Local and Domain Default Groups (http://go.microsoft.com/fwlink/?LinkId=83477).

To clean up server metadata by using Active Directory Users and Computers

1. Open Active Directory Users and Computers: On the Start menu, point to Administrative Tools, and then click Active Directory Users and Computers.
2. If you have identified replication partners in preparation for this procedure and if you are not connected to a replication partner of the removed domain controller whose metadata you are cleaning up, right-click Active Directory Users and Computers <DomainControllerName>, and then click Change Domain Controller. Click the name of the domain controller from which you want to remove the metadata, and then click OK.
3. Expand the domain of the domain controller that was forcibly removed, and then click Domain Controllers.
4. In the details pane, right-click the computer object of the domain controller whose metadata you want to clean up, and then click Delete.
5. In the Active Directory Domain Services dialog box, click Yes to confirm the computer object deletion.
6. In the Deleting Domain Controller dialog box, select This Domain Controller is permanently offline and can no longer be demoted using the Active Directory Domain Services Installation Wizard (DCPROMO), and then click Delete.
7. If the domain controller is a global catalog server, in the Delete Domain Controller dialog box, click Yes to continue with the deletion.
8. If the domain controller currently holds one or more operations master roles, click OK to move the role or roles to the domain controller that is shown.You cannot change this domain controller. If you want to move the role to a different domain controller, you must move the role after you complete the server metadata cleanup procedure.

To clean up server metadata by using Active Directory Sites and Services

1. Open Active Directory Sites and Services: On the Start menu, point to Administrative Tools, and then click Active Directory Sites and Services.
2. If you have identified replication partners in preparation for this procedure and if you are not connected to a replication partner of the removed domain controller whose metadata you are cleaning up, right-click Active Directory Users and Computers <DomainControllerName>, and then click Change Domain Controller. Click the name of the domain controller from which you want to remove the metadata, and then click OK.
3. Expand the site of the domain controller that was forcibly removed, expand Servers, expand the name of the domain controller, right-click the NTDS Settings object, and then click Delete.
4. In the Active Directory Domain Services dialog box, click Yes to confirm the NTDS Settings deletion.
5. In the Deleting Domain Controller dialog box, select This Domain Controller is permanently offline and can no longer be demoted using the Active Directory Domain Services Installation Wizard (DCPROMO), and then click Delete.
6. If the domain controller is a global catalog server, in the Delete Domain Controller dialog box, click Yes to continue with the deletion.
7. If the domain controller currently holds one or more operations master roles, click OK to move the role or roles to the domain controller that is shown.
8. Right-click the domain controller that was forcibly removed, and then click Delete.
9. In the Active Directory Domain Services dialog box, click Yes to confirm the domain controller deletion.

VMM Tricks: VMM Self Service Portal and NLB

A very interesting question in Technet forums about VMM SSP high availability

“Virtual Machine Manager does not support Network Load Balancing (NLB) clusters in Windows Server 2008, which are required in order to distribute the network traffic among self-service users on multiple Web sites.

what problems or issues present this configuration?”

Well, a basic bing did not get any answer for that so I decided to ask a friend “Brandon” from MS Support and goth that answer

“I understand you are concerning why VMM SSP doesn’t support NLB for load balancing. Please correct me if I have any misunderstanding.

We had  intensively discussed this limitation with our development team previously and the main reasons are that it is not a tested scenario and SSP is not stateless.

1. The main thing is that the SSP is not stateless,  thus when a user connects to it he/she can’t bounce around without a loss of state.

2. we haven’t tested this scenario as a major scenario for VMM 2008 R2.

3. We know customers that are using it for fault tolerance purposes. In order for this to work you need to enable persistence on your load balancer.

Currently , I think the only gain from NLB would be to protect against the web server failure, in which case only the users who were assigned to that webserver would get booted off, but when they login again they would hit the remaining server(s) and be OK. So you can get some mild reliability enhancements but not really performance gains (loading balance). “

I had tested SSP load balancing before and it was fine. I did not get the cahnce to test it in huge production environment but I think I will try to do that.

That all for now folks..

Batch Script to Auto Update #Sysinternals Tools

While the Microsoft Sysinternals tools are incredibly powerful and useful, the one feature they lack is the ability to check for new versions. Currently, you have to periodically check the Sysinternals site and compare versions between your system and the most recent official release in order to stay up to date.

As a better solution, Jason Faulkner have created a batch script which will automatically update the Sysinternals tools you have on your system. All you have to do is put the batch script file into the folder where your Sysinternals tools are located and the script does the rest, no configuration is needed.

Check it there

VMM Tricks: VMM Domain Function Level … Why

So most of use already knows that VMM 2008 R2 required Windows 2003 Domain level for the installation and I already blogged about some error that you may face if VMM is connected and authenticated by windows 2000 domain controller in the installation phase.

But it was a new question when one asked me “Why Windows 2003 Domain Level?”

Kerberos authentication is a prerequisite for VMM. To configure your environment to allow users in one Active Directory Domain Services (AD DS) domain to access VMM resources in another domain, you can either ensure that both domains are in the same forest or configure a forest-level trust relationship and use Kerberos authentication. To set up a forest-level trust relationship, both domains must be in Windows Server 2003 forest mode. Windows 2000 Server does not support forest-level trusts.

Windows Server 2003 and Windows 2000 Server environments that contain complex group structures can encounter problems with an access token limitation during authentication.

The Kerberos Access Token in Windows 2000 native mode environment had many limitations and the resolution is just simply to raise domain function level to Windows 2003.

Check MS Addressing Problems Due to Access Token Limitation

http://download.microsoft.com/download/8/f/3/8f36dfe4-47d0-4775-ad5a-5614384921aa/AccessTokenLimitation.doc

VMM tricks: “Virtual Machine Manager lost the connection to the virtual machine because another connection was established to the machine”

After finishing my new test lab to test my new VDI solution I faced this problem.

My Lab:

• 2 Windows 2008 R2 Hyper-V hosts Clustered with failover cluster
• VMM 2008 R2 Server
• A new VM with Windows 2008

Finished my Cluster setup and Installed VMM on 3th server.. Add Cluster hosts went fine without any problems… Then I tried to connect my VM through VMM console and Ooops …

I got this error message “Virtual machine manager lost the connection to the virtual machine because another connection was established to the machine”.

I am using Enterprise administrator account since it is a test lab.. make sure that no other connection form Hyper-V console to this VM. Try to press Reconnect bottom.. No thing same error

mm.. trying to connect using Hyper-V Manager went fine.. Remote desktop connection no problems

Virtual Machine Manager is run as administrator. So I closed the console and right click the SCVMM console, and select “run as administrator”. No Changes .. grrrrrrrrrr.

Removing the Hyper-V Cluster and add it again… Same error

So I created a new User with domain admin privilege and add it to VMM admins group.. trying to connect and it is working.

So the problem was in my Enterprise administrator account.. looks like VMM is having problem with this account, So I deleted it from VMM User roles and added it again and guess what.. it is working.

VMM tricks: How to forcefully remove a Virtual Machine from the SCVMM admin console

Source

My buddy Jonathan Jordan recently came across an interesting issue and wrote up a great solution, and since he’s out today I thought I’d take the liberty of posting it.  If you ever find yourself in a position where you tried to remove a VM but it ended up being stuck in a state of limbo then this one’s for you:

========

Issue: A VM has been removed from SCVMM but still appears in the VMM Admin Console. The VM may have been removed in Hyper-V, SCVMM, or Cluster Administrator. When the VM is right clicked on there are two choices: Repair and Delete. Neither work.

Environment:
Windows 2003 with SCVMM 2007
Windows 2008 with SCVMM 2008

Resolution: If after attempting to remove the VM through normal means the machine name is still present, perform these steps:

> Method 1: Locate the .XML file for the machine and delete it.
- On the host that the VM ran on (or hosts if you are unsure which one), at a command prompt:
Type the drive letter to change to such as C: and hit enter, then…
dir *.xml /s | find /i “vm name”
- Replacing “vm name” with the name of the VM, this will search a volume for the .XML file that describes the VM
- Move the .XML file to a location that SCVMM does not use for hosted VMs or Library files
- Refresh SCVMM Admin Console

> Method 2: Force removal of VM with PowerShell.  If Method 1 did not work, follow the steps below then refresh the SCVMM Admin Console:

- In the SCVMM Admin Console click the ‘PowerShell’ button. Type ‘A’ for ‘Always’ if prompted by security
- Test PowerShell (this will just list information so that you will know that PowerShell is working properly):
Get-VMMServer -ComputerName localhost
- This should just list out some VMM info

- Remove VM: Replace “VM Name” with the name of the VM as it appears in SCVMM
$VM = Get-VM -Name “VM Name”
Remove-VM -VM $VM -Force

Note: If the virtual machine still exists as an .XML file as mentioned in Method 1, refresher will repopulate the VM as being present

VMM tricks: How to manually remove a host from the VMM database

Source

The following steps outline the procedure to manually remove a host cluster from the VMM database. In this specific example, we are removing a two node cluster.

Note The GUIDs listed below are specific to this example. If following these instructions, you will need to determine this same information for your specific environment. This information can be found in the tbl_ADHC_Host.

The following steps outline the manual process of removing all references to the existing host from the VMM database. This should only be used as a last resort option after all other means have been exhausted.

1. First, stop the VMMservice.
2. Then, take a backup of the existing DB using SQL Management studio. (Right click on DB name, choose tasks, click on Back up, chose a full backup).
3. Now, right click on table tbl_ADHC_Host and select view or open rows.
4. Click on the SQL button from the tool bar to get the SQL query for the view.
5. In there, start pasting the commands below one by one and verifying they executed correctly. (To execute, you can highlight the command and press F5 or press the “!” symbol).

DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE (HostID = ’85099616-fdc8-4f85-a702-979a4251f050′)
DELETE FROM tbl_ADHC_HostNetworkAdapter WHERE HostID=’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘
DELETE FROM tbl_ADHC_VirtualNetwork WHERE (HostID = ’85099616-fdc8-4f85-a702-979a4251f050′)
DELETE FROM tbl_ADHC_VirtualNetwork WHERE (HostID = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_ADHC_HostVolume WHERE (HostID = ’85099616-fdc8-4f85-a702-979a4251f050′) OR (HostID = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_ADHC_HostDisk WHERE (HostID = ’85099616-fdc8-4f85-a702-979a4251f050′) OR (HostID = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_WLC_PhysicalObject WHERE (HostId = ’85099616-fdc8-4f85-a702-979a4251f050′) OR (HostId = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_WLC_VObject WHERE (HostId = ’85099616-fdc8-4f85-a702-979a4251f050′) OR  (HostId = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_ADHC_Host WHERE (HostID = ’85099616-fdc8-4f85-a702-979a4251f050′) OR  (HostID = ’8cfff5ad-0990-4a92-bd6d-4d24190e5d5a ‘)
DELETE FROM tbl_ADHC_ClusterDisk WHERE (ClusterID = ’1148853c-cde3-45d2-82ee-bb04259d6a79′)
DELETE FROM tbl_ADHC_HostCluster WHERE (ClusterID = ’1148853c-cde3-45d2-82ee-bb04259d6a79′)
DELETE FROM tbl_ADHC_AgentServerRelation WHERE (AgentServerID = ’648e977e-d10e-40b6-aca2-d9d2dc63b790′)
DELETE FROM tbl_ADHC_AgentServer WHERE (ComputerName = ‘Server_Node1′)
DELETE FROM tbl_ADHC_AgentServer WHERE (ComputerName = ‘Server_Node2′)

6.  Now start the vmmservice and open the VMM console.