Memory Management in Hyper-V (Part-2)

Guest Physical Address Spaces

The size of the GPA space for a partition is the range from 0 to some maximum address that depends on architectural attributes of the virtual machine exposed by the partition.

Each page within a GPA space is in one of three states:

Mapped: A mapped GPA page is associated with a RAM SPA page.

Inaccessible: An inaccessible GPA page may not be read, written, or executed by the partition.

Unmapped: An unmapped GPA page is not associated with a RAM SPA page.

The way in which the GPA space is defined and the behaviour associated with unmapped page accesses differs between the root partition and non-root partitions.

For the root partition:

Its GPA space is identity-mapped—that is, all mapped GPA pages map directly to the SPA page with the same address.

The GPA mappings are defined by the hypervisor at boot time or when SPA RAM ranges are added or removed. All valid SPA RAM pages are therefore always mapped within the root partition’s GPA space. The root partition is not allowed to unmap these. The root partition may, however, modify its access rights to its mapped pages.

Unmapped GPA pages within the root partition’s GPA space (that is, SPA pages that are not part of known SPA RAM ranges) can be accessed by the root partition. This allows the partition to access memory-mapped registers, video memory buffers and so on. Some unmapped pages are off limits even to the root partition. These include SPA pages that correspond to hardware resources that must be managed by the hypervisor for correct operation. For example, the hypervisor does not allow the root partition to directly access the local APIC’s memory-mapped registers.

For non-root partitions:

Its GPA mappings are not necessarily identity-mapped. That is, a GPA does not necessarily refer to the same SPA.

The GPA mappings are defined by the partition’s parent. At the time they are mapped (through a call to HvMapGpaPages), they are specified in terms of the parent’s GPA space. Therefore, these pages must be mapped into the parent’s GPA space; however, the parent is not required to have read, write or execute access to these mapped pages.

When a virtual processor accesses an unmapped GPA page, the hypervisor suspends the virtual processor and sends a message to the partition’s parent. Code within the parent will typically respond by creating a mapping or by emulating the instruction that generated the memory access. In either case, it is up to the software in the parent partition to “unsuspend” the child’s virtual processor.

Page Access Rights

Mapped GPA pages have the following attributes which define the access rights of the partition:

Readable: Data on the page can be read.

Writeable: Data to the page can be written.

Executable: Code on the page can be executed.

These access rights are enforced for explicit accesses performed by the child’s virtual processors. They are also enforced for implicit reads or writes performed by the hypervisor (for example, due to guest page table flag updates).

Access right combinations are limited by the underlying hardware. The following table shows the valid combinations for an x64 system.

Access Type			Description
Read	Write	Exec
·	·	·	Instruction fetches, reads, and writes are allowed
	·	·	Illegal combination
·		·	Instruction fetches and reads are allowed
		·	Illegal combination
·	·		Reads and writes are allowed
	·		Illegal combination
·			Reads are allowed
			No access is allowed

If an attempted memory access is not permitted according to the access rights, the virtual processor that performed the access is suspended (on an instruction boundary) and a message is sent to the parent partition. Code within the parent will typically respond by adjusting the access rights to allow the access or emulating the instruction that performed the memory access. In either case, it is up to the software in the parent partition to “unsuspend” the child’s virtual processor.

Memory accesses that cross page boundaries are handled in a manner that is consistent with the underlying processor architecture. For x64, this means the entire access is validated before any data exchange occurs. For example, if a four-byte write is split across two pages and the first page is writable but the second is not, the first two bytes are not written.

 

GPA Overlay Pages

The hypervisor defines several special pages that “overlay” the guest’s GPA space. The hypercall code page is an example of an overlay page. Overlays are addressed by guest physical addresses but are not included in the normal GPA map maintained internally by the hypervisor. Conceptually, they exist in a separate map that overlays the GPA map.

If a page within the GPA space is overlaid, any SPA page mapped to the GPA page is effectively “obscured” and generally unreachable by the virtual processor through processor memory accesses. Furthermore, access rights installed on the underlying GPA page are not honored when accessing an overlay page.

If an overlay page is disabled or is moved to a new location in the GPA space, the underlying GPA page is “uncovered”, and an existing mapping becomes accessible to the guest.

If multiple overlay pages are programmed to appear on top of each other (for example, the guest programs the APIC to appear on top of the hypercall page), the hypervisor will choose an ordering (which is undefined) and only one of these overlays will be visible to code running within the partition. In such cases, if the “top-most” overlay is disabled or moved, another overlay page will become visible.

Parent partitions that include instruction completion logic should use the hypercalls HvTranslateVirtualAddress, HvReadGpa, and HvWriteGpa to emulate virtual processor memory accesses correctly in the presence of overlays.

HvTranslateVirtualAddress returns a flag indicating whether the specified virtual address maps to an overlay page. HvReadGpa and HvWriteGpa perform the GPA access in the same way the specified virtual processor would have. If an overlay page is present at the specified address, the access is directed to that overlay page. Otherwise, the access is directed to the underlying GPA page.

When the hypervisor performs a guest page table walk either in response to a virtual processor memory access or a call to HvTranslateVirtualAddress, it might find that a page table is located on a GPA location associated with an overlay page. In this case, the hypervisor may choose to do any one of the following: generate a guest page fault, reference the contents of the overlay page, or reference the contents of the underlying GPA mapping. Because this behavior can vary from one hypervisor implementation to the next, it is strongly recommended that guests avoid this situation.

Source Hypervisor Functional Specification 1.0        

Memory Management in Hyper-V (Part-1)

Hyper-V supports isolation in terms of a partition. A partition is a logical unit of isolation, supported by the hypervisor, in which operating systems execute. A hypervisor instance has to have at least one parent partition.

The virtualization stack runs in the parent partition and has direct access to the hardware devices. The parent partition then creates the child partitions which host the guest OSs. A parent partition creates child partitions using the hypercall API, which is the application programming interface exposed by Hyper-V.

A virtualized partition does not have access to the physical processor, nor does it handle its real interrupts. Instead, it has a virtual view of the processor and runs in Guest Virtual Address, which (depending on the configuration of the hypervisor) might not necessarily be the entire virtual address space. A hypervisor could choose to expose only a subset of the processors to each partition. The hypervisor handles the interrupts to the processor, and redirects them to the respective partition using a logical Synthetic Interrupt Controller (SynIC). Hyper-V can hardware accelerate the address translation between various Guest Virtual Address-spaces by using an IOMMU (I/O Memory Management Unit) which operates independent of the memory management hardware used by the CPU.

Child partitions do not have direct access to hardware resources, but instead have a virtual view of the resources, in terms of virtual devices. Any request to the virtual devices is redirected via the VMBus to the devices in the parent partition, which will manage the requests. The VMBus is a logical channel which enables inter-partition communication. The response is also redirected via the VMBus. If the devices in the parent partition are also virtual devices, it will be redirected further until it reaches the parent partition, where it will gain access to the physical devices. Parent partitions run a Virtualization Service Provider (VSP), which connects to the VMBus and handles device access requests from child partitions. Child partition virtual devices internally run a Virtualization Service Client (VSC), which redirect the request to VSPs in the parent partition via the VMBus. This entire process is transparent to the guest OS.

Memory Types

The hypervisor architecture defines three independent address spaces:

System physical addresses (SPAs) define the physical address space of the underlying hardware as seen by the CPUs. There is only one system physical address space for the entire machine.

Guest physical addresses (GPAs) define the guest’s view of physical memory. GPAs can be mapped to underlying SPAs. There is one guest physical address space per partition.

Guest virtual addresses (GVAs) are used within the guest when it enables address translation and provides a valid guest page table.

The resource management state affects, and is affected by, any hypercall that needs to allocate or free memory in the hypervisor. Whether and how a particular call interacts with the resource management state are described as part of the specification for each call. For example, the section on partition creation will specify from whose memory pool the initial resources needed to set up a partition will be deducted.

Memory Pools

For each partition, the hypervisor maintains a memory pool of RAM SPA pages. This pool is managed like a bank account. The number of pages in the pool is referred to as the balance. Pages can be deposited into or withdrawn from the pool.

When a partition makes a hypercall that requires memory, the hypervisor draws the required memory from the pool. If the balance in the pool is insufficient, the call fails. If such a hypercall is made by one guest on behalf of another guest (in another partition), the hypervisor draws the required memory from the pool of the latter partition.

Pages within a partition’s memory pool are managed by the hypervisor. These pages cannot be accessed through any partition’s GPA space. That is, in all partitions’ GPA spaces, they must be inaccessible (mapped such that no read, write or execute access is allowed). In general, the only partition that can deposit into or withdraw from a partition is that partition’s parent.

When a partition is created, the memory pool is initially empty.

The following invariants are maintained:

All memory pool pages are RAM SPA pages that were previously mapped into the GPA space of the parent partition.

All pages in a partition’s memory pool must have been explicitly deposited into that partition’s pool; that is, memory does not migrate between memory pools of different partitions.

Pages in a partition’s memory pool cannot be accessed by any guest; that is, these pages are not mapped into any partition’s GPA space with read, write, or execute privileges.

NUMA Proximity Domains

The ACPI tables include topology information for all processors and memory present in a system at system boot. They also contain information for memory that can be added while the system is running without requiring a reboot. The information identifies sets of logical processors and physical memory ranges, enabling a tight coupling between logical processors and memory ranges. Each coupling is referred to as a Proximity Domain. The latency of memory accesses by logical processors within the same proximity domain is typically shorter than the latency of accesses outside of the proximity domain. The root partition can relay this affinity information to the hypervisor by selecting memory resources or specifying the proximity domain to be used with certain hypercalls.

Source Hypervisor Functional Specification 1.0        

Virtualization of Office Communications Server 2007 R2

Today I got this mail from Omar El Sherif  Microsoft TSP. this is some amazing news for all virtualization fans. finally the OCS will go virtual.

“We are pleased to announce that the Office Communications Group will soon release the virtualization story of Office Communications Server 2007 R2.

  With this type of server deployment, we will support both a fully distributed virtualized topology across several hypervisors and a single server virtualized topology. This will include ability to deploy those topologies onto Windows Server 2008 Hyper-V, or onto any Server Virtualization Validation Program (SVVP) certified partners (http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvp.htm). Note that there are some limitations to this support.

  Only the Presence, IM (including remote access, federation, and PIC) and Group Chat workloads will be supported. The following server roles can be deployed: Consolidated Front-End Servers, Back-End SQL Server 2008 64 bits, Group Chat Channel Servers, Group Chat Compliance Servers and Edge Access Servers. Those virtual machines will be running on Windows Server 2008 64 bits. The topology has been tested to handle up to 40,000 users, including 10,000 group chat users.

  All other workloads will not be supported as part of this announcement. That means voice, video, live meeting and application sharing workloads will not be part of the architecture, even for P2P communications. Therefore audio/video/web conferencing servers, audio/video/web edge conferencing servers, dial-in conferencing, archiving/monitoring server, Communicator Web Access, enterprise voice, or Remote Call Control will not be deployed as part of the “virtualized” pool. When these workloads are required, then a new pool with physical servers should be deployed for those users.

  In order to help customers scope their topologies, the product group will also release Microsoft® Office Communications Server 2007 R2 Capacity Planning Tool, capable of simulating user load for the available workloads. This will help customer to quickly validate the hypervisor load and scalability before going to production.

  Along with this future announcement, a whitepaper detailing the tested architecture, the performance, the way to setup the load simulation tool, and a methodology to select a successful architecture will be released.”

Support for the Microsoft unified communications clients in Application Virtualization environments

http://support.microsoft.com/default.aspx?scid=kb;EN-US;951152

Hyper-V Requirements for ISO Image Files Used in Self-Service

When self-service is implemented in a Hyper-V environment, additional configuration is required to enable self-service users to use ISO images.

The requirements are different for a stand-alone library server than for a library server that also serves as a Hyper-V host.

Hyper-V Library Server-If a library server that stores the ISO image files also is running Hyper-V, the following NTFS permissions are required:

” Give the machine accounts of all other Hyper-V servers Read permission on the library share and in the NTFS folder permissions”

” Add the Network Service account under which the Hyper-V Image Management Service is running to the library share and NTFS permissions. For Windows Server 2008, the default account is Network Service. “

Stand-Alone Library Server-If the library server that stores the ISO image files is not running Hyper-V, the following configuration updates are required:

” Give the machine accounts of all Hyper-V servers Read permission on the library share and to the NTFS folder permissions.

” Configure constrained delegation between the library server and each remote Hyper-V host”

On each Hyper-V server computer object, add the machine account for each library server to the list of services that that Hyper-V server is trusted to delegate credentials to.

You will find these settings on the Delegation tab of the Hyper-V computer object Properties dialog box in Active Directory Domain Services (AD DS). To make this change, you must have administrative rights in AD DS.

Upgrading from SCVMM 2008 to SCVMM 2008 R2 beta

This topic provides step-by-step instructions for upgrading to System Center Virtual Machine Manager (VMM) 2008 R2 Beta from VMM 2008 while retaining your previous VMM data. If you do not want to retain your existing data, uninstall VMM 2008, selecting the Remove data option on the Uninstallation Options page, and then perform a clean installation of VMM 2008 R2 Beta.

Upgrading to VMM 2008 R2 Beta from VMM 2007 is not supported. To upgrade VMM 2007 to VMM 2008 R2 Beta, you must first upgrade VMM 2007 to VMM 2008, and then upgrade to VMM 2008 R2 Beta. For information about upgrading VMM 2007 to VMM 2008, see Migrating from VMM 2007 to VMM 2008 (http://go.microsoft.com/fwlink/?LinkID=142430).

Caution

To avoid any loss of important data, you should upgrade and evaluate VMM 2008 R2 Beta only in a test environment. After you have upgraded a VMM 2008 database to VMM 2008 R2 Beta, there is no way to revert the database back to VMM 2008. If you decide not to continue on to the VMM 2008 R2 release, any data that you have collected after the upgrade will be lost.

Important

After upgrading VMM, all jobs that were run prior to the upgrade will appear in the job history; however, those jobs cannot be restarted after the upgrade. It is recommended that you allow all jobs to complete before upgrading the VMM server. Jobs that are canceled during the upgrade cannot be restarted after the upgrade is completed.

To upgrade from Virtual Machine Manager 2008

1.   Log on to a test computer on which a VMM 2008 server is installed. 2.   On the product media or network share, double-click setup.exe. Important If you set up a network share to install VMM components, you should apply the appropriate security to the share to ensure that no unauthorized or unauthenticated users can access or alter the contents of the share. 3.   On the Setup menu, click VMM Server to open the Upgrade System Center Virtual Machine Manager 2008 Beta Wizard. 4.   On the Confirmation page, verify the list of VMM components that are already installed on this computer, and then to upgrade all installed VMM components, click Upgrade. Note If you want to move one or more of the installed VMM components to a different computer for your VMM 2008 R2 Beta implementation, click Cancel, uninstall those VMM components from this computer, and then run setup.exe again. 5.   On the SQL Server Settings page, provide credentials that have permission to access and update the VMM database by doing one of the following: a.   To use the same credentials that you are logged on with, click Next. b.   To provide a different set of credentials, select the Use the following credentials check box, and then type the credentials. Important If any jobs are running when you started the upgrade, you will receive a warning message. You can do one of the following: It is recommended that you allow all jobs to complete before upgrading the VMM server. Jobs that are canceled during the upgrade cannot be restarted after the upgrade is completed. 6.   On the Completion page, click Close. Important You also must run the Upgrade System Center Virtual Machine Manager 2008 Wizard to upgrade any VMM 2008 components that are installed on other computers, including the VMM Administrator Console and the VMM Self-Service Portal. 7.   To run an on-demand host refresh, in an upgraded VMM Administrator Console, in Hosts view, select one or more virtual machine hosts, and then, in the Actions pane, click Refresh. You can check the status of the host refresh in Jobs view. 8.   After a host has been refreshed, it will have a status of Needs Attention, except for the VMM server if it is also serving as a host. This is because the stand-alone hosts still have the previous version of the VMM agent installed. These hosts are fully functional, so you do not have to immediately update the agents on all hosts. Note So that performance is not adversely affected, it is recommended that you update hosts in batches of 10 to 25. 9.   To update the hosts, in Hosts view, select one or more hosts with a status of Needs Attention, and then in the Actions pane, click Update Agent. You can check the status of the agent update in Jobs view. When the refresh is complete, the host status will be OK. 10.  In the VMM Administrator Console, in Library view, select one or more library servers, and then in the Actions pane, click Refresh. You can check the status of the library server refresh in Jobs view. 11.  After the refresh is complete, in the VMM Administrator Console, navigate to Administration view, and then click Managed Computers. 12.  Select one or more computers that have a role of Library and an agent status of Upgrade Available, and then, in the Actions pane, click Update Agent. You can check the status of the agent update in Jobs view. When the update is complete, the library server will have a version status of Up-to-date.

Source

https://connect.microsoft.com/content/

 

iSCSI Initiator on the Windows 2008 Server Core

 In some case you may try to install Hyper-V role in a server core installation. As Jose mentioned in his blog about Windows Server Failover Clustering with Hyper-V options. You have to use iSCSI SAN for Childs clustering.

In this  scenario, If you implement Windows Server 2008 Failover Clustering at the Hyper-V Child (Guest) level.

In this case, your shared storage must be an iSCSI SAN.

 

To use iSCSI SAN in server core you don’t have to install the iSCSI initiator on the Windows 2008 server core as it is a built-in feature. to use it follow these steps:

1. Logon as the Administrator on the Windows 2008 Server Core

 

2. At the command prompt and go to c:\Windows\system32

 

3. Use the command below to know all the iSCSI related command

 

Use the iSCSIcli /?

Backup Agent Causes Unsupported Cluster Configuration in SCVMM

It was an interesting case on the technet forum. Hyper-V Cluster is existing and the administrator trying to backup it using CommVault.

Every time the administrator try to backup the VMs he got Unsupported cluster configuration in the VMM, because clustered resource group is not configured correctly.

Microsoft point to same in KB 958184 in that scenario:

Hyper-V virtual machine files, such as the configuration.xml file or .vhd files, are saved on a volume in Windows Server 2008.

This volume is mounted on a failover cluster by using a volume GUID.

This volume does not have a drive letter or a mount point assigned.

You try to back up these virtual machine files by using a backup application, such as System Center Data Protection Manager 2007.

In this scenario, the backup operations fail and the virtual machine files cannot be backed up.

This problem occurs because the file path is misreported in the writer metadata that is sent to the backup applications by the Hyper-V Management service (Vmms.exe). Therefore, backup applications cannot recognize the volume, and the backup operations fail.

To solve this problem you have to install the hotfix in KB 958184.

 

What’s New in VMM 2008 R2 Beta

System Center Virtual Machine Manager 2008 (VMM) is a comprehensive management solution for managing virtualized infrastructure running on Windows Server 2008 with Hyper-V, Virtual Server 2005 R2 and VMware ESX through Virtual Center.

Recently, Windows Server 2008 R2 Beta was released which included significant feature improvements to Hyper-V—the underlying hypervisor platform. A corresponding beta version of VMM R2 – the next version of VMM – is due for release shortly. VMM R2 Beta leverages the new platform enhancements and extends the feature set of VMM 2008.

This overview highlights the most important new and significantly enhanced features in the VMM 2008 R2 Beta:
Support for new features of Windows Server 2008 R2 Beta

* Live Migration: – Seen through the VMM console, this enables administrators to move virtual machines from one machine in a virtual host cluster to another with no downtime. This allows administrators greater flexibility in responding to planned or unplanned downtime, provides higher machine availability and more robust fault tolerance within virtualized infrastructure.

The basic requirements for Live Migration are that all hosts must be part of a cluster and host processors must be from the same manufacturer. Additionally all hosts in the cluster must have access to shared storage. No changes are required to existing virtual machines, network, or storage devices in moving from Quick Migration to Live Migration other than upgrading to beta versions of Windows Server 2008 R2 and VMM 2008 R2.

* Hot addition/removal of VHDs: Allows the addition and removal of new virtual hard disks (VHDs) on a running virtual machine. This enables storage growth in virtual machines without downtime. Additionally, ‘live” VHD management allows administrators to take advantage of additional backup scenarios and readily use mission critical and storage-intense applications (eg: SQL Server and Exchange).

* New optimized networking technologies: VMM 2008 R2 Beta supports two new networking technologies – Virtual Machine Queue (VMQ) and TCP Chimney – providing increased network performance while demanding less CPU burden. NICS that support VMQ, create a unique virtual network queue for each virtual machine on a host that can pass network packets directly from the hypervisor to virtual machine. This speeds throughput as it bypasses much of the processing normally required by the virtualization stack. With TCP Chimney, TCP/IP traffic can be offloaded to a physical NIC on the host computer reducing CPU load and improving network performance.

Enhanced storage and cluster support

* Clustered Shared Volumes (CSV): Provides a single, consistent storage space that allows virtual hosts in a cluster to concurrently access virtual machine files on a single shared logical unit number (LUN). CSV eliminates the previous one LUN per virtual machine restriction and coordinates the use of storage with much greater efficiency and higher performance. CSV enables the Live Migration of virtual machines in and out of the shared LUN without impacting other virtual machines. Enabling CSV on failover clusters is straightforward and easy to monitor through the VMM administrator’s console; many storage configuration complexities prior to CSV have been eliminated.

* SAN migration into and out of clustered hosts: This allows virtual machines to migrate into and out of clustered hosts using a SAN transfer, which automatically configures the cluster nodes to recognize and support the new workload.

* Expanded Support for iSCSI SANs: Previously, only one LUN could be bound to a single iSCSI target whereas now — with VMM 2008 R2 Beta — multiple LUNS can be mapped to a single iSCSI target. This provides broader industry support for iSCSI SANs allowing customers more flexibility in choosing storage providers and iSCSI SAN options.

Streamlined process for managing host upgrades:

* Maintenance Mode: Allows administrators to apply updates or perform maintenance on a host server by safely evacuating all virtual machines to other hosts on a cluster using Live Migration or putting those workloads into a saved state to be safely reactivated when maintenance or upgrades are complete. Maintenance mode is enabled for all supported hypervisor platforms on Windows Server 2008 R2 Beta.

Other VMM 2008 R2 Beta enhancements

* Support of disjoint domains: Reduces the complexity of reconciling host servers with differing domain names in Active Directory and DNS. In these situations, VMM 2008 R2 Beta automatically creates a custom service principal name (SPN) configured in both AD and DNS allowing for successful authentication.

* Use of defined port groups with VMware Virtual Center: On installation, VMM 2008 R2 Beta will present available port groups for VMM’s use with VMware Virtual Center thus allowing administrators to maintain control over which port groups are used.

Pre-Register for the Virtual Machine Manager 2008 R2 Beta

Be among the first to get the VMM 2008 R2 Beta when it’s ready. Enroll at Microsoft Connect and receive an e-mail with download instructions the moment VMM R2 beta is available. Just sign in to LiveID and pre-register.