Cooperative Memory Management for Linux Guests on z/VM

by Dave Jones
April 8, 2008

Both approaches have strengths and weaknesses; both can support overcommitment of available real memory.

The z/VM hypervisor takes the second approach, mapping guest virtual memory into the real memory storage of the System z machine. If there aren’t enough real memory frames to contain all the active guests’ virtual memory pages, some pages are moved to expanded storage (XSTOR). Once expanded storage becomes full, the guests’ pages are moved from expanded storage to DASD paging space.

Figure 1 provides a simplified view of the z/VM memory management mechanism, showing some inactive virtual storage pages in each Linux guest. These inactive virtual memory pages must be recovered for use by other guests, whether Linux-based or not.



CMM assists in managing memory constraints in the system as they arise. Based on several performance variables obtained from the system and storage domain CP monitor data records, a resource manager, such as IBM’s Virtual Machine Resource Manager (VMRM), detects such constraints and notifies specific Linux virtual guests when this occurs. The guests can then take the appropriate action to adjust their memory utilization to relieve this system constraint. Figure 2 shows this process.



Once a system memory constraint is detected, the resource manager calculates how much memory each eligible Linux guest should release to relieve the constraint. It then sends each guest a SHRINK request as a CP SMSG command, indicating the amount of storage to release.

A Linux device driver named CMM receives these SHRINK messages. This device driver implements dynamic memory sizing in the Linux guest by a technique known as “ballooning.” The CMM driver allocates storage in Linux and then tells CP that it no longer needs to manage these pages. This has two beneficial effects:

1. It reduces the guest’s working set and effective memory footprint.

2. It forces the guest to reclaim pages in use for read and write cache.

When system memory is no longer constrained, another SHRINK command with a smaller absolute value is issued. These smaller SHRINK requests effectively instruct the Linux guests to reclaim some of the storage previously released.

By growing and shrinking these memory “balloons,” CP real storage usage can be managed to deal with changing memory requirements by individual Linux guest systems and the z/VM system overall. The CMM device driver, which can be either generated directly into the Linux kernel or dynamically loaded as a module, causes this action to happen by issuing a CP X’10’ Release Pages Diagnose command.

CMM vs. CMMA