VMM with dual linux guests and TTY Multiplex

Introduction

For this activity the goal was to load two Linux Virtual Machines (VM) on one host. We built upon our previous single guest example and took inspiration from an existing Libvmm contribution that makes use of VirtIO console.

The repository for this project is located here.

Design

For this example five Protection Domains (PD) are used. Two PD’s are used for each virtual machine, one for the UART serial driver and the remaining two for the transmit (TX) and receive (RX) communication channels to handle the serial data. Input is routed to only one virtual machine and output from both VM's is seen regardless of which VM is currently being controlled. Currently there are only two VM's set up, but this can be tweaked to have multiple VM's running simultaneously.

RX

For RX communication, an interrupt is generated by a key press that is processed by the driver. The driver then adds the keypress to a shared queue and notifies the RX PD to read from the queue and place the key press into a new shared queue with the virtual machine PD. The RX PD will then notify the virtual machine PD that there is data in the queue. In the virtual machine PD this then notifies the VirtIO console to handle the RX data ready to be processed in the guest VM.

TX

For TX communication, the Linux guest puts data in a shared queue through VirtIO. The virtual machine PD receives a notification from the guest and the data is read from the queue and placed in a shared queue with TX PD. A notification is then sent to the TX PD so that it can read this data and place it in a shared queue with the driver. The driver then reads the data from the queue and outputs it to the hosts serial port.

VirtIO

VirtIO is an interface used by virtual machines to use its hosts’ devices through the means of virtual devices. The main role of these devices is to get data to and from the hosts physical hardware. VirtIO consists of three main parts: front-end drivers, back-end devices, and VirtQueues.

Front-end Drivers

Typically, the front-end drivers exist in the hosts kernel. These can be turned on in the config file when creating a Linux kernel with buildroot. They are responsible for accepting I/O requests from user processes, transferring I/O requests to its corresponding back-end VirtIO device and retrieving completed requests from the back-end device.

Back-end Devices

The back-end devices typically exist in the VMM. Libvmm provides implementations for console, network, block and sound devices. They are responsible for accepting I/O requests from the corresponding front-end driver,handling the requests by offloading the I/O operations to the host’s physical hardware, and making the processed and requested data available to the VirtIO driver.

VirtQueues

VirtQueues are shared memory regions that are used by a VirtIO front-end driver and back-end device pair. A VRing is a data structure contained within the VirtQueue that facilitates data transfer between the VirtIO device and driver. Libvmm follows version 1.2 of the VirtIO specification and uses the seL4 Device Driver Framework (SDDF) to implement the VirtQueues for specific devices.

VirtIO Console

In our example we make use of the VirtIO Console device to handle serial I/O data. To use VirtIO Console a node was created in the device tree and a multiple settings turned on in the Linux kernel config file. In the “chosen” node of the device tree, the stdout path and the boot arguments also needed to be set up specifically for the console device. In our initial implementation we passed through 8 additional devices. We then managed to remove the need for 7 of them by disabling them in the device tree. The remaining “gpc” device was needed by VirtIO Console, but at this stage it is not known why.

Timing Considerations

If the two virtual machine PD's have a different priority then the highest priority VM will be loaded first and there will be a long wait (>10 minutes) before control is relinquished to the lower priority VM. To resolve this issue, both of the Virtual machine PD's were made the same priority. Although it is not an issue, this results in the output being muddled together because both VM's output is shown simultaneously.