SVFF: An Automated Framework for SR-IOV Virtual Function Management in FPGA Accelerated Virtualized Environments

TL;DR

This work tackles the challenge of dynamic SR-IOV VF management for FPGA accelerators in virtualized environments. It introduces SVFF, a KVM-based framework that leverages the Xilinx QDMA SR-IOV implementation and a novel host-side pause feature in QEMU to reconfigure VFs with minimal guest disruption. The hardware relies on QDMA IP integrated into the bitstream, while the software layer provides init/reconf automation, QMP-based pause control, and libvirt/QEMU integration to automate VF attach/detach operations. The evaluation demonstrates that the pause approach offers comparable or better performance than traditional detach/attach, reducing downtime and enabling seamless FPGA resource reallocation in data centers and HPC workloads.

Abstract

FPGA accelerator devices have emerged as a powerful platform for implementing high-performance and scalable solutions in a wide range of industries, leveraging their reconfigurability and virtualization capabilities. Virtualization, in particular, offers several benefits including improved security by resource isolation and sharing, and SR-IOV is the main solution for enabling it on FPGAs. This paper introduces the SR-IOV Virtual Function Framework (SVFF), a solution that aims to simplify and enhance the management of Virtual Functions (VFs) on PCIe-attached FPGA devices in Linux and QEMU/KVM environments, solving the lack of SR-IOV re-configuration support on guests. The framework leverages the SR-IOV support in the Xilinx Queue-based Direct Memory Access (QDMA) to automate the creation, attachment, detachment, and reconfiguration of VFs to different Virtual Machines (VMs). A novel pause functionality for the VFIO device has been implemented in QEMU to enable the detachment of VFs from the host without detaching them from the guest, making reconfiguration of VFs transparent for guests that already have a VF attached to them without any performance loss. The proposed solution offers the ability to automatically and seamlessly assign a set of VFs to different VMs and adjust the configuration on the fly. Thanks to the pause functionality, it also offers the ability to attach additional VFs to new VMs without affecting devices already attached to other VMs.

Figures (2)

• Figure 1: SR-IOV Virtual Function Framework
• Figure 2: Device detached (left), attached (center), and paused (right)