For guests that support PCI hotplug (usually enabled via modules: acpiphp pci_hotplug) disks can be hotplugged at run time through the monitor (Human Monitor Protocol, HMP, aka -monitor). This page documents HMP commands used to hotplug virtio-blk and scsi disks into a Linux guest with PCI hotplug support enabled.

Verifying Guest Support

The following kernel config options must be enabled (included or as modules):

• CONFIG_HOTPLUG=y
• CONFIG_ACPI_HOTPLUG_CPU=y
• CONFIG_HOTPLUG_PCI=y

If not built in, then the the following modules need to be loaded *before* attempting hotplug operations in QEMU.

• acpiphp
• pci_hotplug

Adding a 'drive' object in qemu

Using HMP command drive_add we'll specify which bus and pass drive parameters to define a block driver in qemu.

% qemu-system-x86_64 -S -nographic -monitor stdio
QEMU 1.5.0 monitor - type 'help' for more information
(qemu) help drive_add
drive_add [[<domain>:]<bus>:]<slot>
[file=file][,if=type][,bus=n]
[,unit=m][,media=d][,index=i]
[,cyls=c,heads=h,secs=s[,trans=t]]
[,snapshot=on|off][,cache=on|off]
[,readonly=on|off][,copy-on-read=on|off] -- add drive to PCI storage controller

'drive_add' example

Here's a specific example, adding a raw disk, /tmp/test.img. Note we must specify if=none which allows qemu to use this drive with any guest visible device. This drive may be associated with a virtio-blk-pci or a scsi-disk, or even a usb-disk device.

(qemu) drive_add 0 if=none,file=/tmp/test.img,format=raw,id=disk1
OK

You can confirm your new device by querying the block subsystem.

(qemu) info block
ide1-cd0: removable=1 locked=0 tray-open=0 io-status=ok [not inserted]
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]
pflash0: removable=0 file=/usr/share/qemu/bios.bin ro=1 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
disk1: removable=1 locked=0 tray-open=0 file=/tmp/test.img ro=0 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0

Exporting an existing drive into the guest as a device

Once you've defined an a new 'drive' object, to make it visible to the guest it needs to be connected to a device. The typical device would be a virtio-blk-pci, or possibly a scsi-disk. Certain devices may require additional PCI devices. For example, the scsi-disk device does require that a PCI SCSI controller be present in the system. Virtio devices only require a PCI device, which is already present by default in QEMU.

Adding a new device is done via the device_add HMP command.

(qemu) help device_add
device_add driver[,prop=value][,...] -- add device, like -device on the command line

To enumerate all of the supported driver values, you can query the list in the monitor with:

(qemu) device_add ?
name "VGA", bus PCI
name "gus", bus ISA, desc "Gravis Ultrasound GF1"
name "tpci200", bus PCI, desc "TEWS TPCI200 IndustryPack carrier"
name "usb-storage", bus usb-bus
[...snip...]
name "virtio-blk-pci", bus virtio-bus

or via the command line:  qemu-system-x86_64 -device help 

'device_add' example

Here's a specific example. Let's add the drive from above (disk1) as a virtio-blk device, and let's track our new virtio-blk-pci device with a separate id, myvirtio1.

(qemu) device_add virtio-blk-pci,drive=disk1,id=myvirtio1

Confirm the device has been created by querying the pci bus.

(qemu) info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
      id ""
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
      id ""
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xffffffffffffffff [0x000e].
      id ""
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 0.
      id ""
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit prefetchable memory at 0xffffffffffffffff [0x01fffffe].
      BAR1: 32 bit memory at 0xffffffffffffffff [0x00000ffe].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id ""
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 8086:100e
      IRQ 0.
      BAR0: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      BAR1: I/O at 0xffffffffffffffff [0x003e].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      id ""
  Bus  0, device   4, function 0:
    SCSI controller: PCI device 1af4:1001
      IRQ 0.
      BAR0: I/O at 0xffffffffffffffff [0x003e].
      BAR1: 32 bit memory at 0xffffffffffffffff [0x00000ffe].
      id "myvirtio1"

At this point, the device should show up in the guest as well.

Removing disks from Guests

When removing a disk (hotplugged or otherwise) the order of operations matters. In particular, since ACPI hotplug requires guest cooperation, hotplug and hotunplug operations may not complete in the guest. To ensure that the hypervisor retains control of host resources, it is possible to break the relation between a host block device (QEMU's drive object) and the guest visible device (the 'device').

drive_del is responsible for removing the association between the guest and host device. Running this HMP command will close out the host-side resource and prevent any further IO to the device. drive_del should be used in the case that a device_del doesn't complete successfully.

Example of 'device_del'

The use of 'id' parameters is required when adding devices and drive objects so that they can be referenced when deleting them. Here is a list of block devices in the machine:

(qemu) info block
ide1-cd0: removable=1 locked=0 tray-open=0 io-status=ok [not inserted]
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]
pflash0: removable=0 file=/usr/share/qemu/bios.bin ro=1 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
usbdisk1: removable=1 locked=0 tray-open=0 file=/tmp/test.img ro=0 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0

Let's remove usbdisk1 from the system by removing the device.

(qemu) info usb
  Device 0.0, Port 5, Speed 480 Mb/s, Product QEMU USB MSD
(qemu) device_del usbdisk1
(qemu) info usb
(qemu) info block
ide1-cd0: removable=1 locked=0 tray-open=0 io-status=ok [not inserted]
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]
pflash0: removable=0 file=/usr/share/qemu/bios.bin ro=1 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0

Additional Examples

Hotplug USB Disk

Ensure you start QEMU with a usb controller, for example, the usb-ehci controller.

% qemu-system-x86_64 -S -nographic -monitor stdio -device usb-ehci 
QEMU 1.5.0 monitor - type 'help' for more information
(qemu) info pci
  Bus  0, device   0, function 0:
    Host bridge: PCI device 8086:1237
      id ""
  Bus  0, device   1, function 0:
    ISA bridge: PCI device 8086:7000
      id ""
  Bus  0, device   1, function 1:
    IDE controller: PCI device 8086:7010
      BAR4: I/O at 0xffffffffffffffff [0x000e].
      id ""
  Bus  0, device   1, function 3:
    Bridge: PCI device 8086:7113
      IRQ 0.
      id ""
  Bus  0, device   2, function 0:
    VGA controller: PCI device 1013:00b8
      BAR0: 32 bit prefetchable memory at 0xffffffffffffffff [0x01fffffe].
      BAR1: 32 bit memory at 0xffffffffffffffff [0x00000ffe].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0000fffe].
      id ""
  Bus  0, device   3, function 0:
    Ethernet controller: PCI device 8086:100e
      IRQ 0.
      BAR0: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      BAR1: I/O at 0xffffffffffffffff [0x003e].
      BAR6: 32 bit memory at 0xffffffffffffffff [0x0001fffe].
      id ""
  Bus  0, device   4, function 0:
    USB controller: PCI device 8086:24cd
      IRQ 0.
      BAR0: 32 bit memory at 0xffffffffffffffff [0x00000ffe].
      id ""

Now, add the drive and device.

(qemu) drive_add 0 if=none,id=usbdisk1,file=/tmp/test.img 
OK
(qemu) device_add usb-storage,id=usbdisk1,drive=usbdisk1 
(qemu) info usb
  Device 0.0, Port 1, Speed 480 Mb/s, Product QEMU USB MSD