View source for CentOS 8.x KVM Creating Windows VMs with fast disk input output (io)

[[Main Page|Home]] > [[CentOS]] > [[CentOS 8.x]] > [[CentOS 8.x virtualization|Virtualization]] > [[CentOS 8.x KVM|KVM]] > [[CentOS 8.x KVM Creating Windows VMs with fast disk input output (io)|Creating Windows VMs with fast disk input output (io)]]

While creating Windows VM use following recommendations along with mentioned references
* Use virtio drivers for Windows KVM VMs
*: https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso
*:After creating VM use option to "Customize configuration before install".  Add another storage device of type CD/DVD ROM and select above iso.  Then using OS installation to detect hard-disk choose "Load driver" and choose appropriate drivers from CD/DVD (Typically amd64/OS") folder.
* Specifically for Windows 10 all of above also leads to marginal performance.  '''Changing disk from qcow2 to raw has dramatic speed improvements (upto 10x)'''.  If snapshots are must then use LVM instead of raw format.
*: https://serverfault.com/questions/425607/kvm-guest-io-is-much-slower-than-host-io-is-that-normal
*: https://serverfault.com/questions/407842/incredibly-low-kvm-disk-performance-qcow2-disk-files-virtio
*: https://pve.proxmox.com/wiki/Performance_Tweaks
*: https://unix.stackexchange.com/questions/47082/how-to-improve-windows-perfomance-when-running-inside-kvm
* In case you cannot use raw format and use of qcow2 is required, then make sure metadata space is pre-allocated 
*: https://serverfault.com/questions/407842/incredibly-low-kvm-disk-performance-qcow2-disk-files-virtio
* Make sure cache is set to none (Preferred).  Cache option of write-through has very bad performance.  '''Using write-back cache with threads IO gives further 10x improvement''' over other types of cache but may lead to data loss in case of abrupt shutdown.  
*: http://www.ilsistemista.net/index.php/virtualization/11-kvm-io-slowness-on-rhel-6.html?start=4
*: https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ldvq/ldvq_kvm_quick_start.html
* IO Mode to native in case cache is none.  If cache is write-back then IO mode should be threads.
*: https://georgik.rocks/solution-kvm-guest-io-very-slow/
*: https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ldvq/ldvq_kvm_quick_start.html
*: Proxmox seems to be using IO native with cache none with detect-zeros on for its VMs.  ps output on a proxmox machine with Windows and Linux guests is pasted at end for reference.
* Avoid use of IOthreads as explained at
*: https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ldvq/ldvq_kvm_quick_start.html
*: https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.ldva/ldva_t_definingGuestDevices.html
*: This gives error that iothreads are not supported in latest CentOS 7
* Setting CPU model to host-model (host-passthrough) or core2duo might improve things CentOS 7.x Cloudstack 4.11 Guest VM CPU configuration for Windows
*: https://forums.unraid.net/topic/36777-kvm-windows-guests-100-disk-in-task-manager-and-very-slow/
* There are some hyper-V related settings suggested at 
*: https://forums.unraid.net/topic/36777-kvm-windows-guests-100-disk-in-task-manager-and-very-slow/
*: They seem to come by default if you select Guest OS as Windows 10 during VM creation


It would also make sense to change NIC from e1000 to virtio and install related network drivers from the virtio iso linked above.


==Noted benchmark speeds with a few configuration options==
Using ATTO Disk benchmark version 4.01.0f1 observed:
;Cache-no, IO-native, OS-windows10, Disk-virtio raw : Max write:67.29 MB/s, Max read:121.01 MB/s for 48MB block. 
;Cache-writeback, IO-threads, OS-windows10, Disk-ide raw : Max write: 577.88 MB/s, Max read: 602.35MB/s for 32 MB block
;Cache-writeback, IO-threads, OS-windows10, Disk-virtio raw : Max write: 3.83 GB/s, Max read: 6.08GB/s for 512KB block
;Cache-writeback, IO-threads, OS-windows-server-2016, Disk-virtio raw: Max write: 2.29GB/s, Max read:2.89GB/s for 48MB block

Note in a few cases higher read values were available for different block size.  The values above are chosen for max write speed for given configuration.


==Proxmox ps command output including kvm VM parameters==
Since Windows VMs on proxmox seem to work satisfactorily, we can refer disk configuration of proxmox VMs using ps output:
<pre>
root@pve3:/etc/pve/nodes/pve3/qemu-server# ps eaux | grep kvm
root     10682  5.6 11.6 5101516 3815164 ?     Sl   Jan13 4113:18 /usr/bin/kvm -id 125 -name Email-Gateway -chardev socket,id=qmp,path=/var/run/qemu-server/125.qmp,server,nowait -mon chardev=qmp,mode=control -chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5 -mon chardev=qmp-event,mode=control -pidfile /var/run/qemu-server/125.pid -daemonize -smbios type=1,uuid=b44cc125-90a0-42aa-8290-898542c21e9d -smp 2,sockets=1,cores=2,maxcpus=2 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg -vnc unix:/var/run/qemu-server/125.vnc,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 4096 -device pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e -device pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f -device vmgenid,guid=b1a83f6d-61fa-4c5a-a7d3-f149c9a7c7e0 -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device VGA,id=vga,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:a75d96656848 -drive file=/mnt/pve/iso-containers/template/iso/proxmox-mailgateway_6.1-1.iso,if=none,id=drive-ide2,media=cdrom,aio=threads -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -device virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5 -drive file=/mnt/pve/A1/images/125/vm-125-disk-0.qcow2,if=none,id=drive-scsi0,format=qcow2,cache=none,aio=native,detect-zeroes=on -device scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100 -netdev type=tap,id=net0,ifname=tap125i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on -device virtio-net-pci,mac=AE:4C:F3:3D:61:12,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300 -machine type=pc+pve1 LANG=en_US.UTF-8 PATH=/sbin:/bin:/usr/sbin:/usr/bin INVOCATION_ID=aa2e3079e06841dc9c375cb9975a073c JOURNAL_STREAM=9:26421 PVE_DAEMON_LOCK_FD=4 PVE_DAEMON_SOCKET_85=6 LC_ALL=C LVM_SUPPRESS_FD_WARNINGS=1
root     10686  0.0  0.0      0     0 ?        S    Jan13   0:03 [kvm-nx-lpage-re]
root     10731  0.0  0.0      0     0 ?        S    Jan13   0:00 [kvm-pit/10682]
root     24142 27.0 12.9 4814820 4225148 ?     Sl   13:54   1:17 /usr/bin/kvm -id 100 -name windows16-shared -chardev socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait -mon chardev=qmp,mode=control -chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5 -mon chardev=qmp-event,mode=control -pidfile /var/run/qemu-server/100.pid -daemonize -smbios type=1,uuid=30b63868-5460-4497-a988-3b254627a975 -smp 2,sockets=1,cores=2,maxcpus=2 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg -vnc unix:/var/run/qemu-server/100.vnc,password -no-hpet -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,hv_spinlocks=0x1fff,hv_vapic,hv_time,hv_reset,hv_vpindex,hv_runtime,hv_relaxed,hv_synic,hv_stimer,hv_ipi,enforce -m 4096 -device pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e -device pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f -device vmgenid,guid=fd9ffbeb-8a50-4950-b824-491dd0a235fd -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device VGA,id=vga,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:a75d96656848 -drive file=/mnt/pve/A1/images/100/vm-100-disk-0.qcow2,if=none,id=drive-ide0,format=qcow2,cache=none,aio=native,detect-zeroes=on -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=100 -drive if=none,id=drive-ide2,media=cdrom,aio=threads -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -netdev type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown -device e1000,mac=8E:31:32:63:40:27,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300 -rtc driftfix=slew,base=localtime -machine type=pc+pve1 -global kvm-pit.lost_tick_policy=discard LANG=en_US.UTF-8 PATH=/sbin:/bin:/usr/sbin:/usr/bin INVOCATION_ID=aa2e3079e06841dc9c375cb9975a073c JOURNAL_STREAM=9:26421 PVE_DAEMON_LOCK_FD=4 PVE_DAEMON_SOCKET_85=6 LC_ALL=C LVM_SUPPRESS_FD_WARNINGS=1
root     24146  0.0  0.0      0     0 ?        S    13:54   0:00 [kvm-nx-lpage-re]
root     24189  0.0  0.0      0     0 ?        S    13:54   0:00 [kvm-pit/24142]
root     27653  0.0  0.0   6072   892 pts/4    S+   13:59   0:00 grep kvm SHELL=/bin/bash PWD=/etc/pve/nodes/pve3/qemu-server LOGNAME=root XDG_SESSION_TYPE=tty HOME=/root LANG=en_US.UTF-8 SSH_CONNECTION=172.31.1.253 47490 172.31.1.172 22 XDG_SESSION_CLASS=user TERM=xterm-256color USER=root SHLVL=0 XDG_SESSION_ID=1337 XDG_RUNTIME_DIR=/run/user/0 SSH_CLIENT=172.31.1.253 47490 22 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin MAIL=/var/mail/root SSH_TTY=/dev/pts/4 OLDPWD=/etc/pve/nodes/pve3 _=/usr/bin/grep
root     29354  0.1  1.1 2690668 367376 ?      Sl   Jan08 134:12 /usr/bin/kvm -id 101 -name centos7-shared -chardev socket,id=qmp,path=/var/run/qemu-server/101.qmp,server,nowait -mon chardev=qmp,mode=control -chardev socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5 -mon chardev=qmp-event,mode=control -pidfile /var/run/qemu-server/101.pid -daemonize -smbios type=1,uuid=38baec15-ec56-4e25-9833-30de75ad9579 -smp 2,sockets=1,cores=2,maxcpus=2 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg -vnc unix:/var/run/qemu-server/101.vnc,password -cpu kvm64,+lahf_lm,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 2048 -device pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e -device pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f -device vmgenid,guid=f2c0d5d1-8ae5-406f-b4e9-c4f13209a455 -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device VGA,id=vga,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:a75d96656848 -drive if=none,id=drive-ide2,media=cdrom,aio=threads -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -device virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5 -drive file=/mnt/pve/A1/images/101/vm-101-disk-0.qcow2,if=none,id=drive-scsi0,format=qcow2,cache=none,aio=native,detect-zeroes=on -device scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100 -netdev type=tap,id=net0,ifname=tap101i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on -device virtio-net-pci,mac=92:44:3E:17:77:FE,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300 -machine type=pc+pve1 SHELL=/bin/bash PWD=/mnt/backup LOGNAME=root XDG_SESSION_TYPE=tty HOME=/root LANG=en_US.UTF-8 SSH_CONNECTION=172.31.1.253 48852 172.31.1.172 22 XDG_SESSION_CLASS=user TERM=xterm-256color USER=root SHLVL=1 XDG_SESSION_ID=44 XDG_RUNTIME_DIR=/run/user/0 SSH_CLIENT=172.31.1.253 48852 22 PATH=/sbin:/bin:/usr/sbin:/usr/bin MAIL=/var/mail/root SSH_TTY=/dev/pts/4 OLDPWD=/root _=/usr/sbin/qm LC_ALL=C LVM_SUPPRESS_FD_WARNINGS=1
root     29358  0.0  0.0      0     0 ?        S    Jan08   0:03 [kvm-nx-lpage-re]
root     29403  0.0  0.0      0     0 ?        S    Jan08   0:00 [kvm-pit/29354]
</pre>
Here, we can see that proxmox is creating both Windows and Linux VMs with qcow2 format with cache none and IO native.  It is interestingly setting detect-zeroes to on for all disks.



Refer:
* https://www.reddit.com/r/VFIO/comments/80p1q7/high_kvmqemu_cpu_utilization_when_windows_10/
* https://forum.proxmox.com/threads/high-cpu-load-for-windows-10-guests-when-idle.44531/#post-220824
* https://www.reddit.com/r/VFIO/comments/9ec4k9/high_kvmqemu_cpu_utilization_when_windows_10/



[[Main Page|Home]] > [[CentOS]] > [[CentOS 8.x]] > [[CentOS 8.x virtualization|Virtualization]] > [[CentOS 8.x KVM|KVM]] > [[CentOS 8.x KVM Creating Windows VMs with fast disk input output (io)|Creating Windows VMs with fast disk input output (io)]]