performance
2345
Comment:
|
2343
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
= Performance Considerations = |
|
Line 4: | Line 6: |
Ubuntu currently supports 4k page size for all architectures except for ppc64el. While 64k page sizes are beneficial to certain memory bound benchmarks, but there is a penalty, it might be wasteful if you are dealing with small data structures that have to be page aligned. Also, 64k page size could break compatibility with old ARMv7 binaries. 64k page size will need to reconsidered with the introduction of 52bit VA. | Ubuntu currently supports 4k page size for all architectures except for ppc64el. 64k page sizes are beneficial to certain memory bound benchmarks, but there is a penalty, it might be wasteful if you are dealing with small data structures that have to be page aligned. Also, 64k page size could break compatibility with old ARMv7 binaries. 64k page size will need to reconsidered with the introduction of 52bit VA. |
Line 10: | Line 12: |
* Transparent huge pages | |
Line 15: | Line 16: |
It has been observed that on Cavium Thunder X2 setting the kernel command line parameter '''iommu.passthrough=1''', Flexible I/O Tester Synthetic Benchmark (Fio) performance (with 4k page size) was comparable to that of 64k pages. | It has been observed that on Cavium Thunder X2 setting the kernel command line parameter iommu.passthrough=1, Flexible I/O Tester Synthetic Benchmark (Fio) performance (with 4k page size) was comparable to that of 64k pages. |
Line 21: | Line 22: |
* Specify the number of hugepages, here I am using 512 as an example, you might need to change that depending on your use case. | * Specify the number of hugepages (''here I am using 512 as an example, you might need to change that depending on your use case.'') |
Performance Considerations
Table of Contents |
4k page size vs 64k page size
Ubuntu currently supports 4k page size for all architectures except for ppc64el. 64k page sizes are beneficial to certain memory bound benchmarks, but there is a penalty, it might be wasteful if you are dealing with small data structures that have to be page aligned. Also, 64k page size could break compatibility with old ARMv7 binaries. 64k page size will need to reconsidered with the introduction of 52bit VA.
Improve performance benchmarks with 4k pages
There are ways to get comparable performance using 4k page size and avoid the penalties of 64k pages.
- IOMMU Passthrough.
- Hugetlbfs
IOMMU Passthrough
Setting iommu.passthrough to 1 on th kernel command line bypasses the IOMMU translation for DMA, setting it to 0 uses IOMMU translation for DMA. This will need to be set at the time of deployment (using preseeds) or by editing the appropriate grub configuration files and reboot the system for the changes to take effect.
It has been observed that on Cavium Thunder X2 setting the kernel command line parameter iommu.passthrough=1, Flexible I/O Tester Synthetic Benchmark (Fio) performance (with 4k page size) was comparable to that of 64k pages.
Hugetlbfs
This is a runtime feature that can be enabled from userspace, and is currently supported by applications like Java, Qemu and benchmarks like Flexible I/O Tester Synthetic Benchmark (Fio).
Enable hugetlbfs
Specify the number of hugepages (here I am using 512 as an example, you might need to change that depending on your use case.)
sudo sysctl -w vm.nr_hugepages=512
- Mount the hugetlbfs
sudo mkdir /hugetlbfs sudo mount -t hugetlbfs none /hugetlbfs
Using hugetlbfs with Fio benchmark
With Fio benchmark you can enable mmaphuge for iomem and mem options.
sudo touch hugepages/file sudo fio -rw=read -blocksize=128k -iodepth=128 -buffered=0 -direct=1 -ioengine=libaio -runtime=180 -filename=/dev/nvme0n1 -name=test -time_based -group_reporting -numjobs=4 -iomem=mmaphuge -mem=mmaphuge:/home/ubuntu/hugepages/file -output=output.txt
ARM64/performance (last edited 2022-07-31 21:16:38 by xypron)