## page was renamed from ARM/Server/QA ||'''Table of Contents'''<
> <>|| = Arm Server QA Testing = == Overview == This page captures the current high level ARM Server QA progress. The results below are checks to ensure that work-flows, applications, and benchmarks have been attempted and are ideally functional on ARM Server so that these fundamental tools can be utilized by others to validate server functionality and performance. After the initial sweep these tests are planned to be additions to the automated platform testing to ensure this functionality remains consistent across releases/updates. Please feel free to discuss and then add items as seen fit for typical server functionality and workloads; testing help would also be greatly appreciated to ensure quality and thoroughness. == Overall Status == === Status Key === ||~-'''Color'''-~||~-'''Definition'''-~|| ||~-'''White/Clear'''-~||~-Test/Benchmark/Application is pending/untested/unknown-~|| ||~-'''Green'''-~||~-Test/Benchmark/Application has no glaring problems, installs/runs/passes-~|| ||~-'''Yellow'''-~||~-Test/Benchmark/Application has minor problems to resolve, partial failures-~|| ||~-'''Red'''-~||~-Test/Benchmark/Application has major problems, missing/fails/fails to install-~|| === Status Items === || '''Area''' || '''Pri/Test''' || '''Test-Versions''' || '''Release/Kernel''' || '''Date''' || '''Brief Notes, Bug links, Problem Links''' || || Application || P2 nginx || -- || -- || -- || -- || || Application || P2 ruby || -- || 11.10/2.6.38-1309-omap4 || 20110722 || none || || Benchmark || P1 pts/apache || 2.2.19-1ubuntu1 || 11.10/2.6.38-1309-omap4 || 20110720 || -- || || Benchmark || P1 pts/java-scimark2 || 6b23~pre3-0ubuntu2 || 11.10/2.6.38-1309-omap4 || 20110718 || pts/java-scimark2, PTS 3.2.0-1 || || Benchmark || P1 pts/openssl || -- || 11.10/2.6.38-1309-omap4 || 20110720 || fails install, PTS 3.2.0-1 || || Benchmark || P1 pts/pgbench || -- || 11.10/2.6.38-1309-omap4 || 20110718 || fails install, PTS 3.2.0-1 || || Benchmark || P1 pts/phpbench || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P1 pts/tiobench || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P1 pts/pybench || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P1 pts/pyopencl || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P1 pts/pyopencl || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P2 pts/aio-stress || -- || 11.10/2.6.38-1309-omap4 || 20110718 || fails install, PTS 3.2.0-1 || || Benchmark || P2 pts/ddbench || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P2 pts/cachebench || -- || -- || -- || -- || || Benchmark || P2 pts/fio || -- || 11.10/2.6.38-1309-omap4 || 20110718 || fails install, PTS 3.2.0-1 || || Benchmark || P2 pts/fs-mark || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P2 pts/hdparm || -- || -- || -- || -- || || Benchmark || P2 pts/iozone || -- || 11.10/2.6.38-1309-omap4 || 20110718 || PTS 3.2.0-1 || || Benchmark || P3 pts/idle || -- || -- || -- || -- || || Benchmark || P3 pts/idle-power-usage || -- || -- || -- || -- || || Benchmark || P3 pts/postmark || -- || -- || -- || -- || || Benchmark || P3 pts/stream || -- || -- || -- || -- || || Benchmark || P4 pts/systester || -- || 11.10/2.6.38-1309-omap4 || 20110718 || fails install, PTS 3.2.0-1 || || Benchmark || P4 pts/povray || -- || -- || -- || -- || || Benchmark || P4 pts/ramspeed || -- || -- || -- || -- || || Benchmark || P4 pts/scimark2 || -- || -- || -- || -- || || Benchmark || P4 pts/sqlite || -- || -- || -- || -- || || Benchmark || P4 pts/stresscpu2 || -- || -- || -- || -- || || Boot || P1 u-boot pxe || -- || 11.10/2.6.38-1309-omap4 || 20110720 || none || || Network || P1 loopback || -- || -- || -- || -- || || Network || P1 vlan || -- || 11.10/2.6.38-1309-omap4 || 20110722 || none || || Security || P1 IPsec || lp:qa-regression-testing || 11.10/3.0.0-1204-omap4 || 20110905 || || || Security || P1 Entropy || lp:qa-regression-testing || 11.10/3.0.0-1200-omap4|| 20110824 || Takes several days to run simple test. Weeks for full test. || || Server || P1 Raid1 || -- || 11.10/3.0.0-1203-omap4 || 20110824 || Use netinstall to image system || || Network || iSCSI || -- || 11.10/3.0.0-1204-omap4 || 20110905 || Requires a bit of manual configuration. iSCSI-root fails to boot. See [[https://bugs.launchpad.net/bugs/838809 |LP:838809 ]] || || Server || CEPH ClusterFS || -- || 11.10/3.0.0-1204-omap4 || 20110925 || || == Additional Testing Details and Notes == == Installation == === PXE Install === * SD Card Method 1. Format blank SD card as vfat (type LBA) and enable boot flag 1. copy MLO from latest u-boot-linaro-omap4-panda deb package - Must be first file copied to SD 1. copy u-boot.bin from latest u-boot-linaro-omap4-panda deb package 1. create pxelinux.cfg file on your tftp server following standard pxeconfig guidelines. Note the kernel and initrd need to have the u-boot crc added. Use mkimage to generate them or use the netboot installer files. == Test Suites == === Phoronix === * Setup: "sudo apt-get install phoronix-test-suite" * Run: "sudo phoronix-test-suite run [test] [test] ..." [Priority One - P1]: 1. pts/apache [System/Apache] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/java-scimark2 [Processor/Java] - Partial / tests 2,4,5 fail to run (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/nginx [System/Nginx] - Pending 1. pts/network-loopback [Network] - Pending 1. pts/openssl [Processor] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/pgbench [System/PostgreSQL] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/phpbench [System/PHP] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/tiobench [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/pybench [System/Python] - Pending 1. pts/pyopencl [System/Python] - Pending [Priority Two - P2]: 1. pts/aio-stress [Disk] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/ddbench [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/cachebench [Processor] - Pending 1. pts/fio [Disk] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/fs-mark [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) 1. pts/hdparm-read [Disk] - Pending 1. pts/iozone [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) [Priority Three - P3]: 1. pts/idle [Disk] - Pending 1. pts/idle-power-usage [Disk] - Pending 1. pts/postmark [Disk] - Pending [Other - Misc]: 1. pts/povray [Processor] - Pending 1. pts/ramspeed [Memory] - Pending 1. pts/scimark2 [Processor] - Pending 1. pts/sqlite [Disk] - Pending 1. pts/stream [Memory] - Pending 1. pts/stresscpu2 [Processor] - Pending 1. pts/systester [Processor] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4) == Kernel Testing == 1. linux-image-3.0.0-1-omap4_3.0.0-1.0~9765dd1_armel.deb (20110719) 1. linux-image-3.0.0-1-omap4_3.0.0-1.0~dd1d9db_armel.deb (20110720) == Ruby on Rails == 1. sudo apt-get install -y rubygems sqlite3 libsqlite3-ruby libsqlite3-dev 1. sudo gem1.8 install rails 1. sudo bundle install 1. rails new panda-test 1. cd panda-test ; rails server -d == vlan testing == 1. sudo modprobe 8021q 1. sudo apt-get install vlan 1. sudo vconfig add eth0 10 1. sudo ip addr add 10.0.0.10/24 dev eth0.10 1. ip addr # should see eth0 and eth0.10 1. Setup another machine on the same net with 10.0.0.22 for it's ip address 1. ping 10.0.0.22 == IPsec == This test requires two systems. Current test is from qa-regression-testing suite. 1. bzr clone lp:qa-regression-testing 1. sudo apt-get -y install python-unit ipsec-tools racoon 1. setup vlan on both systems per above instructions 1. Further instructions are contained in qa-regression-testing/scripts/test-ipsec-tools.py. == Entropy == This test requires the qa-regression-testing/scripts/test-rng.py and it's dependencies. The listed execution takes ~2 days to run. The full test takes several weeks. 1. bzr clone lp:qa-regression-testing 1. cd qa-regression-testing/scripts 1. sudo install-packages test-rng.py 1. sudo ./test-rng.py -t dieharder-kernel --report == Raid == This test requires 2 or more drives attached to the test system. To run this, follow the instructions for testing RAID1 [[http://testcases.qa.ubuntu.com/Install/ServerRAID1|here]]. Use the netboot install instead of the preinstalled images, as the preinstalled have no established mechanism to move to a different boot medium. == iSCSI Rootfs == This test requires a lot of manual setup and configuration. At a minimum, two systems are needed: target (host to filesystem), and the initiator (system sending iSCSI commands to target). Instructions can be gleened from http://testcases.qa.ubuntu.com/Install/ServeriSCSIRoot. == CEPH ClusterFS == This test setup requires a minimum of 3 platforms, more is better. 1 platform for the storage (OSD) & metadata (MDS), one system as the monitor (MON), and one client. More OSD & MDS systems are preferable and scale with each individual system added. Monitors need to be odd nombers according to the [[http://ceph.newdream.net/wiki/|documentation]]. For this test, 6 panda systems were used: 2 OSD, 2 MDS, 1 MON, and 1 client. Due to hardware availability, only 1 physical SATA drive was connected to each OSD & MDS, with the MON and client running from SD. No system errors were encountered. == LUKS == === Tests === * Basic LUKS functionality (with http://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/current/): * `cryptsetup luksFormat` on a separate partition. * `cryptsetup luksOpen` on this partition and appearance in `/dev/mapper/`. * `mkfs.ext4` on this LUKS partition and verification that the partition really is encrypted on a separate computer. * Add this partition to `/etc/fstab` and `/etc/crypttab` and check that the system correctly prompts for the passphrase and mounts the partition on boot. * Installer and root partition encryption (with http://ports.ubuntu.com/dists/oneiric/main/installer-armel/current/images/omap4/netboot/): * Run the installer with the guided encrypted LVM option * Check that the system correctly prompts for the passphrase, mounts the partition and the system still works on boot. === Test Results === Everything appears to work fine, except: 1. When using LUKS+LVM, the installer does not set a boot option of `console=ttyO2,115200` so the passphrase prompt on boot never appears. This presumably applies to non-system LUKS+crypttab+fstab on netinst as well. [[https://bugs.launchpad.net/ubuntu/+source/debian-installer/+bug/848782|LP: #848782]]. 2. When prompting for the passphrase outside the initrd, the passphrase is echoed back to the user in plaintext over the serial console. Other kernel messages also appear asynchronously over the top during passphrase entry. == LXC == * Run the following commands to install, create, and start lxc: {{{ sudo apt-get install lxc sudo lxc-create -n armhf-precise -t ubuntu -- -r precise sudo lxc-start -n armhf-precise }}} * Login as "ubuntu" with password "ubuntu" * Run the following to ensure basic functionality: {{{ sudo apt-get update sudo apt-get upgrade }}}