Server

Arm Server QA Testing

Overview

This page captures the current high level ARM Server QA progress. The results below are checks to ensure that work-flows, applications, and benchmarks have been attempted and are ideally functional on ARM Server so that these fundamental tools can be utilized by others to validate server functionality and performance. After the initial sweep these tests are planned to be additions to the automated platform testing to ensure this functionality remains consistent across releases/updates. Please feel free to discuss and then add items as seen fit for typical server functionality and workloads; testing help would also be greatly appreciated to ensure quality and thoroughness.

Overall Status

Status Key

Color

Definition

White/Clear

Test/Benchmark/Application is pending/untested/unknown

Green

Test/Benchmark/Application has no glaring problems, installs/runs/passes

Yellow

Test/Benchmark/Application has minor problems to resolve, partial failures

Red

Test/Benchmark/Application has major problems, missing/fails/fails to install

Status Items

Area

Pri/Test

Test-Versions

Release/Kernel

Date

Brief Notes, Bug links, Problem Links

Application

P2 nginx

--

--

--

--

Application

P2 ruby

--

11.10/2.6.38-1309-omap4

20110722

none

Benchmark

P1 pts/apache

2.2.19-1ubuntu1

11.10/2.6.38-1309-omap4

20110720

--

Benchmark

P1 pts/java-scimark2

6b23~pre3-0ubuntu2

11.10/2.6.38-1309-omap4

20110718

pts/java-scimark2, PTS 3.2.0-1

Benchmark

P1 pts/openssl

--

11.10/2.6.38-1309-omap4

20110720

fails install, PTS 3.2.0-1

Benchmark

P1 pts/pgbench

--

11.10/2.6.38-1309-omap4

20110718

fails install, PTS 3.2.0-1

Benchmark

P1 pts/phpbench

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P1 pts/tiobench

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P1 pts/pybench

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P1 pts/pyopencl

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P1 pts/pyopencl

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P2 pts/aio-stress

--

11.10/2.6.38-1309-omap4

20110718

fails install, PTS 3.2.0-1

Benchmark

P2 pts/ddbench

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P2 pts/cachebench

--

--

--

--

Benchmark

P2 pts/fio

--

11.10/2.6.38-1309-omap4

20110718

fails install, PTS 3.2.0-1

Benchmark

P2 pts/fs-mark

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P2 pts/hdparm

--

--

--

--

Benchmark

P2 pts/iozone

--

11.10/2.6.38-1309-omap4

20110718

PTS 3.2.0-1

Benchmark

P3 pts/idle

--

--

--

--

Benchmark

P3 pts/idle-power-usage

--

--

--

--

Benchmark

P3 pts/postmark

--

--

--

--

Benchmark

P3 pts/stream

--

--

--

--

Benchmark

P4 pts/systester

--

11.10/2.6.38-1309-omap4

20110718

fails install, PTS 3.2.0-1

Benchmark

P4 pts/povray

--

--

--

--

Benchmark

P4 pts/ramspeed

--

--

--

--

Benchmark

P4 pts/scimark2

--

--

--

--

Benchmark

P4 pts/sqlite

--

--

--

--

Benchmark

P4 pts/stresscpu2

--

--

--

--

Boot

P1 u-boot pxe

--

11.10/2.6.38-1309-omap4

20110720

none

Network

P1 loopback

--

--

--

--

Network

P1 vlan

--

11.10/2.6.38-1309-omap4

20110722

none

Security

P1 IPsec

lp:qa-regression-testing

11.10/3.0.0-1204-omap4

20110905

Security

P1 Entropy

lp:qa-regression-testing

11.10/3.0.0-1200-omap4

20110824

Takes several days to run simple test. Weeks for full test.

Server

P1 Raid1

--

11.10/3.0.0-1203-omap4

20110824

Use netinstall to image system

Network

iSCSI

--

11.10/3.0.0-1204-omap4

20110905

Requires a bit of manual configuration. iSCSI-root fails to boot. See LP:838809

Server

CEPH ClusterFS

--

11.10/3.0.0-1204-omap4

20110925

Additional Testing Details and Notes

Installation

PXE Install

  • SD Card Method
    1. Format blank SD card as vfat (type LBA) and enable boot flag
    2. copy MLO from latest u-boot-linaro-omap4-panda deb package - Must be first file copied to SD
    3. copy u-boot.bin from latest u-boot-linaro-omap4-panda deb package
    4. create pxelinux.cfg file on your tftp server following standard pxeconfig guidelines. Note the kernel and initrd need to have the u-boot crc added. Use mkimage to generate them or use the netboot installer files.

Test Suites

Phoronix

  • Setup: "sudo apt-get install phoronix-test-suite"
  • Run: "sudo phoronix-test-suite run [test] [test] ..."

[Priority One - P1]:

  1. pts/apache [System/Apache] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  2. pts/java-scimark2 [Processor/Java] - Partial / tests 2,4,5 fail to run (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  3. pts/nginx [System/Nginx] - Pending
  4. pts/network-loopback [Network] - Pending
  5. pts/openssl [Processor] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  6. pts/pgbench [System/PostgreSQL] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  7. pts/phpbench [System/PHP] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  8. pts/tiobench [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  9. pts/pybench [System/Python] - Pending
  10. pts/pyopencl [System/Python] - Pending

[Priority Two - P2]:

  1. pts/aio-stress [Disk] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  2. pts/ddbench [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  3. pts/cachebench [Processor] - Pending
  4. pts/fio [Disk] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  5. pts/fs-mark [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)
  6. pts/hdparm-read [Disk] - Pending
  7. pts/iozone [Disk] - Completes (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)

[Priority Three - P3]:

  1. pts/idle [Disk] - Pending
  2. pts/idle-power-usage [Disk] - Pending
  3. pts/postmark [Disk] - Pending

[Other - Misc]:

  1. pts/povray [Processor] - Pending
  2. pts/ramspeed [Memory] - Pending
  3. pts/scimark2 [Processor] - Pending
  4. pts/sqlite [Disk] - Pending
  5. pts/stream [Memory] - Pending
  6. pts/stresscpu2 [Processor] - Pending
  7. pts/systester [Processor] - FAILS / fails install (pts=3.2.0-1, 11.10, 2.6.38-1309-omap4)

Kernel Testing

  1. linux-image-3.0.0-1-omap4_3.0.0-1.0~9765dd1_armel.deb (20110719)
  2. linux-image-3.0.0-1-omap4_3.0.0-1.0~dd1d9db_armel.deb (20110720)

Ruby on Rails

  1. sudo apt-get install -y rubygems sqlite3 libsqlite3-ruby libsqlite3-dev
  2. sudo gem1.8 install rails
  3. sudo bundle install
  4. rails new panda-test
  5. cd panda-test ; rails server -d

vlan testing

  1. sudo modprobe 8021q
  2. sudo apt-get install vlan
  3. sudo vconfig add eth0 10
  4. sudo ip addr add 10.0.0.10/24 dev eth0.10
  5. ip addr # should see eth0 and eth0.10
  6. Setup another machine on the same net with 10.0.0.22 for it's ip address
  7. ping 10.0.0.22

IPsec

This test requires two systems. Current test is from qa-regression-testing suite.

  1. bzr clone lp:qa-regression-testing
  2. sudo apt-get -y install python-unit ipsec-tools racoon
  3. setup vlan on both systems per above instructions
  4. Further instructions are contained in qa-regression-testing/scripts/test-ipsec-tools.py.

Entropy

This test requires the qa-regression-testing/scripts/test-rng.py and it's dependencies. The listed execution takes ~2 days to run. The full test takes several weeks.

  1. bzr clone lp:qa-regression-testing
  2. cd qa-regression-testing/scripts
  3. sudo install-packages test-rng.py
  4. sudo ./test-rng.py -t dieharder-kernel --report

Raid

This test requires 2 or more drives attached to the test system. To run this, follow the instructions for testing RAID1 here. Use the netboot install instead of the preinstalled images, as the preinstalled have no established mechanism to move to a different boot medium.

iSCSI Rootfs

This test requires a lot of manual setup and configuration. At a minimum, two systems are needed: target (host to filesystem), and the initiator (system sending iSCSI commands to target). Instructions can be gleened from http://testcases.qa.ubuntu.com/Install/ServeriSCSIRoot.

CEPH ClusterFS

This test setup requires a minimum of 3 platforms, more is better. 1 platform for the storage (OSD) & metadata (MDS), one system as the monitor (MON), and one client. More OSD & MDS systems are preferable and scale with each individual system added. Monitors need to be odd nombers according to the documentation. For this test, 6 panda systems were used: 2 OSD, 2 MDS, 1 MON, and 1 client. Due to hardware availability, only 1 physical SATA drive was connected to each OSD & MDS, with the MON and client running from SD. No system errors were encountered.

LUKS

Tests

  • Basic LUKS functionality (with http://cdimage.ubuntu.com/ubuntu-server/daily-preinstalled/current/):

    • cryptsetup luksFormat on a separate partition.

    • cryptsetup luksOpen on this partition and appearance in /dev/mapper/.

    • mkfs.ext4 on this LUKS partition and verification that the partition really is encrypted on a separate computer.

    • Add this partition to /etc/fstab and /etc/crypttab and check that the system correctly prompts for the passphrase and mounts the partition on boot.

  • Installer and root partition encryption (with http://ports.ubuntu.com/dists/oneiric/main/installer-armel/current/images/omap4/netboot/):

    • Run the installer with the guided encrypted LVM option
    • Check that the system correctly prompts for the passphrase, mounts the partition and the system still works on boot.

Test Results

Everything appears to work fine, except:

  1. When using LUKS+LVM, the installer does not set a boot option of console=ttyO2,115200 so the passphrase prompt on boot never appears. This presumably applies to non-system LUKS+crypttab+fstab on netinst as well. LP: #848782.

  2. When prompting for the passphrase outside the initrd, the passphrase is echoed back to the user in plaintext over the serial console. Other kernel messages also appear asynchronously over the top during passphrase entry.

LXC

  • Run the following commands to install, create, and start lxc:

    sudo apt-get install lxc
    sudo lxc-create -n armhf-precise -t ubuntu -- -r precise
    sudo lxc-start -n armhf-precise
  • Login as "ubuntu" with password "ubuntu"
  • Run the following to ensure basic functionality:

    sudo apt-get update
    sudo apt-get upgrade

ARM/QA/Server (last edited 2012-02-28 22:35:22 by mahmoh)