Tests that should be systematically done by Ubuntu Developers before uploading NetworkManager

Automated testing

• There are automated unit tests run during the build process of the network-manager package; should any of those tests fail, the package build will not succeed.

• Automated autopkgtest tests are being run when a new version is uploaded to Launchpad. See NetworkManager AutoPkg Test. These tests may fail for a number of reasons.

Manual testing

This is essentially the same process as done by the NetworkManager developers upstream before doing a release; see the GNOME wiki on NetworkManager/SmokeTesting.

• Above all, NetworkManager should be run locally for a minimum of one day; and if possible one full week before uploading. (Can be less than a week if the changes are minimal)

Basic smoke tests

• Ensure connection to the following types of WiFi access points using a common wifi driver (like ath9k or iwlwifi)

  • Open
  • WEP
  • WPA-PSK
  • WPA-Enterprise with PEAP/MSCHAPv2 authentication (when available; should be tested regularly)
  • WPA-Enterprise with EAP-TLS certificate based authentication (when available; should be tested regularly)
  • WPA-Enterprise with EAP-TLS certificate based authentication (when available; should be tested regularly)
• Ensure that an "always ask" WPA Enterprise connection asks for the password each time
• Ensure that a DHCPv4 based wired connection works
• Ensure that a static IP wired connection works
• Ensure that the following work for an "automatic" wired IPv6-only connection (tested regularly):
  • Where the RA does not include M or O options, NM obtains a global IPv6 address and a default route
  • Same as above plus RDNSS (and DNSSL if you have kernel 3.5 or later)
  • Where the RA includes the O (otherconf) option, NM runs DHCPv6 and correctly handles the returned options
  • Where the RA includes the M (managed) option, NM runs DHCPv6 and correctly handles the returned options
• Ensure that an "automatic" wired IPv4 + IPv6 dual stack connection works (tested regularly):
  • IPv4 is DHCPv4 based
  • IPv6 is RA based
• Ensure that at least one VPN plugin can successfully connect to the VPN server
• Ensure that when no network connections are defined, the wired interface receives a default "Wired connection 1" connection that successfully activates with DHCP
• Suspend / resume: Suspend the machine and ensure the state of the connections is restored on resume from suspend.

PPP Smoketesting

• Ensure that the mobile devices in the test devices list can connect successfully and pass traffic. (cyphermox)

• Ensure that killing pppd with a SIGTERM causes the NM device to enter the FAILED state

Should add soon:

• Testing that Bluetooth DUN connections can be established.

Advanced WiFi Smoketesting

• (if possible) Ensure WPA-Enterprise connections survive multiple roaming attempts between APs in the same SSID

• Test WEP, WPA, and open connections on a variety of drivers (ath5k, ath9k, iwlagn, rt2800, b43, bcma, orinoco, p54). See TestDeviceList for the devices available for regular testing. More drivers are tested in Canonical Certification labs when possible.

• Ensure that connecting to a new "hidden" SSID network works correctly when using the "Connect to hidden network..." applet option
• Ensure that if no CA certificate path is given that the security warning dialog shows and warns about missing CA certificate

Should be added (missing permanent infrastructure):

• Test other 802.1x authentication methods like TLS, TTLS, PAP, CHAP, GTC, etc

Advanced Wired Smoketesting

• Ensure connection fallback from a DHCP connection to a different static IP connection works when the network does not support DHCP

To be added (missing permanent infrastructure):

• Ensure a wired 802.1x connection works correctly using the MD5

Bluetooth Smoketesting

• Ensure that pairing a PAN capable phone works and creates a new NM connection
• Ensure that the new PAN connection connects and passes traffic
• Ensure that moving the phone out of range causes the NM device to enter the FAILED state for both DUN and PAN

• Ensure that deleting a paired device from the bluetooth applet removes the phone's connection from NetworkManager too

To be added:

• Ensure that pairing a DUN capable phone works and creates a new NM connection (cyphermox doesn't have a DUN capable phone)
• Ensure that the new DUN connection connects and passes traffic

VPN Testing

These tests ensure functioning VPN support. These tests are valid on any device with a WiFi or Ethernet adapter.

• Using system-settings, configure an OpenVPN connection, specifying:

  • Server/[Port] - the hostname ( or IP address ) and port ( optional ) of the OpenVPN service ( eg. us.openvpn.mycompany.com )

  • Use this VPN for - Its own network ( don't change this unless specifically instructed for a specific VPN )

  • Type - OpenVPN ( currently the only type supported )

  • Protocol - UDP ( again unless instructed otherwise )

  • Authentication Type - Certificates (TLS)

  • Client Certificate - as supplied by the VPN provider

  • Private Key - as supplied by the VPN provider

  • Key Password (optional) - as supplied by the VPN provider

  • CA Certificate - as supplied by the VPN provider

  • Use additional TLS authentication - checked ( unless instructed otherwise )

  • TLS Key - as supplied by the VPN provider

  • Key direction - as supplied by the VPN provider

  • Verify peer certificate - checked ( unless instructed otherwise )

  • Peer certificate TLS type - Server

  • Cipher - as supplied by the VPN provider

  • Compress Data - checked ( unless instructed otherwise ).

    • Note - this might be referred to as LZO compression.

• Activate the VPN; the output of nmcli should look like this:

        DEVICE   TYPE      STATE                 CONNECTION                
        tun0     tun       connected             tun0                      
        wlan0    wifi      connected             hyperion                  

• At this point, you should verify that the internet is accessible, and that resources on the internal VPN network(s) are reachable and/or resolvable ( ie. pinging a hostname at least resolves to an IP address.

• Disable VPN; confirm that internal resources can no longer be accessed, and that the internet is still accessible.

CategoryNetworking