Testing

Differences between revisions 175 and 176
Revision 175 as of 2014-12-18 18:12:24
Size: 14971
Editor: wxl
Comment: wee 15.04 update
Revision 176 as of 2015-01-23 17:07:17
Size: 14984
Editor: saqman2060
Comment:
Deletions are marked like this. Additions are marked like this.
Line 107: Line 107:
Whenever you are testing, keep in mind few notes: Whenever you are testing, keep in mind these few notes:
Line 111: Line 111:
 1. Using a Development Release is not suitable for daily production machine.
 1. The more you break your installation, the better. That is why, to play it safe, better to use Virtual Machines, Spare Testing Machines and/or USB Drives, specially with Alpha 1 and Alpha 2. Beta Releases are a bit more stable but still under heavy development.
 1. Using a Development Release is not suitable for daily production machines.
 1. '''The more you break your installation, the better'''. That is why, to play it safe, better to use Virtual Machines, Spare Testing Machines and/or USB Drives, specially with Alpha 1 and Alpha 2. Beta Releases are a bit more stable but still under heavy development.

General information

The Testing Team is responsible for using formal test cases to check images of Lubuntu. This allows the Release Team to easily gauge the health of the images. This makes it possible to tell whether an image is functional enough to justify a release and determine where resources need to be allocated to make sure an image gets released.

Testing consists of:

  1. Testing milestone images

    • Our most important job is preparing for release. A few days before a milestone is due (according to our release schedule), the daily is plucked and becomes the QA (Quality Assurance) test version (these are called Alpha, Beta, and Final). Once it is confirmed that the QA version works, it then becomes the released version.

  2. Daily image testing

    • Daily images are constantly being tested to give us an early warning notice of any problems that could affect Milestone releases. This also gives us an opportunity to do more heavy testing to discover any bugs we hadn't noticed before.
  3. Specific testing

    • Specific testing allows us to help with development.

All release stages are tracked by ISO Tracker where you can get the latest builds, see and allocate any bugs.

For more information about testing, please see the Main QA team.

Updates

  • Ubuntu has dropped the amd64+mac image altogether. If you have an affected machine, just use the amd64 image. If you have troubles, please make sure you have the latest firmware update for your machine and then file a bug if there are still problems.
  • We did not release a ppc image for 14.10 due to various issues. We are going to focus on supporting the architecture for LTS releases. Thus, we need to focus testing in two areas:
    1. Because we plan to include LXQt in the future and we are not sure of Qt support for ppc, daily testing (on release+1) will be essential.
    2. We will encourage ppc users to use LTS versions, so heavy testing should be done on the LTS dailies.
  • There are blueprints for 15.04. Keep watching there for the official plan and work to do this cycle, which will dictate our focus. The real question is whether or not LXQt will be included or not (though it WILL come in the future).

  • There is a prototype of Lubuntu with LXQT (aka lubuntu-next) available. Info and instructions from the boss included.

  • LXQt can also be installed on an existing system by installing lxqt-metapackage from the lubuntu-daily PPA.

Testing Versions

  • Lubuntu 14.04 point releases are maintenance releases, do not expect too many changes.
  • Lubuntu 14.10 is now released.
  • Lubuntu 15.04 Alpha1 is now released. Daily testing and milestone testing will be needed here.

Getting Involved

  • Get a Launchpad account and join the Lubuntu QA team.

  • Join the main mailing lists so that you receive information about where Lubuntu is heading. Use the lubuntu-qa list for testing discussions.

  • On Freenode IRC, join #ubuntu-quality and #ubuntu-release. Note these are general channels shared across all flavors, so you may want to use #lubuntu-offtopic for specific questions about Lubuntu. #lubuntu is another option but we try to keep this specifically to support, so it's not recommended.
  • Start your first test!

Daily Builds

These iso's are automatically generated every 24 hours using the latest updates on the system from the devs. They are available from ISO tracker. They are there to check that bugs that are resolved between the Milestone releases do not break the install. They also are used to confirm that any fix for a bug that seriously affects an initial install which is released for testing now works. Daily builds are suspended when pre-milestone testing is being carried out (see below).

Milestone releases

Alpha, Beta and Final (formerly known as Release Candidates (RC)) are also tested using ISO tracker. If you want to help out in this important area of testing, please read through Procedures for further details. These appear a couple of days before the actual Milestone release so that we can check they are okay to become Milestone releases.

QA testing is to ensure the actual released install iso works, if you can, please get involved in the qa testing.

Once a Milestone release passes the QA testing, it becomes a Milestone Release and is listed on the Releases as such.

Note this also includes point releases on LTS versions. Basically, a point release brings in all the various bug fixes into one 'new' image. To get in to the build they have to pass SRU testing.

If you would to know more about how this all works, have a read of Stages of testing.

Specific Testing

QA tests

You can do more specific tests, like ones done for the Ubuntu ISO.

Laptop Testing

Laptops never cease to have their little 'quirks'. You can help on this important area by heading over to Laptop Testing for full details.

Applications test cases

You can also test specific programs :

Performance tests

You can test some aspects of Lubuntu performance with the following programs :

PPA Testing

Sometimes during the test cycle one of the developers may ask you to test something specific for them, or their team. For more details of this important area, head over to PPA testing

There are now packages for LXDE components built from upstream git, and re-built when a new revision is committed. It's hosted on the daily build PPA for Lubuntu-dev, packages for currently supported releases plus the development release are available. If you are working on a Lubuntu bug, please do test with these ppa areas to check if it has been fixed upstream.

Bugs against LXQt should be filed on the upstream git if you're sure it's an LXQt problem, otherwise file against lubuntu-next on Launchpad.

General Information

Before you get started

This section is dedicated to the current development version of Lubuntu. As with all alphas and betas they are not suitable for a production environment, please take the time to read Common Questions for Testing.

Whenever you are testing, keep in mind these few notes:

  1. Make sure to Backup your important data. If you are using Linux, the best and easier way is to make a copy of your /home folder or partition. If you want to do a full system backup, please see this link and this link too.

  2. You can use Virtual Machines - for example See this. You can use USB Drives or External HDD. You can use your machine. That is totally up to you but please, refer to #1 Smile :)

  3. Using a Development Release is not suitable for daily production machines.
  4. The more you break your installation, the better. That is why, to play it safe, better to use Virtual Machines, Spare Testing Machines and/or USB Drives, specially with Alpha 1 and Alpha 2. Beta Releases are a bit more stable but still under heavy development.

  5. The most important part of testing is to actually install the system and check how the installation process will work. This is very important. Also, if you have say Alpha 1 installed, it is less helpful to just upgrade it to Alpha 2 or Beta 1. Please, do a fresh new install - you can either zsync your ISO or download again.

  6. Finally, always remember: Better Safe Than Sorry Smile :)

  7. To save downloading the whole iso again for the testing version, simply copy your trusty image of what ever architecture replacing the old codename with the new one, e.g. 'trusty' with 'utopic' and use zsync.

Why does lubuntu have so many images?

As this question does get asked, there is a page here that explains it.

When images build

The Alternates take approximately 30 - 45 minutes to complete. The Desktops take approximately 90 minutes to complete.

The timing of the auto builds can be found here.

Debugging images

You can also see the status of images at https://launchpad.net/~ubuntu-cdimage/+livefs/ubuntu/RELEASENAME/lubuntu (change RELEASENAME to something like trusty, utopic, vivid, etc). Note if it says successfully built, that does not mean it's uploaded to the tracker yet.

You can also see the logs of automated builds. This is a good way to troubleshoot why an image is not there.

If you do notice that builds are not appearing as expected, please contact wxl and let him know or check with #ubuntu-release.

Rebuilding a Release Candidate

These are carried out manually, during this time the release team do update the notice panel. Please ask on #ubuntu-release if you have questions.

Bugs

NOTICE: always subscribe Lubuntu Packages Team to bugs encountered within Lubuntu.

Within bugs related to Lubuntu, you will see bugs raised by, or allocated to Julien Lavergne. Please feel free to add to the comments but do NOT alter the status of these bugs as they are being dealt with by our head of development in readiness for the fix being released.

Please head over to All about bugs for further information on how bug reporting works and why it is so important, as well as how to write a proper bug report.

An important part of QA is not only making bug reports, but making them in such a way that they can be triaged. Additionally, triage should be something that QA considers part of their workload. Bug reports are there to help developers know what to work on next. The only way development happens is if these bug reports are clear and clearly marked. That being said, look at the Bug Squad for info on how to write and triage reports. There is a subteam called Bug Control that you need to be approved for that can handle changing all statuses and priorities.

Partial Upgrades

During the release cycle, things will get broken. You can really reduce these occurrences by taking the time to read Partial Upgrades.

Known Issues

All the known issues for a particular release are mentioned in the Announcement email, and are available to see at ISO Tracker.

Manual test of ISO and CD

On the help-pages of Ubuntu there is an extensive guide on how to MD5SUM. In our case the terminal would be LXTerminal.

PPC

Except for Ubuntu Server, Lubuntu is the only flavor supporting Power PCs. Please do join to help Lubuntu have a release for that system.

Unwanted packages

Some packages can be automatically installed, but are not wanted on a default installation. To find the package which automatically installed the package that you don't want :

  • Install apt-rdepends
  • run "apt-rdepends -r --show=Depends the_unwanted_package" => It will show which packages depend on the_unwanted_package.

  • run "apt-rdepends -r --show=Recommends the_unwanted_package" => It will show which packages recommend the_package_unwanted (Recommended packages are installed by default).

  • You may have to run the commands several times to see the complete chain of depends / recommends.

Other tools

Jenkins runs automated tests. In particular, there's one that deals with Ubiquity that would be a good way to see if we're having problems with the Desktop ISO.

Lubuntu/Testing (last edited 2018-01-15 04:06:43 by tsimonq2)