Testing

Differences between revisions 1 and 161 (spanning 160 versions)
Revision 1 as of 2010-08-18 20:48:51
Size: 4250
Editor: gilir
Comment:
Revision 161 as of 2014-08-28 18:10:14
Size: 11845
Editor: wxl
Comment: adding jenkins, cleaning things up
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
= Testing Lubuntu =
The alpha 3 for 10.10 is out, '''Pre-releases of Maverick are *not* encouraged for anyone needing a stable
system or anyone who is not comfortable running into occasional, even frequent breakage. They are, however, recommended for Ubuntu developers and those who want to help in testing, reporting, and fixing bugs.'''
Please ensure that you read [[https://wiki.ubuntu.com/Lubuntu/Developers/Maverick/Alpha/3| About 10.10 a3]] for where the project is up to and known issues.
Line 9: Line 5:
It is recommended that you join the [[https://wiki.ubuntu.com/Lubuntu/GettingInvolved|lubuntu mailing list]] so as to keep abreast of changes. As always, please try to use the torrent download & be a part of it. Torrent Download shares the work between several computers and should be much faster. ## remove the turn off area once the new build happens
## = This area is currently turned off - Will be re activated come 14.10 =
## = Testing Lubuntu =
Line 11: Line 9:
The Server Download depends on one computer, if that is busy, the download speed will slow; if it gets too many requests it will refuse them (think 200 people, one computer... it just cannot keep up).
Line 13: Line 10:
[[http://people.ubuntu.com/~gilir/lubuntu-maverick-alpha3.iso.torrent|Torrent Download]] == Before Getting Started ==
This section is dedicated to the current development version of Lubuntu. As with all alphas and betas they are not suitable for a production environment, please take the time to read [[http://ubuntuforums.org/showthread.php?t=1594833 | Common Questions for Testing]].
Line 15: Line 13:
[[http://people.ubuntu.com/~gilir/lubuntu-maverick-alpha3.iso|Server Download]]
Line 17: Line 14:
[[http://phillw.net/lubuntu-maverick-alpha3.iso|Secondary Server Download]]
Line 19: Line 15:
[[http://people.ubuntu.com/~gilir/md5sum.txt|md5CheckSums]] You should not need this, as the md5checksum and checking is detailed below.
== Minimal Installation and 64 Bit ==
By using [[http://archive.ubuntu.com/ubuntu/dists/maverick/main/installer-i386/current/images/netboot/|32 Bit minimal]] or
[[http://archive.ubuntu.com/ubuntu/dists/maverick/main/installer-amd64/current/images/netboot/|64 Bit minimal]]
You can add the '''lubuntu-desktop''' package, or the '''lubuntu-core''' package. [[https://wiki.ubuntu.com/Lubuntu/DocumentationHelp/MinimalInstall|Minimal Install Instructions]] is where to get further deatils, along with a work-around for the "Unmanaged connection" bug.
Whenever you are testing, keep in mind few notes:

 1. Make sure to [[https://help.ubuntu.com/community/BackupYourSystem | Backup]] your '''important data'''. If you are using Linux, the best and easier way is to make a copy of your '''/home''' folder or partition. If you want to do a full system backup, please see [[https://help.ubuntu.com/community/BackupYourSystem | this link]] and [[http://www.howtogeek.com/108869/how-to-back-up-ubuntu-the-easy-way-with-dj-dup/ | this link too]].
 1. You can use [[http://en.wikipedia.org/wiki/Virtual_machine | Virtual Machines]] - for example [[https://help.ubuntu.com/community/VirtualMachines | See this]]. You can use USB Drives or External HDD. You can use your machine. That is totally up to you but please, refer to #1 :)
 1. Using a Development Release is not suitable for daily production machine.
 1. The more you break your installation, the better. That is why, to play it safe, better to use Virtual Machines, Spare Testing Machines and/or USB Drives, specially with Alpha 1 and Alpha 2. Beta Releases are a bit more stable but still under heavy development.
 1. '''The most important part of testing is to actually install the system and check how the installation process will work'''. This is very important. Also, if you have say Alpha 1 installed, it is less helpful to just upgrade it to Alpha 2 or Beta 1. Please, do a fresh new install - you can either [[https://help.ubuntu.com/community/ZsyncCdImage | zsync]] your ISO or download again.
 1. Finally, always remember: '''Better Safe Than Sorry''' :)

== Start Testing ==
 * '''Lubuntu 14.04 point releases are maintenance releases, do not expect too many changes'''

 * '''Lubuntu 14.10 is a maintenance release, do not expect too many changes'''

[[https://wiki.ubuntu.com/UtopicUnicorn/Beta1/Lubuntu | Beta1]] is now released, sadly, there are no PPC versions due to lack of testing. The Dailies will now be turned back on.



== Use old ISOs ==

To save downloading the whole iso again for the testing version, simply copy your trusty image of what ever architecture replacing the old codename with the new one, e.g. 'trusty' with 'utopic' and use zsync. (You can, of course, simply do a '''mv''', but I like to keep my older iso's handy).

== 14.04 point releases ==

Basically, a point release brings in all the various bug fixes into one 'new' image. To get in to the build they have to pass [[https://wiki.ubuntu.com/StableReleaseUpdates | SRU testing]].

== What will 14.10 bring? ==

Our head of development has stated what is planned for 14.10 and beyond [[https://wiki.ubuntu.com/Lubuntu/Testing/14.10 |here]].


== Why does lubuntu have so many options? ==

As this question does get asked, there is a [[https://wiki.ubuntu.com/Lubuntu/Testing/Why_So_Many_ISOs | page here]] that explains it.

== Getting Involved ==

 * Please do [[https://wiki.ubuntu.com/Lubuntu/ContactUs#Contact_Us | Join the main mailing lists]] so that you receive information about where lubuntu is heading.

 * '''For Testing Questions''', we ask that you join the [[https://launchpad.net/~lubuntu-qa | lubuntu QA team]] so you can discuss bugs and proposed resolutions.

 * Always ask if you are in doubt :)
 

== Bugs ==

Within bugs related to lubuntu, you will see bugs raised by, or allocated to '''Julien Lavergne'''. Please feel free to add to the comments but do '''NOT''' alter the status of these bugs as they are being dealt with by our head of development in readiness for the fix being released.

Please head over to '''[[https://wiki.ubuntu.com/QATeam/Overview/#Bugs | All about bugs]]''' for further information on how bug reporting works and why it is so important.


== Testing ==
Testing is split into distinct, but joined areas. The '''Daily Builds''', the '''QA-testing of Milestone releases''' and the '''Milestone releases''' themselves.

To not get overly complicated, think of it as that we have a [[https://wiki.ubuntu.com/UtopicUnicorn/ReleaseSchedule | schedule]] to keep to. A few days before a Milestone is due, the daily is plucked and becomes the QA (Quality Assurance) test version for the Milestone release. Once it is confirmed that the QA version works, it then becomes the [[https://wiki.ubuntu.com/Lubuntu/Testing#Milestone_Releases | Milestone]].

All release stages are tracked by [[http://iso.qa.ubuntu.com/ | ISO Tracker]] where you can get the latest builds, see and allocate any [[https://wiki.ubuntu.com/Lubuntu/ReportingBugs | Bugs]].

So, in order of how they happen.

=== Daily Builds ===

These iso's are automatically generated every 24 hours using the latest updates on the system from the devs. They are available from [[http://iso.qa.ubuntu.com/|ISO tracker]]. Using the [[https://help.ubuntu.com/community/ZsyncCdImage | zsync (or rsync)]] option allows you to update your [[https://help.ubuntu.com/community/BurningIsoHowto |iso]] to any of the various dailies you choose to follow without having to re-download the entire iso. They are there to check that bugs that are resolved between the Milestone releases do not break the install. They also are used to confirm that any fix for a bug that seriously affects an initial install which is released for testing now works.

 * Daily builds are suspended when pre-milestone testing is being carried out (see below).

=== When they build ===

The Alternates take approximately 30 - 45 minutes to complete.
The Desktops take approximately 90 minutes to complete.

==== 14.04 ====

Timed to start at 15:29 UTC

==== 14.10 ====

Timed to start at 16:29 UTC

Please note from the release team: ''Sure, as long as it's clear that it's subject to change - We're not intending to make any promises here. We won't change them around frivolously or anything but it's possible.''

The timing of the auto build of lubuntu can be found [[http://bazaar.launchpad.net/~ubuntu-cdimage/ubuntu-cdimage/mainline/view/head:/etc/crontab | here ]]. If you do notice that builds are not appearing as expected, please contact [[wxl]] to let him know.

=== QA testing of Milestone releases ===

Alpha, Beta and Release Candidates (RC) are also tested using [[http://iso.qa.ubuntu.com/|ISO tracker]]. If you want to help out in this important area of testing, please read through [[https://wiki.ubuntu.com/Testing/ISO/Procedures | Procedures]] for further details. These appear a couple of days before the actual Milestone release so that we can check they are okay to become Milestone releases.

'''QA testing is to ensure the actual install iso works, if you can, please get involved in the qa testing'''

==== Rebuilding a Release Candidate ====

These are carried out manually, during this time the release team do update the [[http://iso.qa.ubuntu.com/qatracker | notice panel]]. Please ask on #ubuntu-release if you have questions.

=== Milestone Releases ===

Once a Milestone release passes the QA testing, it becomes a Milestone Release and is listed on the [[http://cdimage.ubuntu.com/lubuntu/releases/ | Releases]] as such.

If you would to know more about how this all works, have a read of [[https://wiki.ubuntu.com/QATeam/Overview/TheStages | Stages of testing]].

Line 25: Line 116:
During the release cycle, things will get broken. You can really reduce these occurrences by taking the time to read [[http://ubuntuforums.org/showthread.php?t=1479146|Partial Upgrades]]. The discussions at [[http://ubuntuforums.org/forumdisplay.php?f=385|Maverick Meerkat Testing]] are also a great resource. During the release cycle, things will get broken. You can really reduce these occurrences by taking the time to read [[http://ubuntuforums.org/showthread.php?t=1479146|Partial Upgrades]].
Line 27: Line 118:
== Check downloaded image file and CD ==
 1. Get an LXTerminal session started (Accessories --> LXTerminal).
 2. Go into the directory with the '''lubuntu-maverick-alpha3.iso''' file in it. If you have it on your desktop, then it would be {{{
cd Desktop
}}}
 3. To check you are in the correct directory {{{
ls lubuntu-maverick-alpha3.iso
}}} will return {{{
lubuntu-maverick-alpha3.iso
}}}
 4. Once you have confirmed you are in the correct directory {{{
md5sum lubuntu-maverick-alpha3.iso
}}}
It should report back '''aaa7f31fd712cd7376859594849d4806'''. If it does not, the download is corrupt and you need to re-download.
 5. Burn the cd (help on this can be found at [[https://help.ubuntu.com/community/BurningIsoHowto|Burning an ISO]], there is a screen cast for using xfburn at [[http://lubuntu.net/node/38 | Burning CDs/DVDs (using XFBurn)]]). Always burn at a slow speed (4X, or slower).
 6. Check the CD {{{
dd if=/dev/cdrom bs=2048 count=282225 | md5sum
}}}
=== Known Issues ===
All the known issues for a particular release are mentioned in the Announcement email, and are available to see at [[http://iso.qa.ubuntu.com/ | ISO Tracker]].
Line 46: Line 121:
It should also report back '''aaa7f31fd712cd7376859594849d4806'''. If not, the CD did not burn correctly. === Manual test of ISO and CD ===
On the help-pages of Ubuntu there is an extensive guide on [[https://help.ubuntu.com/community/HowToMD5SUM|how to MD5SUM]]. In our case the terminal would be [[https://wiki.ubuntu.com/Lubuntu/DocumentationHelp/LXTerminal| LXTerminal]].
Line 48: Line 124:
== Test cases ==
There is some test cases for people who want to test if there is no regression in each release (stable or developement)
 * [[http://testcases.qa.ubuntu.com/Applications/Pcmanfm|Pcmanfm]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXPanel|Lxpanel]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXSession|Lxsession]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXTerminal|Lxterminal]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXAppearance|Lxappearance]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXTasks|Lxtasks]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXInput|Lxinput]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXRandr|Lxrandr]]
=== PPC and Intel Macs ===

There is now an area for [[https://wiki.ubuntu.com/Lubuntu/Testing/PPC%26Mac64 | Power PCs and Intel Macs]]. Please do join to help lubuntu have a release for that system.

 
== Specific Testing ==

=== QA tests ===
You can do more specific tests, like ones done for the Ubuntu ISO : http://iso.qa.ubuntu.com/qatracker/test/5090

=== Laptop Testing ===

Laptops never cease to have their little 'quirks'. You can help on this important area by heading over to [[https://wiki.ubuntu.com/Testing/Laptop/Procedures | Laptop Testing]] for full details.

=== Applications test cases ===
You can also test specific programs :
 * [[http://testcases.qa.ubuntu.com/Applications/Pcmanfm | PCmanFM]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXPanel | LXPanel]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXSession | LXSession]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXTerminal | LXTerminal]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXAppearance | LXAppearance]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXTasks | LXTask]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXInput | LXInput]]
 * [[http://testcases.qa.ubuntu.com/Applications/LXRandr | LXRandR]]

=== Performance tests ===
You can test some aspects of Lubuntu performance with the following programs :
 * [[http://gtkperf.sourceforge.net/ | gtkperf]] : Test the performance of the gtk theme.
 * [[http://www.phoronix-test-suite.com/ | phoronix-test-suite ]]: General benchmarks and test suite.
 * [[https://wiki.ubuntu.com/BootCharting | bootchart]] : Test boot process.

=== Unwanted packages ===
Some packages can be automatically installed, but are not wanted on a default installation. To find the package which automatically installed the package that you don't want :
 * Install apt-rdepends
 * run "apt-rdepends -r --show=Depends the_unwanted_package" => It will show which packages depend on the_unwanted_package.
 * run "apt-rdepends -r --show=Recommends the_unwanted_package" => It will show which packages recommend the_package_unwanted (Recommended packages are installed by default).
 * You may have to run the commands several times to see the complete chain of depends / recommends.

=== PPA Testing ===
Sometimes during the test cycle one of the developers may ask you to test something specific for them, or their team. For more details of this important area, head over to [[https://wiki.ubuntu.com/Lubuntu/Testing/PPA_Testing | PPA testing ]]

=== LXDE PPA ===

There are now packages for LXDE components built from upstream git, and re-built when a new revision is committed. It's hosted on the daily build PPA for [[https://launchpad.net/~lubuntu-dev/+archive/lubuntu-daily | Lubuntu-dev]], packages for currently supported releases plus the development release are available. If you are working on a lubuntu bug, please do test with these ppa areas to check if it has been fixed upstream.

The use of this ppa is as per the above section.

= Other tools =

Jenkins runs automated tests. In particular, there's [[https://jenkins.qa.ubuntu.com/view/Ubiquity/view/Lubuntu/|one that deals with Ubiquity]] that would be a good way to see if we're having problems with the Desktop ISO.

Before Getting Started

This section is dedicated to the current development version of Lubuntu. As with all alphas and betas they are not suitable for a production environment, please take the time to read Common Questions for Testing.

Whenever you are testing, keep in mind few notes:

  1. Make sure to Backup your important data. If you are using Linux, the best and easier way is to make a copy of your /home folder or partition. If you want to do a full system backup, please see this link and this link too.

  2. You can use Virtual Machines - for example See this. You can use USB Drives or External HDD. You can use your machine. That is totally up to you but please, refer to #1 Smile :)

  3. Using a Development Release is not suitable for daily production machine.
  4. The more you break your installation, the better. That is why, to play it safe, better to use Virtual Machines, Spare Testing Machines and/or USB Drives, specially with Alpha 1 and Alpha 2. Beta Releases are a bit more stable but still under heavy development.
  5. The most important part of testing is to actually install the system and check how the installation process will work. This is very important. Also, if you have say Alpha 1 installed, it is less helpful to just upgrade it to Alpha 2 or Beta 1. Please, do a fresh new install - you can either zsync your ISO or download again.

  6. Finally, always remember: Better Safe Than Sorry Smile :)

Start Testing

  • Lubuntu 14.04 point releases are maintenance releases, do not expect too many changes

  • Lubuntu 14.10 is a maintenance release, do not expect too many changes

Beta1 is now released, sadly, there are no PPC versions due to lack of testing. The Dailies will now be turned back on.

Use old ISOs

To save downloading the whole iso again for the testing version, simply copy your trusty image of what ever architecture replacing the old codename with the new one, e.g. 'trusty' with 'utopic' and use zsync. (You can, of course, simply do a mv, but I like to keep my older iso's handy).

14.04 point releases

Basically, a point release brings in all the various bug fixes into one 'new' image. To get in to the build they have to pass SRU testing.

What will 14.10 bring?

Our head of development has stated what is planned for 14.10 and beyond here.

Why does lubuntu have so many options?

As this question does get asked, there is a page here that explains it.

Getting Involved

  • Please do Join the main mailing lists so that you receive information about where lubuntu is heading.

  • For Testing Questions, we ask that you join the lubuntu QA team so you can discuss bugs and proposed resolutions.

  • Always ask if you are in doubt Smile :)

Bugs

Within bugs related to lubuntu, you will see bugs raised by, or allocated to Julien Lavergne. Please feel free to add to the comments but do NOT alter the status of these bugs as they are being dealt with by our head of development in readiness for the fix being released.

Please head over to All about bugs for further information on how bug reporting works and why it is so important.

Testing

Testing is split into distinct, but joined areas. The Daily Builds, the QA-testing of Milestone releases and the Milestone releases themselves.

To not get overly complicated, think of it as that we have a schedule to keep to. A few days before a Milestone is due, the daily is plucked and becomes the QA (Quality Assurance) test version for the Milestone release. Once it is confirmed that the QA version works, it then becomes the Milestone.

All release stages are tracked by ISO Tracker where you can get the latest builds, see and allocate any Bugs.

So, in order of how they happen.

Daily Builds

These iso's are automatically generated every 24 hours using the latest updates on the system from the devs. They are available from ISO tracker. Using the zsync (or rsync) option allows you to update your iso to any of the various dailies you choose to follow without having to re-download the entire iso. They are there to check that bugs that are resolved between the Milestone releases do not break the install. They also are used to confirm that any fix for a bug that seriously affects an initial install which is released for testing now works.

  • Daily builds are suspended when pre-milestone testing is being carried out (see below).

When they build

The Alternates take approximately 30 - 45 minutes to complete. The Desktops take approximately 90 minutes to complete.

14.04

Timed to start at 15:29 UTC

14.10

Timed to start at 16:29 UTC

Please note from the release team: Sure, as long as it's clear that it's subject to change - We're not intending to make any promises here. We won't change them around frivolously or anything but it's possible.

The timing of the auto build of lubuntu can be found here. If you do notice that builds are not appearing as expected, please contact wxl to let him know.

QA testing of Milestone releases

Alpha, Beta and Release Candidates (RC) are also tested using ISO tracker. If you want to help out in this important area of testing, please read through Procedures for further details. These appear a couple of days before the actual Milestone release so that we can check they are okay to become Milestone releases.

QA testing is to ensure the actual install iso works, if you can, please get involved in the qa testing

Rebuilding a Release Candidate

These are carried out manually, during this time the release team do update the notice panel. Please ask on #ubuntu-release if you have questions.

Milestone Releases

Once a Milestone release passes the QA testing, it becomes a Milestone Release and is listed on the Releases as such.

If you would to know more about how this all works, have a read of Stages of testing.

General Testing

During the release cycle, things will get broken. You can really reduce these occurrences by taking the time to read Partial Upgrades.

Known Issues

All the known issues for a particular release are mentioned in the Announcement email, and are available to see at ISO Tracker.

Manual test of ISO and CD

On the help-pages of Ubuntu there is an extensive guide on how to MD5SUM. In our case the terminal would be LXTerminal.

PPC and Intel Macs

There is now an area for Power PCs and Intel Macs. Please do join to help lubuntu have a release for that system.

Specific Testing

QA tests

You can do more specific tests, like ones done for the Ubuntu ISO : http://iso.qa.ubuntu.com/qatracker/test/5090

Laptop Testing

Laptops never cease to have their little 'quirks'. You can help on this important area by heading over to Laptop Testing for full details.

Applications test cases

You can also test specific programs :

Performance tests

You can test some aspects of Lubuntu performance with the following programs :

Unwanted packages

Some packages can be automatically installed, but are not wanted on a default installation. To find the package which automatically installed the package that you don't want :

  • Install apt-rdepends
  • run "apt-rdepends -r --show=Depends the_unwanted_package" => It will show which packages depend on the_unwanted_package.

  • run "apt-rdepends -r --show=Recommends the_unwanted_package" => It will show which packages recommend the_package_unwanted (Recommended packages are installed by default).

  • You may have to run the commands several times to see the complete chain of depends / recommends.

PPA Testing

Sometimes during the test cycle one of the developers may ask you to test something specific for them, or their team. For more details of this important area, head over to PPA testing

LXDE PPA

There are now packages for LXDE components built from upstream git, and re-built when a new revision is committed. It's hosted on the daily build PPA for Lubuntu-dev, packages for currently supported releases plus the development release are available. If you are working on a lubuntu bug, please do test with these ppa areas to check if it has been fixed upstream.

The use of this ppa is as per the above section.

Other tools

Jenkins runs automated tests. In particular, there's one that deals with Ubiquity that would be a good way to see if we're having problems with the Desktop ISO.

Lubuntu/Testing (last edited 2018-01-15 04:06:43 by tsimonq2)