<> ||<>|| = CI / Testing = == Implemented == || '''Project''' || '''Kind of CI/testing''' || '''Status''' || '''Test results where(?)''' || || click-package-index || pre-merge testing || [[https://lists.launchpad.net/ubuntu-appstore-developers/msg00337.html|implemented]] || [[https://jenkins.ubuntone.info/job/click-package-index-merges/|code]] and [[https://jenkins.ubuntone.info/job/canonical-is-charms-click-package-index-merges/|charm]] || || click || run during builds || [[http://bazaar.launchpad.net/~click-hackers/click/trunk/files/head:/click/tests/|code]] || || || ubuntu-download-manager || run during builds + per-merge testing || [[http://bazaar.launchpad.net/~ubuntuone-hackers/ubuntu-download-manager/trunk/files/head:/ubuntu-download-manager-tests/|code]] || || || click-updown || pre-merge testing || implemented || [[https://jenkins.ubuntone.info/job/ubuntuone-servers-merges/|code]] and [[https://jenkins.ubuntone.info/job/ubuntuone-servers-merges/|charm]] || || software-center-agent || pre-merge testing || implemented || [[https://jenkins.ubuntone.info/job/software-center-agent-merges/|code]] || || || || || || == Planned == || '''Project''' || '''Kind of CI/testing''' || '''Status''' || || click-package-index || real-data testing run nightly || ? ([[https://lists.launchpad.net/ubuntu-appstore-developers/msg00337.html|planned]]) || || ubuntu-download-manager || integration tests || ? ([[https://lists.launchpad.net/ubuntu-appstore-developers/msg00338.html|planned]]) || || app confinement || integration tests || ? ([[https://lists.launchpad.net/ubuntu-appstore-developers/msg00324.html|planned]]) || || app scope || integration tests || ? [[https://lists.launchpad.net/ubuntu-appstore-developers/msg00429.html|planned]]) || == Blocked == || '''Project''' || '''Status''' || || SDK (!QtCreator) || [[http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/saucy/qtcreator/saucy/files/head:/tests/|Tests available]], but our infrastructure doesn't allow them to be run easily || || || || = Integration tests = || '''Step''' || '''Project(s)''' || '''Test number/status''' || '''More test number/status''' || || '''Dev uses ubuntu-sdk (qtcreator) to create a .click package''' || !QtCreator / Click ||<#FFFFE0> A || || || The .click is uploaded to the review website || myApps || ||<|2 #FFFFE0> B || || Our reviewers check the .click and approve it in the website || Click review tools || || || Dev marks the .click as published in the website || myApps || || || || '''unity-scope-click gets the list of .clicks from the click index webservice''' || unity-scope-click ||<#FFFFE0> C || || || dash shows those results || unity || || || || User taps on one app, unity-scope-click opens the app preview with details from webservice || unity-scope-click || || || || '''User taps on “Install”, ubuntu-download-manager starts downloading''' || unity-scope-click / ubuntu-download-manager || || || || Download progress is shown in a different dash preview that holds the progressbar || unity-scope-click || || || || '''Download finishes, ubuntu-download-manager calls “pkcon install-local”''' || ubuntu-download-manager / click ||<|3 #FFFFE0> D || || || '''pkcon uses packagekit dbus api to talk to click pk plugin''' || click / packagekit || || || '''click does the unpacking, calls hooks to create apparmor profile and .desktop files''' || click / apparmor || || || '''unity-scope-click creates list of installed packages from .desktop files''' || unity-scope-click || ||<#FFFFE0> E || || '''user taps on installed app, app is started with the right apparmor profile''' || unity-scope-click / apparmor / upstart-app-launch ||<#FFFFE0> F || || == Test descriptions == A. Keep a list of app projects, check all of them out and run [[https://code.launchpad.net/~ubuntu-sdk-team/qtcreator-plugin-ubuntu/trunk|lp:qtcreator-plugin-ubuntu]] scripts on them (whenever `qtcreator-plugin-ubuntu` changes), generate `.click` packages. Validate packages. * Importance: * Work items+estimate: * Determine initial list of projects (Core apps? Touch+Core apps?). * Write a broken app(?). * --(Make click review tools available on Launchpad.)-- * Determine which CI tools let us most easily branch code, run QtC tools, then run validation scripts on generated .click tools. * Write code to run the tests automatically. (Not very explicit work item, right?) * Hook up with CI infrastructure. * Open questions: * Which other qtcreator-plugin-ubuntu scripts do we want to test? * For which would we need QtC running? * For which an attached device? * Status: B. Keep a list of `.click` packages. Run validation scripts on them server side (whenever scripts or click change). * Importance: * Work items+estimate: * Status: C. ? Maybe some validation of the results from the store? * Importance: * Work items+estimate: * Test-run of current review scripts across current list of approved and rejected apps. * Reach out to app authors of approved apps to let them know of breakage, if we find any. * Set up a list of known-good and known-failing tests. * Write code to run the tests automatically. * Hook up with CI infrastructure. * Status: D. Keep a list of `.click` packages (whenever relevant bits - which? - change), check if everything gets installed in the right place and generated files make sense. * Importance: * Work items+estimate: * check that "click list --manifest" (as the phablet user) returns a manifest array that includes an entry for the package that was just installed, and that the "_directory" entry within that points to a directory that contains the package's contents. * Open questions: * Which paths does other code rely on? * Do we have validation code for any of the generated files? * Status: E. ? Check for duplicates? Check for missing icons, etc? * Importance: * Work ite,s+estimate: * Status: * Open questions: * Does autopilot help us here? * Can we mock navigation and verify data in the scope easily? F. ? * Importance: * Work items+estimate: * Status: