Newbie guide

What you need to know if you are new to the citrain landing team:

What do do

• read this document
• join #ubuntu-ci-eng and highlight on "trainguards"
• wait for new rows in the spreadsheet (queuebot will notify you) and perform the "steps for a landing" described below
• wait for stuff to be ready for publishing (the bot will tell you about it) and perform the "steps for publishing" below
• the expectation is to do up to 50% of your work time on landings

What is it

The citrain is a tool to coordinate landing of new features. This includes building/testing and publishing.

• Its all in one google spreadsheet https://wiki.ubuntu.com/citrain that mostly updates itself automatically (its a frontend to jenkins)

• First column in the spreadsheet is important! Keep attention to the number of free silos, try to keep 1 silo free for emergency landings
• You have a preproduction code silo in silo-000. You only have access to it if you check "use preproduction silo"
• The spreadsheet has multiple page (sheets), the main one is "pending", but there is also "Archive" that contains *all* of the history.
• Merge proposal to land (column F) is important, order of the branches is important. Branches are not limited to a single project, but merges within the same project must target the same branch (typically trunk)
• Additional source packages to land: landers will prepare a source package for you to dput into the silo PPA. Then, you run a build. If a build was already done, you can just run "WATCH_ONLY" to let the bot rescan the ppa content.
• QA sign off needed: only in effect if we are in TRAINCON-0 - and then *only* stuff with QA-sign-off: granted can enter the image

• Ready? : Once a request is marked as ready:Yes, assign to a silo via "landing team tools > assign silo" menu item (more see below)

Jenkins jobs parameters

• By default, CI-Train was designed so that all jobs needs no parameters for the default action. Those parameters are overrides for special cases and should only be used as such. The only "generally used" override is the ACK_PACKAGING when there are packaging changes.

Steps for a landing

• landing for a request:
  • check if it has a testplan
  • check if the package is locked already. CI-train will tell you if it's locked. There is an available override (but who lands first will force the second to rebuild to take latest content or its publication will fail).
  • check that its ready
  • go to the "Landing team tools" menubar and click on "assign silo"
    • this generates a request-id that is valid for 5min
    • opens a dialog that you need to check
    • the jenkins tab opens, click on "proceed"
    • opens a jenkins job page
    • check that the speadsheet updates after some time
    • queuebot will ping the lander once the silo is ready for building.

    • the details for the landing are in the silo dashboard, http://people.canonical.com/~platform/citrain_dashboard/#?q=

    • its the job of the lander to click on "BUILD" in the dashboard

  • once the thing is ready and the lander has executed the test plan he/she sets the request to Tested:Yes

  • once that is the case (queuebot or the lander will ping you), we go to the dashboard and click on "PUBLISH"
  • this brings you to jenkins
  • if there are changes in debian/* it generates a diff and that needs approval first via the "ACK_PACKAGING" box in the build jenkins subpage and press "build"
  • the status columns of the spreadsheet (and dashboard) will update automatically when the package migrate (like, it's a NEW package, it's in UNAPPROVED, -proposed, -release pocket). Be ready to help the landers with your expertise if something is stuck in -proposed for too long for instance, or ensure that packages in NEW are treated…

  • lander will click on "merge & clean" once the package is in the release pocket (queuebot will ping the landers).

Steps for publishing

• when a landing is built queuebot bot will say so in the #ubuntu-ci-eng channel
• check what landing number it is (e.g. landing-017) and go to the dashboard of that number

• WARNING always check that "TESTING PASS" field (column J) in the spreadsheet is set to YES

• click on "publish" on the dashboard, that links to a jenkins page

  • WARNING publish means to publish into the REAL archive - do not publish NEW packages without consulting

• click on "build"
• if there are packaging changes the circle on the left will turn yellow and you need to review the build artifact with the packaging diff (e.g. packaging_changes_nuntium_0.1+14.10.20140702.2-0ubuntu1.diff), then click on "Build with Parameters" on the left and select ACK_PACKAGING
• the status should be "blue" (successful), you can close the jenkins tab now
• now the lander needs to clean the silo, the bot will notify him/her

Steps for doing a sync landing

• sync landings are used for easily copying packages from one archive to another, like ubuntu -> ubuntu-rtm

• they can work in modes to copy from PPA's, archives or other landing silos
• if you want to sync packages from a silo to another silo:
  • example for syncing a ubuntu landing from a silo to an ubuntu-rtm silo
  • create a landing entry targeting ubuntu-rtm/14.09

  • in the 'Additional source packages to land' write 'sync:<silo_you_want_to_copy_from>', e.g. sync:4

  • leave 'Merges...' empty!
  • remember - if you create an ubuntu-rtm silo and write sync:4 you will copy from the ubuntu silo number 4
  • assign the silo for the landing line normally
  • now every time someone clicks 'Build' on the sync silo, it will try performing a binary copy of all packages in the source silo PPA
  • if the source silo gets freed, cleaned or removed, the sync:x silo will automatically retarget the source archive
• if you want to sync packages from a distro archive to a silo:
  • example for syncing a package from ubuntu to an ubuntu-rtm silo
  • create a landing entry targeting ubuntu-rtm/14.09

  • in the 'Additional source packages to land' write 'sync:<distro>,<series> <package1> <package2> <...>', e.g. sync:ubuntu,utopic unity8

  • leave 'Merges...' empty!
  • assign the silo for the landing line normally
  • now every time someone clicks 'Build' on the sync silo, it will try performing a binary copy of the selected packages from the source archive
• the rule is: whenever someone asks for any landing, we usually fill in a mirror sync landing for the same thing but targeting the second distribution

What about CI-Airline

• will replace all of CI-Train
  • no spreadsheet, but still manual details filing (so equivalent)
  • no jenkins

  • see http://blueprints.launchpad.net/ubuntu/+spec/core-1311-ci-airline

  • designed by didrocks, worked on by citeam

  • full spec and vision: https://docs.google.com/a/canonical.com/presentation/d/1LQK5Xll3D-G_zSFgFFDZiCmIZ8P33b_C7crefECsquA/edit#slide=id.p

• will be ready ~July 2014

Terms

• Silo are PPAs that are used to prepare a landing
• Landers are trusted people of the upstream project who can drive landing and do the final testing

• TrainCon-0: when no new image is generated for 7 business days this "emergency" state is reached and only critical fixes can go in

• CI Train Sheriff is the guy/gal currently doing landings

Links

• Main landing spreadsheet: https://wiki.ubuntu.com/citrain (refreshes every 5min)

• Jenkins that is controlled via the dashboard: http://people.canonical.com/~platform/citrain_dashboard/

• Backend code (running on jenkins): lp:cupstream2distro. The CI-train code is under the citrain/ directory. It uses the cupstream2distro library (initially designed for the Daily Release process).

• Testing results: http://ci.ubuntu.com/smokeng/utopic/touch/

• Low-level JSON with the silo status: http://people.canonical.com/~platform/citrain/

• Who is who: https://wiki.ubuntu.com/citrain/LandingTeam

• https://wiki.ubuntu.com/citrain/FAQ

• The Daily release FAQ is valid under citrain: https://wiki.ubuntu.com/DailyRelease/FAQ (apart from "When do changes need to land in a coherent piece?" and "I want one of my commit not being part of debian/changelog")