ContributingDeveloperApplication

Differences between revisions 4 and 5
⇤ ← Revision 4 as of 2024-12-20 16:56:29 →
Size: 12283
Editor: hyask
Comment:
← Revision 5 as of 2024-12-31 14:29:01 → ⇥
Size: 13067
Editor: hyask
Comment:
Deletions are marked like this. Additions are marked like this.
Line 24: Line 24:
Most of my work revolves around automated testing infrastructure. Most of my work revolves around automated testing infrastructure (autopkgtest), but also keeping alive a bunch of services like the Error Tracker, Security Britney, Merge O Matic, etc...
Line 32: Line 32:
This is currently the biggest part of my work. This infrastructure is very much central to the Ubuntu development, and keeping that running without hiccup is top priority. The [[https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud|list of contributions]] made to that infrastructure is a bit long, but here are some highlights: This is currently one of the biggest part of my work. This infrastructure is very much central to the Ubuntu development, and keeping that running without hiccup is top priority. The [[https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud|list of contributions]] made to that infrastructure is a bit long, but here are some highlights:
Line 40: Line 40:
  * https:%%//%%code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/456376   * https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/456376
Line 44: Line 44:
This was a welcomed change that added the display of running and queued tests on top of the existing test results for a given package. Further details can be found on the ''%%ubuntu-devel%%'' mailing list thread. This was a welcomed change that added the display of running and queued tests on top of the existing test results for a given package. Further details can be found on the ''ubuntu-devel'' mailing list thread.
Line 50: Line 50:

==== Port everything to use the Python ''amqp'' module, and run on the latest Ubuntu ====

''amqplib'' has been deprecated for a long time, and was finally removed from the archive. Moreover, a few old-ish patterns made the code unable to work correctly on modern Python and Ubuntu.

  * [[https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/473908|Merge into master : skia/switch_worker_to_amqp : lp:~hyask/autopkgtest-cloud : Git : Code : autopkgtest-cloud]] - Worker part
  * [[https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/476752|Merge into master : skia/switch_web_to_amqp : lp:~hyask/autopkgtest-cloud : Git : Code : autopkgtest-cloud]] - Web part
Line 57: Line 64:
Added a sensible ''%%vimrc%%'' to the production machines to reduce the risk of silly mistakes. Added a sensible ''vimrc'' to the production machines to reduce the risk of silly mistakes.
Line 59: Line 66:
  * https:%%//%%code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/463822   * https://code.launchpad.net/~hyask/autopkgtest-cloud/+git/autopkgtest-cloud/+merge/463822
Line 61: Line 68:
Refreshed the cloud-init userdata file we use to deploy ''%%armhf%%'' LXD remotes, and deployed new remotes in the new ''%%bos03%%'' datacenter. Refreshed the cloud-init userdata file we use to deploy ''armhf'' LXD remotes, and deployed new remotes in the new ''bos03'' datacenter.
Line 74: Line 81:
Since this is the core tool around which our infrastructure is built, I’ve also had the opportunity to contribute to ''%%autopkgtest%%''. We have an ongoing effort to upstream the Ubuntu patches, so I help there by providing feedback, tests, and some code review. Since this is the core tool around which our infrastructure is built, I’ve also had the opportunity to contribute to ''autopkgtest''. We have an ongoing effort to upstream the Ubuntu patches, so I help there by providing feedback, tests, and some code review.
Line 76: Line 83:
  * https:%%//%%salsa.debian.org/ci-team/autopkgtest/-/commits/master?author=Skia   * https://salsa.debian.org/ci-team/autopkgtest/-/commits/master?author=Skia
Line 88: Line 95:
The crash reporting script is responsible for checking on the crashes reported by apport, making sure they get pushed to the error tracker, and setting the test status according to what was found. I’ve rewritten the whole script from a tiny piece of ''%%bash%%'' to a more readable piece of ''%%python%%'', adding along the way much improved logging, and support for an “ignore list”. The ignore list allows us to set back to green tests that have a crash that is already reported and known, so that we more easily catch new issues happening while the first crash is being worked upon. The crash reporting script is responsible for checking on the crashes reported by apport, making sure they get pushed to the error tracker, and setting the test status according to what was found. I’ve rewritten the whole script from a tiny piece of ''bash'' to a more readable piece of ''python'', adding along the way much improved logging, and support for an “ignore list”. The ignore list allows us to set back to green tests that have a crash that is already reported and known, so that we more easily catch new issues happening while the first crash is being worked upon.
Line 94: Line 101:
Until recently, the tool was not able to correctly test multiple upgrade in a row, like ''%%bionic%%'' → ''%%focal%%'' → ''%%jammy%%'' → ''%%noble%%''. This is now possible, although not perfect yet, but I’ve already enabled that kind of profile to catch issues happening during upgrades from older Ubuntu releases. Until recently, the tool was not able to correctly test multiple upgrade in a row, like ''bionic'' → ''focal'' → ''jammy'' → ''noble''. This is now possible, although not perfect yet, but I’ve already enabled that kind of profile to catch issues happening during upgrades from older Ubuntu releases.
Line 101: Line 108:
I’ve contributed to getting a baseline of green CI for the ''%%vmtests%%'' test suite, mostly by skipping all the broken tests. The goal was to make Jenkins email reports start being useful again by showing regressions to the team of Foundations developers responsible for ''%%curtin%%'' development. I’ve contributed to getting a baseline of green CI for the ''vmtests'' test suite, mostly by skipping all the broken tests. The goal was to make Jenkins email reports start being useful again by showing regressions to the team of Foundations developers responsible for ''curtin'' development.
Line 108: Line 115:
I’ve added the ''%%arm64+largemem%%'' architecture variant to the automated image promotion system. This was in the end a simple patch (almost a [[https://www.openbsd.org/lyrics.html#62|two-line diff]] ;-)), but required to dig into the code to see what was needed to support architecture variants. I’ve added the ''arm64+largemem'' architecture variant to the automated image promotion system. This was in the end a simple patch (almost a [[https://www.openbsd.org/lyrics.html#62|two-line diff]] ;-)), but required to dig into the code to see what was needed to support architecture variants.
Line 126: Line 133:
  * Continue improving ''%%autopkgtest-cloud%%'', both from the architecture/infrastructure and the user/Ubuntu developer points of view. There are a lot of friction points that would need to be addressed, and the goal is to reach a point where the infrastructure is stable enough so that the team maintaining it can have time to improve other things. One example is [[https://bugs.launchpad.net/auto-package-testing/+bug/2069751|this quite fundamental bug]] I’ve recently opened, that I’d like to close during the coming months.   * Continue improving ''autopkgtest-cloud'', both from the architecture/infrastructure and the user/Ubuntu developer points of view. There are a lot of friction points that would need to be addressed, and the goal is to reach a point where the infrastructure is stable enough so that the team maintaining it can have time to improve other things. One example is [[https://bugs.launchpad.net/auto-package-testing/+bug/2069751|this quite fundamental bug]] I’ve recently opened, that I’d like to close during the coming months.

WIP

This is a ported and updated version of my original application on Discourse, that I can't edit anymore for some reason.

Florent ‘Skia’ Jacquet - Ubuntu Contributing Developer application

Hello there! I’m Skia and would like to apply to Ubuntu Contributing Developer.

About me

My name is Florent Jacquet, but just call me Skia, even IRL, as everyone does.

I first installed Xubuntu 7.10 alternate edition on a 64MB RAM machine in 2008. The machine was already 10 years old by that time, but the lightweight system made it usable again, and it was great! Since then, I’ve installed a lot of {X,K,}Ubuntu on all my relative’s machines: parents, grand-parents, uncles, cousins, high-school companions, high school’s machines themselves, university friends and associations, you name it. I’ve personally distro-hoped for a while, but always kept either Debian or Ubuntu on my servers or machines that I wanted stable.

I now work for Canonical as part of Foundations/Ubuntu QA/Release Management team.

Contact information

Launchpad: hyask Matrix: @skia:matrix.hya.sk

Contributions

Most of my work revolves around automated testing infrastructure (autopkgtest), but also keeping alive a bunch of services like the Error Tracker, Security Britney, Merge O Matic, etc...

You can find a quite exhaustive list of my work here, as I post weekly to the Foundations Team Updates.

Since this would probably be a bit inconvenient to review, here are highlights of my most interesting contributions:

autopkgtest-cloud

This is currently one of the biggest part of my work. This infrastructure is very much central to the Ubuntu development, and keeping that running without hiccup is top priority. The list of contributions made to that infrastructure is a bit long, but here are some highlights:

Deploying the whole thing locally and improving the developer documentation

To begin my work on this topic, I’ve worked to have a local deployment of the whole infrastructure. That led to many small improvements, mostly oriented towards developers. This also made me familiar with how to self-host and administer a MicroStack.

Improved UI on per-package pages

This was a welcomed change that added the display of running and queued tests on top of the existing test results for a given package. Further details can be found on the ubuntu-devel mailing list thread.

Port everything to use the Python ''amqp'' module, and run on the latest Ubuntu

amqplib has been deprecated for a long time, and was finally removed from the archive. Moreover, a few old-ish patterns made the code unable to work correctly on modern Python and Ubuntu.

Many other small improvements and bugfixes

Improved logs displayed for running jobs, by also including the head with the command line and various other useful information.

Added a sensible vimrc to the production machines to reduce the risk of silly mistakes.

Refreshed the cloud-init userdata file we use to deploy armhf LXD remotes, and deployed new remotes in the new bos03 datacenter.

Investigated and fixed a long standing bug causing the RabbitMQ to restart about every two hours when the infra is under high load.

Even more stuff recently, as this is basically my daily job.

autopkgtest

Since this is the core tool around which our infrastructure is built, I’ve also had the opportunity to contribute to autopkgtest. We have an ongoing effort to upstream the Ubuntu patches, so I help there by providing feedback, tests, and some code review.

I’ve also uploaded the SRUs of 5.32 to Jammy and Mantic to get familiar in the process, and will certainly do more in the future.

auto-upgrade-testing

Besides daily monitoring of the service’s results, and maintaining our list of tested profiles, I’ve also made a few improvements on the software itself.

Improving the crash reporting

The crash reporting script is responsible for checking on the crashes reported by apport, making sure they get pushed to the error tracker, and setting the test status according to what was found. I’ve rewritten the whole script from a tiny piece of bash to a more readable piece of python, adding along the way much improved logging, and support for an “ignore list”. The ignore list allows us to set back to green tests that have a crash that is already reported and known, so that we more easily catch new issues happening while the first crash is being worked upon.

Support for multi-LTS upgrades

Until recently, the tool was not able to correctly test multiple upgrade in a row, like bionic → focal → jammy → noble. This is now possible, although not perfect yet, but I’ve already enabled that kind of profile to catch issues happening during upgrades from older Ubuntu releases.

curtin

I’ve contributed to getting a baseline of green CI for the vmtests test suite, mostly by skipping all the broken tests. The goal was to make Jenkins email reports start being useful again by showing regressions to the team of Foundations developers responsible for curtin development.

UTAH

I’ve added the arm64+largemem architecture variant to the automated image promotion system. This was in the end a simple patch (almost a two-line diff ;-)), but required to dig into the code to see what was needed to support architecture variants.

Security Britney

As part of my work on ProposedMigration, I’ve had to extinguish a few fires in Security Britney, which also helped make me more familiar with how Britney is organized in general.

Foundations +1 maintenance

I also regularly participate in Foundations +1 maintenance. It usually mostly consist of helping a package transition by making its tests pass in autopkgtest by (re)running it with the right triggers, but can sometimes lead to some real fixes that need to be uploaded. Here is the list of my sponsored uploads to Ubuntu.

Future goals

I’m not alone in deciding the roadmap for the team I work with, but here are some ideas of things I would like to work on:

  • Continue improving autopkgtest-cloud, both from the architecture/infrastructure and the user/Ubuntu developer points of view. There are a lot of friction points that would need to be addressed, and the goal is to reach a point where the infrastructure is stable enough so that the team maintaining it can have time to improve other things. One example is this quite fundamental bug I’ve recently opened, that I’d like to close during the coming months.

  • Getting started to help maintain stuff around apport and the error tracker, starting with adding some automated integration testing to the apport retracer mechanism.
  • Get closer to the Foundations team to provide more general help from a QA perspective.
  • Improve my packaging skills to eventually be able to apply to some upload rights and participate more in building Ubuntu itself.

Endorsements

Simon Chopin

General feedback

Skia has been one of the handful of people maintaining our critical autopkgtest infrastructure as well as other bits of our release infrastructure, and he's extremely competent at it. Notably, he often interacts with the wider dev community on IRC (and Matrix too I think?) whenever we need help dealing with those systems.

This alone would qualify him easily as a Contributing Developer, but in addition he's also well on his way on the path to gain upload rights, with regular packaging contributions.

Specific Experiences of working together

I often interact with him on infra topics (I love load-testing the autopkgtest cloud Wink ;-) ). In addition, I've sponsored a few packages for him, e.g. an autopkgtest SRU.

skia/ContributingDeveloperApplication (last edited 2025-01-13 18:42:49 by dbungert)