Ubuntu Wiki

Dev Week -- Soyuz and all that Jazz - Celso Providelo -- Mon, Sep 1

(03:02:20 PM) cprov-lunch: Hi guys, who is here for "Soyuz and all that Jazz"
(03:02:22 PM) cprov-lunch is now known as cprov
(03:02:49 PM) charliecb: me
(03:03:01 PM) charliecb: hi cprov
(03:03:11 PM) ***Iulian is here too.
(03:03:11 PM) cprov: charliecb: hi
(03:03:18 PM) rulus: _o/
(03:03:36 PM) cprov: aha, anyone else ?
(03:03:55 PM) nellery:  me!
(03:04:18 PM) ***jpds is here too.
(03:04:44 PM) laga: me!
(03:04:47 PM) cprov: Okay, I have a nice chart representing Soyuz for your appreciation -> https://wiki.ubuntu.com/CelsoProvidelo/SoyuzInfrastructureOverview
(03:05:49 PM) cprov: For the records I blame daniel and his fancy title for the low-audience, "Soyuz and all that Jazz"  is too fancy :)
(03:05:52 PM) ***Shunpike is here
(03:07:09 PM) cprov: So, I'm here today for clarifying how the whole Launchpad infrastructure for distribution management (Soyuz) work.
(03:07:09 PM) DreamThief: cprov: i'm here too. and you're right. rock 'n' roll with soyuz might have been a better title ;)
(03:08:10 PM) cprov: how all the parts are glued together from source uploads until package archives.
(03:09:05 PM) cprov: DreamThief: ehe,  "Rock'n roll with soyuz" sounds a like a very good title for the next session ;)
(03:09:08 PM) bullgard4: cprov: So, 'Soyuz' is another name for 'Launchpad infrastructure'?
(03:10:29 PM) cprov: bullgard4: soyuz is the part that supports distributions/uploads/packages/archives. The model of what we call distribution management system.
(03:10:40 PM) DreamThief: bullgard4: questions in #ubuntu-classroom-chat *scnr*
(03:10:44 PM) cprov: QUESTION: laga: cprov: maybe you'd like to start with a description what soyuz actually does. because i have no clue.
(03:11:29 PM) cprov: laga: soyuz is a group of sub-system plugged on the current Launchpad model of the world which is able to:
(03:11:38 PM) cprov: 1. process source uploads;
(03:11:55 PM) cprov: 2. build source packages
(03:12:26 PM) cprov: 3. publish sources and binaries in an archive that can be used by apt
(03:13:29 PM) cprov: those 3 major tasks are implemented in a multi-{distribution, distroseries, archive}  context.
(03:13:54 PM) cprov: laga: does is make things clearer ?
(03:14:55 PM) cprov: QUESTION: laga: cprov: yes, but what's a distroseries?
(03:15:40 PM) cprov: laga: dapper, hardy and intrepid are distroseries. Distroseries is a "branch" of a distribution in the VCS analogy.
(03:16:17 PM) cprov: QUESTION: is it possible (in later stages) to adapt Soyuz so that is it able to work with other not-apt based distros (not that I'm personally intrested in it, but it sounds bad if it only supports apt)
(03:17:30 PM) cprov: nasam: yes, currently we only support the debian package format. However since the very beginning we have designed soyuz models to support other packaging formats, for example RPMs
(03:19:09 PM) cprov: so, keep the questions coming. Meanwhile let me elaborate on the 3 major soyuz sub-systems
(03:22:19 PM) cprov: take the upload-processing part as a subsystem that when fed with uploads will result in a reusable model of that package within Launchpad.
(03:23:13 PM) cprov: so Launchpad will be able to track bugs on it, associate bzr branches with it and more importantly publish that package in one or more archives.
(03:23:40 PM) cprov: QUESTION: define "reusable model".
(03:24:22 PM) cprov: laga: read 'reusable' by all the necessary metadata necessary to perform the tasks described above.
(03:25:39 PM) cprov: laga: specifically on debian packages, all the original files are in librarian (launchpad file storage) and debian control fields are directly available from our database.
(03:28:21 PM) cprov: Now few words about the source-building systems. This component is composed by a master part which is in charge of dispatching sources and collecting binaries of a farm of launchpad-buildd (sbuild-based) machines.
(03:29:32 PM) cprov: I.e. we pass them a DSC  and get back one or more DEBs corresponding to their architecture.
(03:30:38 PM) cprov: the DEBs are collected as a binary upload (.changes + N debs) which is given back to the upload-processing system.
(03:31:28 PM) cprov: One important difference between soyuz as used in ubuntu and DAK in debian is the fact that we only accept binary uploads coming from our build farm.
(03:31:45 PM) cprov: binary uploads coming from users are rejected.
(03:32:25 PM) cprov: this way ubuntu can guarantee that all binaries distributed were generated in the same controlled environment.
(03:33:21 PM) cprov: kevjava: QUESTION: How is REVU related to Soyuz?
(03:34:31 PM) cprov: currently REVU is implemented externally to what we call Soyuz, it's a third-party application.
(03:35:26 PM) cprov: I know that Michael (NCommander) is looking into using LP (specifically the PPA part) to build sources submitted to REVU.
(03:35:55 PM) cprov: and we will be working to make integration easier in the next cycle.
(03:37:26 PM) cprov: kevjava: QUESTION: What type of programming languages is Soyuz written in?  Is development done all internal to Canonical (since Ubuntu is currently the only product using it)?
(03:39:05 PM) cprov: Soyuz is written purely in python and the code was designed internally, but as everyone already know, as part of Launchpad it is also going to be released under a free-license in the next 12 (it's 11 already) months.
(03:39:18 PM) cprov: QUESTION: is soyuz already open sourced?
(03:39:31 PM) cprov: laga: no, not yet :(
(03:40:12 PM) cprov: more on the archive-publishing sub-system
(03:40:57 PM) mcas is now known as mcas_away
(03:41:23 PM) cprov: are mentioned above, this component is able to establish relationships (publication) from any Source/BinaryPackageRelease with one or many Archives
(03:42:03 PM) cprov: As in source foo_1.0 and its binaries can be published in Celso's PPA and debian and ubuntu PRIMARY archive.
(03:43:07 PM) cprov: which represents a clean workflow for distribution derivatives.
(03:45:28 PM) cprov: emet: QUESTION: Does Soyuz build the Ubuntu ISOs?
(03:46:13 PM) cprov: emet: not yet, but this task is included in the planned extensions of our buildd-farm.
(03:47:22 PM) cprov: additionally to source-builds the launchpad-buildd machine (ans its master part) will be extended to generate ISO images for hosted archives
(03:47:57 PM) cprov: also to assembly source packages from branches (bzr-builddeb/NoMoreSourcePackages)
(03:48:25 PM) cprov: laga: QUESTION: you mentioned "archives". what exactly is that? a package repository like archive.ubuntu.com?
(03:49:32 PM) cprov: laga: good question, it was obscure. Yes,  archive == repository in this context.
(03:49:52 PM) cprov: emet: QUESTION: what builds the ubuntu ISOs? can you go from a bunch of source packages to an LiveCD ISO automatically?
(03:51:00 PM) cprov: emet: currently, for ubuntu, its is a external task, archive-admins operate that in a semi-automated way.
(03:51:44 PM) cprov: emet: no trace of ISOs is stored in the Launchpad database.
(03:51:58 PM) cprov: but fear not, it's going to change very soon.
(03:53:20 PM) Cristatus: date -u
(03:53:31 PM) Cristatus: hmm....
(03:53:32 PM) cprov: uhu
(03:53:36 PM) cprov: emet: QUESTION: About how many people develop Soyuz?
(03:53:39 PM) Cristatus: how does that command work?
(03:53:53 PM) picard_pwns_kirk: Cristatus: in a shell
(03:54:03 PM) Cristatus: oh!
(03:54:07 PM) cprov: emet: we recently became 3 full-time developers.
(03:54:08 PM) breize: ^^
(03:54:24 PM) cprov: laga: QUESTION: what were your biggest annoyances with soyuz during its development?
(03:54:59 PM) cprov: laga: by far, the pressure of working in a system that needs to be working 24/7
(03:55:17 PM) mrguser is now known as hubuntu-
(03:55:28 PM) Cristatus left the room ("Leaving").
(03:55:43 PM) cprov: laga: we have to balance new features with ultimate reliability.
(03:57:03 PM) cprov: emet: QUESTION: What source control system do you use internally?
(03:57:21 PM) cprov: emet: bzr ;)
(03:57:37 PM) cprov: tacone: QUESTION: will it ever build VM appliances ?
(03:58:20 PM) cprov: tacone: I see it as a natural evolution of 'building ISOs', but I haven't seen any bug filled about this feature. Maybe you should file one.
(04:00:56 PM) cprov: okay, shall we wrap-up and give place for the next session ?
(04:01:32 PM) cprov: tacone: QUESTION: ponies ?
(04:01:41 PM) cprov: tacone: eventually ;)
(04:02:12 PM) cprov: that was a fabulous session end ...
(04:03:32 PM) pedro_: all done?
(04:04:05 PM) cprov: pedro_: yes, stage is yours
(04:04:12 PM) pedro_: cprov: great, thanks a lot!
(04:04:16 PM) cprov: Thanks you!