2004-12-11
#Ubuntu-Meeting
Ubuntu-Meeting Log 2004-11-??
Topic:Technical Board Meeting - Tuesday 30 November 2004 at 16:00 UTC Agenda: https://www.ubuntulinux.org/wiki/TechnicalBoardAgenda pitti so I copied the ascii file to http://people.ubuntu.com/~pitti/langpacks.txt pitti Hi mdz 05:01 mdz pitti: your agenda item was masked by a markup error, I didn't see it until I tried to edit the page pitti mdz: huh? pitti mdz: it was fine for me 05:01 mdz http://www.ubuntulinux.org/wiki/TechnicalBoardAgenda 05:02 mdz elmo,fabbione,kamion,keybuk,lamont,mvo_,sabdfl 05:03 fabb1one test 05:03 fabb1one ok much better 05:03 Keybuk quite finished? ;o) 05:04 sabdfl hi all 05:04 Keybuk btw, I'd just like to say, for the record, that Cadbury's are evil 05:04 mdz ok, we seem to have a quorum 05:04 pitti Keybuk: what's Cadbury? 05:04 elmo lol 05:05 Keybuk pitti: they make chocolate 05:05 elmo pitti: chocolate manufacturer 05:05 sabdfl pitti: it's a backup weapon to the jellybean 05:05 mdz the only predetermined agenda item is: language packs pitti understands :-) 05:05 pitti mdz: that was me, BTW 05:05 Keybuk (and then feel ill) 05:05 Keybuk :p 05:05 mdz if I'm not mistaken, this is carried over from the last TB meeting :-) 05:05 sabdfl mdz: we had a long discussion during your break 05:05 mdz ...which I have only skimmed 05:05 pitti mdz: yes, I distilled the discussion to http://people.ubuntu.com/~pitti/langpacks.txt and to ubuntu-devel 05:06 pitti however, the discussion on the ml wasn't very active 05:06 pitti so I would like to talk about it here 05:06 sabdfl i have a tech question for pitti 05:06 pitti we don't need to agree about every single detail 05:07 sabdfl if we set the default build process for packages so that english is installed by default by the package 05:07 pitti but we should agree on the general architecture 05:07 pitti sabdfl: this is the default, btw 05:07 sabdfl and had a mechanism so that a local build could install it's own translations for specific locally required languages, overriding the lang pack 05:07 sabdfl would this resolve the conflicts issue? 05:07 sabdfl largely 05:08 pitti sabdfl: you mean if you locally rebuild the debs from the source package? 05:08 sabdfl IOW i imagine someone building the package locally would automatically get (a) english and (b) the languages that their Ubuntu langpacks are installed for 05:08 sabdfl yes 05:08 sabdfl without having to rebuild the langpack 05:08 pitti it should be easy to confine the build to some languages, yes 05:08 Keybuk package would conflict with langpack in that situation, no? 05:08 Kamion isn't it quite important that local builds = our builds by default? 05:09 pitti this essentially means to delete all other languages from /usr/share/locale 05:09 mdz Kamion: yes 05:09 fabbione Kamion: yes 05:09 sabdfl Kamion: they would, in the sense that the deb would include english by default, as would ours 05:09 pitti but this would have similar drawbacks as F5 05:09 Kamion sabdfl: I mean in all senses, i.e. same list of files 05:09 mdz if we're going to do language packs, the language pack should become the central point for translation activity 05:09 Kamion that's really important for developers testing stuff 05:09 pitti I really thought over this issue 05:10 pitti IMHO putting the translations into proper debs leads to nothing but trouble 05:10 sabdfl agreed 05:10 pitti either our archive size or our mirrors explode 05:10 pitti and we become totally incompatible to the rest of the deb-world 05:10 mdz pitti: what do you mean by proper debs? 05:10 mdz e.g., language packs? 05:11 mdz I think there are certainly benefits 05:11 pitti mdz: putting the po files into a real deb (as in the F2 options) 05:11 Kamion pitti: nod - the approach of grabbing stuff straight to /usr/share/locale/ is looking kind of appealing right now, considering all the alternatives 05:11 pitti mdz: vs. managing the po files (and other stuff) aside from debs 05:11 mdz it allows us to safely update translations without touching the package itself 05:11 sabdfl F2-3: we should expect upwards of 200-500 languages within a year 05:11 pitti I still think that F4 is the sanest variant 05:11 smurfix_ pitti: that kills every otion but F4 (and F6 ;-) 05:11 pitti with some ideas from F3-2 05:12 pitti i. e. download new translations to e. g. /var/cache/locale 05:12 mdz pitti: did you make some estimates for disk space requirements? 05:12 sabdfl Kamion: you mean, taking /usr/share/local/ out of dpkg control, and managing it separately? 05:12 pitti and provide this as an alternative gettext root 05:12 Kamion sabdfl: with respect I find that hard to believe, considering 10+ years of free software translation work that have produced maybe 20 languages with plausible coverage 05:12 mdz 200-500 is quite a lot more than any package we currently ship 05:12 pitti mdz: the user would only download the languages and translations that he wants 05:12 sabdfl Kamion, mdz: these are clear goals of ubuntu/rosetta 05:13 pitti mdz: so disk space is minimal, without any deb overhead 05:13 Kamion sabdfl: I know, I'm just saying I find it hard to believe. 05:13 pitti mdz: and overhead of unwanted translations 05:13 Mithrandir pitti: why would F2 include conflicts? Replaces should be enough. 05:13 mdz sabdfl: yes, I'm saying that we need data to know what that would really look like in terms of resources 05:13 sabdfl fair nuff, just so you understand that's why i'm pushing this so hard now 05:13 pitti Mithrandir: I think it's a bad idea to have a language pack replace a complete application deb 05:14 Mithrandir pitti: Replaces: doesn't mean that. 05:14 pitti sabdfl: I do _not_ propose to change anything in /usr/share/locale, btw 05:14 sabdfl what about this? 05:14 Kamion sabdfl: don't think /usr/share/locale/ should be taken out of dpkg's control; pitti's F4 suggests somewhere in /var 05:14 pitti sabdfl: I only want to have additional translatiosn in /var/cache/locale 05:14 sabdfl could we update the ubuntu gettext to look in *two* locations? and take the newer? 05:14 pitti Kamion: yes 05:14 sabdfl language packas would go in /usr/share/langpack/ 05:15 pitti sabdfl: this should be easy for the majority of packages 05:15 mdz gnome-panel has 70 locales totalling 4.2M 05:15 pitti sabdfl: however, we need to manually fix packages which link gettext statically Mithrandir thinks /usr/local/share/locale would be the right place, but that's a detail. 05:15 Kamion avoiding any location that is currently used produces the huge benefit that we do not become instantly incompatible with everyone 05:15 sabdfl then the local build could go into a diff location 05:15 Kamion Mithrandir: it's still distro-managed though 05:15 smurfix_ pitti: that makes sense anyway 05:16 pitti Mithrandir: I would favor /var; /usr might be r/o 05:16 sabdfl the main thing being that the langpack would not put files anywhere a locally built deb would expect to be able to do 05:16 pitti My idea is to have a language pack which controls the downloading and installation 05:16 Mithrandir pitti: depends on where the langpacks comes from -- if they are .debs, then /usr is r/o => you lose. 05:16 mdz I like pitti's idea of treating the two pools of translations separately 05:16 pitti Mithrandir: as I said, I don't think it's a good idea to put updated translations in deb 05:17 mdz but it highlights a question about the overall process 05:17 Mithrandir pitti: why not? 05:17 mdz is our goal that these translations be passed upstream? 05:17 sabdfl mdz: yes, through rosetta 05:17 pitti Mithrandir: I explained in detail in my list 05:17 Mithrandir pitti: how else are you going to make sure that cruft is removed? 05:17 mdz if so, upstream inherits a lot of these problems 05:17 pitti Mithrandir: it's nothing but trouble 05:17 pitti Mithrandir: this must be handled by the download manager (i. e. the language pack) 05:17 mdz their source tarball grows, their default install size grows for source-based installs and for other packagers as well 05:17 Kamion mdz: upstream don't really, Debian does though 05:17 smurfix_ mithrandir: either you have LOTS of debs or your update is a 99% no-op 05:17 pitti Mithrandir: it shouldn't be too hard to sort out cruft in gettext files 05:18 Kamion most upstreams don't seem to care about source tarball size :) 05:18 smurfix_ mithrandir: both is not nice 05:18 sabdfl btw: bazaar 1.0 release in progress now 05:18 Mithrandir pitti: I'm talking about cruft as in "handling languages which are no longer on the system". You would be reinventing dpkg. 05:18 Kamion how about /usr/share/locale/extra/ or something? 05:18 Mithrandir Kamion: do people care about installed-size? 05:18 pitti Mithrandir: maybe, but using dpkg is even worse... 05:19 Kamion would be nice to keep within FHS directories 05:19 pitti Mithrandir: deleting a language pack can just remove /var/cache/locale/<lang> 05:19 smurfix_ mithrandir: it's a somewhat different problem space 05:19 pitti Mithrandir: s/delete/purge/ 05:19 Kamion or at least get this standardised somehow, since it sounds as if everyone will run into death-by-language-bloat soon 05:19 mdz Mithrandir: if installed-size were not an issue, I think language packs would be a non-issue as well 05:19 mdz well, almost 05:20 pitti mdz: there are also troubles with mirrors and pacakge rebuilds 05:20 mdz there would still be the issue of .deb size 05:20 pitti mdz: installed-size of language packs is the least evil point 05:20 Mithrandir mdz: I'm asking whether it's really an issue or whether we just have a bunch of whiners who can be ignored. (To be harsh) 05:20 mdz Mithrandir: any whiners are hypothetical at this stage 05:20 Kamion Mithrandir: CD size is pretty hard and fast 05:20 mdz we're extrapolating what would happen if the number of translations exploded 05:20 smurfix_ kamion: teaching dpkg to basically ignore some directories should be reasonably simple 05:21 mdz smurfix: that would only address installed-size, and not .deb size 05:21 Kamion smurfix_: that approach is pretty fragile and hard to reconfigure later though 05:21 Kamion and, as mdz said 05:21 pitti smurfix: why you want to do that? 05:21 sabdfl easy, Mithrandir 05:21 Kamion smurfix_: if somebody says "please can I have French translations now", you have to reinstall everything 05:21 Mithrandir sabdfl: I didn't intend to stomp on any toes -- I was wondering if we were discussing a real problem or not. :) 05:21 mdz we are 05:22 mdz though we are trying to discuss it before we actually have it 05:22 pitti Can we at least agree to drop the F2 options? 05:22 sabdfl in terms of where i would like ubuntu to be in two years time, which is much more widely translated than rhat, suse or debian, yes this will be a real issue 05:22 pitti i. e. extract translations during build into separate debs? 05:22 smurfix_ kamion: assuming that the files can't be downloaded from rosetta-or-whatever 05:22 sabdfl fine by me 05:23 mdz wait 05:23 mdz F2 are the only options which address the problem of .deb size 05:23 mdz oh, also F5 05:23 sabdfl our build process will have to be different 05:23 pitti mdz: ? 05:23 Mithrandir mdz: F5 means the shipped debs are different from what you get from apt-get -b source $package, though. 05:24 sabdfl in that we will not include the translations in a default .deb build 05:24 mdz Mithrandir: yes 05:24 mdz F5 has big problems in terms of package management 05:24 mdz but it does address the problem of .deb size 05:24 Mithrandir pitti: F3, F4 doesn't shave any size off the shipped debs. 05:24 pitti mdz: the point is that _anything_ using debs created by us makes troble 05:24 elmo F4 could be combined with a stripping 05:24 Mithrandir pitti: which we want to do, in order not to hit the 650MB limit. 05:24 mdz we could do a slight modification of F5 which results in having the same .debs in the archive and on the CD 05:24 elmo F4+strip down to English +whatever we can fit is by far the most sane option IMHO 05:25 pitti Mithrandir: yes, but _we_ have to create debs regularly 05:25 mdz local builds break 05:25 pitti Mithrandir: as opposed to F4, when the _user_ decices when to download what 05:25 Kamion sabdfl: I really think patching the source packages is the only correct way to modify the build process 05:25 sabdfl Kamion: yes, i expect that 05:25 pitti Mithrandir: this simplifies both building and downloading a lot 05:25 Mithrandir Kamion: I agree with you on that. 05:25 Kamion sabdfl: ok, good, alleviates my major concern :) 05:25 mdz sabdfl: based on our experience so far with merges, that would be a prohibitively large workload for our team 05:26 elmo mdz: okay, F4 +strip in binary and source 05:26 pitti Kamion: but I think if we can manage without patching source packages at all, this would be even better 05:26 mdz patching every source package is not an option for us 05:26 smurfix_ elmo: why both binary and source? Agreeing on either makes more sense imho 05:27 mdz elmo: hmm 05:27 Kamion mdz: the fact that a source package specifies what goes into its binary packages is an invariant; breaking that just for scheduling reasons is so wrong 05:27 elmo mdz: it's not every source, nly what's on the CD 05:27 elmo we can't not patch what's on the CD if you want local builds to work, AFAICS 05:27 Kamion if we're going to modify the build process then that change must be in Ubuntu 05:27 Kamion (i.e. not buildds) 05:27 mdz elmo: I think I like that very much 05:27 elmo smurfix: eh? the binary have to be modified to get the space down, the source has to be modified, if you want a local rebuild to procue an installable .deb 05:27 smurfix_ I'd prefer mangling the binaries when they arrive from the autobuilder. 05:27 mdz (F4 + stripping) 05:28 Kamion smurfix_: so wrong, so wrong 05:28 mdz that avoids the conflict issues with local builds 05:28 pitti Am I right that stripping and F4 are actually completely orthogonal? 05:28 Kamion think I agree with elmo here 05:29 pitti so we can discuss that independently? 05:29 Mithrandir mdz: and then leaving /var/cache/locales non-dpkg-managed? 05:29 mdz pitti: yes 05:29 mdz good plan 05:29 mdz we can decide separately how to best remove translations from .debs, and how to provide the translations separately 05:29 smurfix_ elmo: why would a local rebuild create a conflict? 05:29 mdz the former is the difficult bit 05:30 Keybuk stripping how? 05:30 mvo_ I think the later is tricky too :) 05:30 Kamion mdz: adding dh_striplocales (say) to debian/rules wouldn't be that bad, would it? 05:30 Kamion if it's a one-liner ... 05:30 Keybuk at buildd-time, or in the source package? 05:30 elmo smurfix: oh, okay, it wouldn't necessarily, but it wouldn create a different .deb, and there's a lot to be said for idempotency of builds 05:30 Mithrandir Kamion: aka rm -rf debian/$packagename/usr/share/locale ? :) 05:30 sabdfl it should be a one-time one-liner 05:30 pitti mdz: stripping binary debs is easy, but stipping source packages? We have to change every orig.tar.gz... 05:31 Kamion pitti: and then various upstream projects will accuse us of forking 05:31 pitti Kamion: how would that help to reduce the source package size? 05:31 Kamion (c.f. openssh) 05:31 Keybuk Kamion: the bit where it would go doesn't tend to change hugely -- though it'd trip on a major change to debian/rules and need manually fixing 05:31 Kamion Keybuk: yeah, but it should be mostly automergeable 05:31 Keybuk 9/10 times 05:31 Keybuk maybe 99/100 too 05:31 sabdfl pitti: source package size is not the issue 05:32 pitti elmo: do you actuallly mean "strip the source package" or "strip during building the debs from the source package"? 05:32 Kamion sabdfl: source CDs are a reality, various people want them 05:32 elmo pitti: latter 05:32 pitti sabdfl: ah, okay. I misunderstood 05:32 pitti elmo: okay, that's easier 05:32 mdz Keybuk: is it feasible to detect the case where we _only_ made this change, and fast-track it? 05:32 elmo kamion: point them at > 650Mb ISOs, what do we care? 05:32 sabdfl Kamion: they can be multi-cd sets 05:32 Keybuk mdz: yah 05:32 sabdfl dvd 05:32 Kamion sabdfl: they're already 3 CDs 05:32 Kamion sabdfl: turning them into 30 seems a bit non-trivial 05:32 sabdfl true :-) 05:32 Kamion (3 CDs for hoary as it stands) 05:32 elmo Kamion: don't you like a challenge?? 05:33 Kamion elmo: depends how fast you want auckland's disk space used up 05:33 mdz last I heard, we weren't hurting for disk space 05:33 Keybuk mdz: wish we were based on darcs rather than arch now though :p could have a "add dh_striplocales before dh_builddeb" change token :p 05:33 sabdfl ok, girls, focus ;-) 05:33 Kamion mdz: we will be if daily CD builds multiply by ten in size 05:33 mdz Kamion: I don't think we quite need _daily_ source CDs 05:34 Kamion mdz: we already have them, 'cos building CDs daily is about the only sane way to make sure they keep working 05:34 pitti before we discuss the details, is there anybody who thinks that strip+separate download is _not_ the solution? 05:34 sabdfl i don't think we need to keep them around though 05:34 mdz Kamion: if it's only as a test, we can throw away the result if the space become sa problem 05:34 mdz what sabdfl said 05:34 Kamion mdz: true 05:34 Mithrandir pitti: I wish they are handled by dpkg, but it seems people disagree with that, so. 05:35 mdz so, considering that we must remove translations from .debs in order to maintain the One True CD, I think we only have two options for the "remove translations from .debs" piece 05:35 mdz (source or binary) 05:35 pitti Mithrandir: if you find a way to sanely handle translations in deb, tell it to us :-) 05:35 sabdfl +1 source 05:35 mdz source is more correct in every way from a pure technical perspective 05:35 pitti Mithrandir: I would prefer proper debs from a cosmetic viewpoint too 05:36 pitti mdz: binary is easier, but source cleaner 05:36 Kamion source => change source package, binary => adjust binary package after build? 05:36 mdz Kamion: correct 05:36 Mithrandir pitti: they are just a bunch of files. Wrap them up in a deb; make a daily deb of all the files for a language if you so desire. 05:36 Keybuk source++ 05:36 Kamion definitely +1 source inasmuch as I don't get a TB vote 05:36 Keybuk Mithrandir: "Dear Mirrors, hello, here's SOME DEB" 05:36 pitti Mithrandir: either the mirrors or the number of debs would explode 05:36 mdz however, source thrusts us into a situation of having modified several times as many packages as we currently do 05:37 smurfix_ hmm, if source, how do the stripped-off translations end up where people can download them? 05:37 mdz so it is dependent on a greater level of merge automation 05:37 Mithrandir Keybuk: one per language; the biggest language on my system is french, that is about 8MB, compresses down to about 2.7MB. 05:37 Keybuk mdz: from 233 to "all of them" ? 05:37 pitti mdz: we can modify debhelper to do it automatically 05:37 pitti smurfix: but them into rosetta, I think 05:37 mdz some have claimed that we would only modify the packages on the CD 05:37 Keybuk Mithrandir: 200 languages, every day, half a gigabyte to each mirror each day 05:37 mdz which I think would be about 3x as many packages as we currently modify 05:38 Mithrandir Keybuk: only build if they have changed that day, then. 05:38 Kamion I think modifying dh_builddeb or something around that level in Ubuntu might be just about acceptable 05:38 Mithrandir Keybuk: I doubt we'll have changes to each language each day. 05:38 Kamion although it's a bit worrying 05:38 Keybuk Mithrandir: we upload source every day, leading to a language-pack change every day, for every language 05:38 mdz Kamion: except for non-debhelper packages 05:38 mdz Kamion: do you feel the same way about modifying dpkg-deb? ;-) 05:38 Kamion mdz: yeah, pretty trivial number though 05:38 Kamion mdz: I feel deep abhorrence for that idea; layering violation 05:38 Mithrandir Keybuk: I thought translations were to be handled completely in rosetta? 05:39 mdz Kamion: I agree, fwiw 05:39 pitti Mithrandir: but you will have a daily change in at least one package, which would trigger a rebuild of the whole langpack 05:39 Kamion debhelper is allowed to know about details of Debian policy as regards package layout; dpkg-deb is just an archiver 05:39 Mithrandir pitti: uhm, why would that be? 05:39 Kamion so s/Debian/Ubuntu/ and it looks barely tolerable 05:39 Kamion the problem is that that would break rebuilds of Debian packages 05:39 Kamion which would be a bad thing 05:39 elmo WRT debhelper, why don't we just declare it build-essential for us, then we can dh_strip_locale to any package even if it doesn't use debhelper? 05:39 pitti Mithrandir: if you have one deb per language, it just is like this 05:39 mdz Kamion: how so? 05:39 mdz I think all source-based solutions share that problem 05:40 pitti Mithrandir: if you have one deb per language per package, the number of debs explodes 05:40 Kamion mdz: no they don't, modifying our source packages doesn't 05:40 Kamion i.e. our source packages say what we want them to do 05:40 mdz Kamion: the source package modification doesn't make it into Debian 05:40 mdz you said rebuilds of Debian packages 05:40 pitti mdz: I wouldn't change dpkg-deb, only debhelper 05:40 Kamion mdz: uh-huh, but people rebuild Debian packages on Ubuntu 05:40 Mithrandir pitti: I'm talking about one deb per language. And if we do get 200 languages, well, then we have half a gig extra data to push. 05:40 Kamion mdz: e.g. backports 05:41 elmo Mithrandir: dude, please read the logs for the last meeting, we went over this TO DEATH 05:41 Kamion if some maintainer script in a Debian package does stuff to /usr/share/locale (and I believe some of them do), there'd be hell to pay 05:41 Keybuk yeah, I think some people would panic if I uploaded dpkg with all the translations stripped out 05:41 pitti Mithrandir: you have to push half a gig for _every_ language every day 05:41 Mithrandir elmo: ok 05:41 Keybuk I vote for dh_striplocales 05:41 Keybuk or dh_stripmessages 05:41 pitti Mithrandir: right now my /usr/share/locale is about 200 MB, but that will increase 05:41 Mithrandir pitti: /query? 05:41 Kamion Keybuk++ 05:41 mdz ok 05:42 pitti Mithrandir: ? 05:42 mdz so the broad solution of "modify the source package" seems to be favoured 05:42 smurfix_ kamion:not if we modify gettext and put the rosetta translations somewhere else 05:42 mdz within that solution, we have some choices 05:42 mdz - modify debian/rules 05:42 mdz - modify a debhelper tool (and, if debhelper is not used by the package, debian/rules also) 05:42 mdz - others? 05:42 pitti mdz: + modify rules 05:43 Kamion - add a debhelper tool to simplify the modification of debian/rules 05:43 fabbione - something that is absolutely shared by every package? 05:43 pitti mdz: least surprise 05:43 mdz Kamion: let's consider the debian/rules option under the assumption that we'll make the change trivial 05:43 mdz by whatever means 05:44 mdz fabbione: the only thing that falls into that category is dpkg-deb, and that's been declared evil 05:44 fabbione mdz: aren't we? ;) 05:44 mdz modifying debian/rules is going to mean adding a build-dependency as well 05:45 mdz in order to guarantee a consistent result 05:45 Keybuk mdz: or modifying build-essential Kamion could live without the build-dep for the purposes of expediency ... 05:45 mdz hmm, even to guarantee that it builds 05:45 mdz modifying debhelper lets us forego the build-dep 05:45 Kamion upload debhelper, wait for elmo/lamont to install it in all chroots, upload everything else 05:45 mdz if it's built with a debhelper that doesn't contain the change, they just don't get a stripped package 05:45 elmo modify build-essential .. 05:46 Kamion if we add a new debhelper program then the build will fail if it's missing 05:46 Kamion which is good enough 05:46 mdz hmmm 05:46 mdz and then the user is left scratching their head over why the package doesn't build 05:46 fabbione lamont will kill somebody just the amount of mails he will get :-) 05:46 elmo we need to ensure that if a user does 'apt-get build-dep gnupg', and tries to build ubuntu's source, it'll work 05:46 Kamion mdz: hoary'll ship with this modified debhelper presumably 05:46 elmo the only way to guarantee that is to modify our build-essential 05:47 mdz we'd break building Ubuntu debs on Debian, unless our debhelper change goes upstream 05:47 Kamion we could even ask joeyh to include the tool in mainstream debhelper 05:47 elmo we could (haha) || true it, or enclose it in a if [ -e ... ] type test 05:47 mdz I think he'd accept the dh_striplocales variety 05:47 Kamion mdz: we've already broken building a lot of Ubuntu .debs on Debian mind you 05:48 pitti Kamion: essentially it would just throw away any po files, right? 05:48 mdz perhaps not the dh_builddeb variety 05:48 Kamion mdz: plenty of them require Ubuntu-specific build-deps 05:48 mdz Kamion: really? 05:48 Kamion I think dh_striplocales is more correct anyway 05:48 Kamion mdz: sure, take GNOME 05:48 mdz I thought there were quite few of those 05:48 Kamion mdz: or our d-i 05:48 mdz d-i, sure 05:49 mdz Keybuk: didn't you have some numbers on how many packages don't use debhelper in warty/main? 05:49 Kamion might be worth ignoring all packages without translations 05:49 haggai I think you should try to find a solution that is upstreamable so that co-operative Debian maintainers can add the tool call to their source to reduce the debian->ubuntu diff 05:49 mdz 745/1022 I it looks like 05:49 mdz Kamion: I think the plan is to translate those, too :-) 05:49 Keybuk mdz: yeah, it came down about 30 don't 05:50 sabdfl we have a hard limit at this stage on those which don't use gettext at all 05:50 Kamion mdz: how about the ones without strings? :) 05:50 mdz 745/1022 have a build-dep on debhelper 05:50 elmo wow, that's less than I expected 05:50 Keybuk mdz: build-dep-indep too 05:50 sabdfl ok, i think we have an emergin consensus 05:50 mdz grep-dctrl -FBuild-Depends,Build-Depends-Indep -nsPackage debhelper 05:50 mdz 745 05:51 Keybuk mdz: grep-dctrl in hoary is broken 05:51 mdz oh, sweet 05:51 Keybuk grep-dctrl -s Package -FBuild-Depends,Build-Depends-Indep 05:51 Keybuk debhelper /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_hoary_main_source_Sources 05:51 Keybuk | wc -l 05:51 Keybuk 952 05:51 Keybuk Actually it catches over 90%, and of the remaining 98 packages only 28 05:51 Keybuk contain translations anyway (using the "does it depend on gettext or 05:51 Keybuk po-debconf" test). 05:51 pitti sabdfl: I think so too, since the guys are already talking about specific details 05:51 Keybuk -- 05:51 pitti sabdfl: but I think the general strategy works and is the sanest one 05:53 mdz given some magic which saves us from the crushing merge workload, the preferred solution for stripping locales is a dh_striplocales one 05:54 pitti agreed 05:54 sabdfl ok, next! 05:54 mdz can we really take that for granted at this stage? 05:55 mdz language packs are a hoary goal 05:55 mdz this level of divergence almost requires that we can put most of these merges on full automatic mode 05:55 mdz including the upload 05:55 sabdfl mdz: let's evaluate that after the first week of Mataro Sessions 05:56 sabdfl we'll have a handle on hct at that point 05:56 Keybuk hct doesn't help any more than mom 05:56 Keybuk a little less, in fact, because it requires tla/baz to play nice 05:56 elmo it's worth noting, that we're not running out of space yet either, so they may be a goal but they're not a 100% we can't ship without this 05:56 sabdfl yes 05:56 mdz elmo: however, it is a high priority item on Mark's hoary list 05:57 Keybuk mom works by breaking apart, mutating and driving patch itself 05:57 mdz which is fairly close to the same semantics 05:57 Keybuk we don't get that power with arch 05:57 Kamion (we have 55MB free on the powerpc CD, which is tightest) 05:57 mdz Kamion: and we need to get a ppc64 kernel image onto it :-/ 05:57 mdz Keybuk: yet :-P 05:57 Kamion mdz: should be OK ... 05:58 mdz that'll eat about 15M 05:58 mdz guestimating from the powerpc one 05:58 Kamion mdz: depends which of powerpc/power3/power4 we need 05:58 Kamion (can we bounty benh to fix the MMU crap that means we need those three? :-/) 05:59 elmo can't we pay/beg/bribe/hold hostage someone to get rid of the stupid sub-arch split... doh 05:59 mdz ok, so regarding stripping locales, we're fairly settled on dh_striplocales, on the assumption that Keybuk can provide the level of support we need for the merges 05:59 pitti sabdfl: is there actually a "next" item on the list? 05:59 mdz Kamion: let's talk about that a bit later 06:00 mdz Kamion: doesn't sound unreasonable at first hearing 06:00 mdz pitti: yes 06:00 mdz the second half is the approach for providing the translations in separate .debs 06:00 mdz it sounds like we were moving in the following direction there: 06:01 mdz - buildds collect the translations via dh_striplocales and drop them into a pool 06:01 mdz - some automated process runs periodically and builds .debs from the pool 06:01 elmo debs are crack for this 06:01 mdz - the .debs contain translations in some directory which is not /usr/share/locale 06:01 mdz oh? was that decided at the last meeting? 06:01 elmo I dunno, I'm just saying what I think 06:02 mdz ok, s/\.debs/language packs/ for now 06:02 pitti mdz: My idea is to have language-support-XX manage the download, install, and housekeeping 06:02 pitti mdz: but not stuffing the po files into a deb itself 06:02 mvo_ I think the problems reamins the same whatever format we use 06:02 pitti a reasonable compromise would be to build a deb on the fly on the user's machine 06:02 mdz pitti: I think that introduces much more complexity than we want 06:03 pitti mdz: but if we push debs, we have all these problems 06:03 mvo_ if we don't use deb we need to have package manager like functionality in a language-pack-get tool 06:03 pitti mdz: assembling debs on the fly on user's hosts seems much more sane then 06:03 Mithrandir pitti: I'm not too comfortable with that -- l-s-xx needs to be able to acquire, unpack, record what's installed and uninstall the files later. 06:03 mdz pitti: I think that has the same problems 06:03 pitti mvo_: but only downloading, copying and cleaning 06:03 mdz there is a lot of value in having a language pack blob which actually contains the translations 06:03 pitti Mithrandir: why record? 06:03 mdz which can be placed on the CD, passed around by the user, etc. 06:04 Mithrandir pitti: it needs to know what it has installed so it knows what it should update. 06:04 pitti mdz: but the whole point of all this is to avoid debs in the first place 06:04 pitti at least debs we provide on our mirrors 06:04 Mithrandir elmo: you think .deb files are insane because of the mirror load? 06:04 mdz it is? 06:04 Mithrandir elmo: or anything else too? 06:05 elmo Mithrandir: Packages files are useless for them and too large 06:05 elmo and yes, I don't desperately want them in the pool proper 06:05 pitti Mithrandir: not only because of the mirror load, also because of the download redundancy 06:05 pitti The user should be able to download only the translations for packages he actually uses 06:05 mdz elmo: don't desperately want, or desperately don't want? 06:05 haggai could new functionality for registering conffiles etc with dpkg (Keybuk?) be used to register new translations with the dpkg database? 06:06 mdz pitti: what it sounds like you're describing is a .deb which can't be installed without access to <someplace>.ubuntu.com 06:06 pitti mdz: ? 06:06 mvo_ when will he download them? will he need to do a apt-get upgrade and a lang-pack upgrade? 06:06 mdz pitti: <pitti> mdz: My idea is to have language-support-XX manage the download, install, and housekeeping 06:06 pitti mdz: l-s-XX would download the translations, create a deb and isntall it 06:06 mdz pitti: correct 06:06 Mithrandir mvo_: think install from cd, for instance. 06:06 pitti mdz: you need access to Rosetta, right 06:06 elmo mdz: depends what format the debs are, one per source or one per lang? 06:06 mdz that means that installing l-s-XX would require that the target system be able to talk to <someplace>.ubuntu.com 06:07 elmo mdz: the latter can be over my dead or P45-ed body 06:07 pitti mdz: but you need that anyway to get _any_ update 06:07 mdz which, in a wide variety of environments, just won't be true 06:07 Keybuk haggai: doesn't work, conffiles still need a replaces:/--force-overwrite 06:07 pitti mdz: building the deb can happen on the client 06:07 smurfix_ mithrandir: so the files are on CD instead of being downloaded 06:07 elmo mdz: the former.. I'm pretty unhappy about too, but I'd need to look at proper stats to be sure 06:07 mdz pitti: that isn't true, users have lots of ways of moving around .deb files 06:07 mdz pitti: the user needs to be able to insert a CD and obtain a language pack that way 06:07 pitti mdz: they can build the deb on a place with net access 06:08 haggai Keybuk: I didn't mean that. I meant, the ability to register a new file with the language-support deb 06:08 pitti mdz: they can as well just copy the po tree to their computer (with the help of l-s-xx) 06:08 mdz pitti: this defeats the purpose of providing a plug-and-play language pack 06:09 mdz what is the problem that we would hope to solve with such a solution? 06:09 pitti mdz: I still don't understand the problem 06:09 mdz large downloads for people tracking the development branch? 06:09 pitti mdz: where the deb is built, or who does it makes no difference 06:09 mdz pitti: the problem is that it's essentially an installer .deb, and installer .debs are inherently problematic 06:09 pitti mdz: we can put it onto a CD as well 06:09 sabdfl we can accept a certain lag during the freeze 06:09 Mithrandir mdz: large downloads for people who are running stable but want updated translations, I think. 06:09 sabdfl so, say, langpacks are updated weekly rather than daily 06:10 mdz sabdfl: right 06:10 mvo_ sabdfl: sounds good too me 06:10 sabdfl i think elmo is right that there are benefits to avoiding the pool altogether for this 06:10 mdz and we can provide a separate repository with daily updated language packs for those who really want them 06:10 pitti guys, it is just insane to publish translation debs in regular intervals 06:10 smurfix_ why do we need .debs anyway? 06:10 sabdfl as is mithrandir in that it's a small step towards reinventing dpkg :-) 06:10 mdz if the problem is that the .deb is too large, we break it up 06:10 Mithrandir .. you know, dpkg started out as a perl script. We could reinvent it in python. 06:10 sabdfl but... fundamentally, a langmgr on an ubuntu desktop needs to keep track of the languages that desktop needs, and sync in the data 06:10 pitti let the user choose which translations he want, at which intervals and give him an easy way to update his system with a "pull" strategy 06:10 sabdfl it might even use rsync! 06:11 Kamion Mithrandir: actually it was a shell script before that ... 06:11 Kamion (IIRC) 06:11 mdz sabdfl: why does it need to do that? 06:11 Keybuk you could always use the deb + delta idea ... 06:11 pitti mdz: what do you mean by installer deb? 06:11 mdz sabdfl: the user selects their languages, and from that point forward it's a simple upgrade 06:11 mvo_ Keybuk: +1 06:12 mdz pitti: a .deb which doesn't contain the data that it is meant to provide, but instead downloads it over the network (or requires that the user do so) 06:12 pitti mdz: with per-package debs or per-language? 06:12 mdz Keybuk: not for hoary we can't 06:12 sabdfl mdz: you mean, if we have an ubuntu-xx langpack in the pool? elmo hates that idea, and he's bigger than me, and i happen to agree with him 06:12 pitti mdz: you can also install readymade translation debs 06:12 mdz sabdfl: I'm going to go out on a limb and ask him to justify his hatred :-P 06:12 mdz sabdfl: you, too, if you agree 06:12 pitti mdz: it should be _possible_ to assemble them on the fly, not required 06:12 sabdfl the langpack debs will drive the mirrors crazy 06:13 sabdfl we can publish the translations as an rsyncable tree 06:13 Mithrandir sabdfl: if they're only updated when they need to, or weekly or something as well? 06:13 mdz as I said 06:13 mdz if the problem is that the .debs are too big 06:13 mdz then we break them up 06:13 elmo mdz: it's utter crack. if you change one translation in one source package, this n hundred Mb unrsyncable (and not normally rsynce danyway) .deb changes 06:13 sabdfl then dpkg/apt need to scale to more debs than they should have to 06:13 Mithrandir mdz: then you get a huge amount of them. 06:13 Kamion mdz: doesn't help, lots of them will generally change at once 06:13 pitti ARGH 06:13 mdz depends on how you break them up 06:13 pitti this is the very same discussion we had last time 06:14 Kamion mdz: either you have a source message which changes and lots of translations change, or you have a common translation of a frequently occurring string that changes 06:14 Kamion mdz: you can't win either way 06:14 elmo pitti: yes, I know. let's petition sabdfl to not let mdz go on holiday and miss TB meetings ;-) 06:14 pitti pushing debs to mirrors is crack 06:14 sabdfl elmo: only if you promise not to go on holiday too ;-) 06:14 smurfix_ I agree with sabdfl. Just use rsync and ignore all files belonging to packages you don't have installed. Problem solved. 06:15 pitti smurfix: when using /var/cache/locale, we don't even have the possibility of hitting pacakge conflicts 06:15 sabdfl smurfix_ it's easier than that, since the desktop install generally installs a solid well-defined set of packages 06:15 sabdfl we only do this for those 06:15 mdz I think you guys are optimising for the development branch case 06:15 mdz when we need to optimise for the stable release case 06:15 sabdfl yes, understood 06:15 mdz these problems are non-issues for the stable release 06:15 pitti mdz: you cannot force users to download huge amounts of debs for small translation changes 06:16 Mithrandir sabdfl: you'd need to ship something to rsync off on the cd, then. That is a lot more work than shipping one more deb. 06:16 pitti mdz: s/cannot/it wouldn't be the best solution/ 06:16 sabdfl but i /really/ dislike the idea of switching something this big on only at release time, we discussed that last time 06:16 elmo mdz: dude, this is not "optimization", this is "having a mirror network vs. not" 06:16 smurfix_ pitti: exactly. local packages use /usr/share. So check there too, and pick the newer one. 06:16 elmo mdz: if we implemented this with even the number of languages/translations we have today, we'd have zero mirrors within a month 06:16 elmo hell, _I_ wouldn't mirror us over the GB LAN :-P 06:17 sabdfl elmo: i love how you tell the truth :-p 06:17 smurfix_ mithrandir: so park the rsync-master file tree onto the cd ..? 06:17 mdz elmo: ok, I understand. let's try to change our approach to adapt to that, rather than throwing it away 06:17 Mithrandir smurfix_: have you touched debian-cd? 06:17 Kamion smurfix_: er ... uh-huh. 06:17 mdz I'll say again that this is strictly a development branch issue 06:17 Mithrandir smurfix_: doing that would be very icky and interesting. 06:17 Mithrandir smurfix_: I do not think Kamion would like you afterwards. :P 06:17 pitti mdz: so we need a system that is both able to install premade language debs, prepare these and also update directly from the network 06:17 mdz and I do not think that we should sacrifice having the feature look the way it should in the final release 06:18 Mithrandir mdz: I'm worried that elmo is so much against it, though.. 06:18 smurfix_ kamion: On the language-pack CD of course. Maybe packaged inside a cpio tree if too many small files are too icky. 06:19 Kamion smurfix_: I think all this is highly premature 06:19 Mithrandir smurfix_: that sounds like you're reinventing rpm rather than dpkg, then. 06:19 mdz Mithrandir: the issues that elmo raises apply only to the development branch, and I think the use case is different enough that we may need to solve the problem in two slightly different ways 06:19 Mithrandir mdz: mirrors mirror all, though 06:20 Kamion smurfix_: language pack CD structures aren't something we've even started to think about yet, and I'm not convinced that they're something we ought to be generating centrally anyway 06:20 pitti mdz: why, if stable users should be able to download their daily translation updates as well, it affects stable too 06:20 mdz we are not going to force a huge delta increase on mirrors 06:20 Kamion better to give people an easy way to build an Ubuntu CD for their language 06:20 elmo mdz: dude, it's not just the development branch. what about a security update that changes a translation? 06:20 mdz the sky is NOT falling 06:20 mdz elmo: they don't 06:20 elmo bullshit 06:20 elmo you've uploaded new upstream versions before - it's rare, but it happens 06:20 Kamion mdz: also security issues exist in translations; consider format string errors 06:21 mdz if we have a clusterfuck like that, we put the changed translations into the security update .deb 06:21 smurfix_ Well, I'm brainstorming on "how could this work" rather than "how do we make .debs to make it work", so ... 06:21 elmo mdz: and what Replaces: the langpack? 06:22 Kamion and if the langpack is a .deb we then run into the non-commutativity of Replaces: 06:22 Kamion (muttermutterdpkgbugsmuttermutter) 06:22 mdz elmo: uses a different path 06:22 Mithrandir versioned conflicts with the broken version of the langpack. (Yes, it's broken, but it works-ish) 06:22 elmo mdz: and how do apps know which to use? 06:22 Kamion Mithrandir: urgh 06:22 Kamion Mithrandir: yay for removing langpacks while upgrading packages 06:23 mdz elmo: they prefer the one provided by the package, if present (generally not) 06:23 Mithrandir Kamion: doesn't apt/dpkg DTRT and upgrade? 06:23 smurfix_ ... or the one timestamped newer. 06:23 elmo mdz: that technology exists for locales already? 06:23 mdz elmo: if it doesn't, it seems completely trivial to implement 06:23 Kamion Mithrandir: not if an updated langpack isn't available yet 06:24 Mithrandir Kamion: then we have to make sure it's available. (Yes, I know that's not trivial either) 06:24 Keybuk Mithrandir: dpkg does not upgrade on << conflicts. 06:24 Mithrandir Keybuk: what about apt and dselect? 06:25 mdz Mithrandir: depends 06:25 elmo mdz: *shrug* ok then. so how/when do you propose to transition to the stable model as part of the development process? 06:26 mdz depends on how we approach the development-branch model 06:26 elmo I still think you're trying to fit a square into a round hole, 'cos the round hole is familiar and looks pretty, but that's probably just me 06:26 pitti err, what is our "stable model"? and why it should be any different from the development one? 06:26 mdz pitti: put yourself in end-user mode for a moment and consider the problem 06:26 mdz "I want my Ubuntu to speak <language>" 06:26 pitti mdz: stable versions do not have any fewer updates than unstable ones 06:26 elmo oh, hang on 06:26 elmo mdz this doesn't work 06:27 mdz what's "this"? 06:27 elmo mdz: one of the reasons for all this crack, is sabdfl's desire to have translation updates out of bound 06:27 elmo i.e. in stable, still get translation updates 06:27 pitti mdz: apt-get install l-s-<language> with a net connection works 06:27 elmo so there is no stable model 06:27 pitti mdz: if you don't have a net connection, make it easy to copy the files from another host which has 06:27 pitti mdz: but that is orthogonal to the particular solution anyway 06:28 Mithrandir elmo: push updated translation-debs to warty-updates once a week? 06:28 mdz pitti: "you must have Internet access in order to do that, otherwise use this manual process"? 06:28 sabdfl mdz: what elmo said. but not about the pretty familiar round hole 06:28 mdz that answer won't fly for a lot of deployments 06:28 pitti mdz: how do you want to download stuff without net? 06:28 pitti mdz: it doesn't matter if you download debs or po file tarballs 06:28 pitti mdz: "you" == the language pack which manages the stuff 06:29 mdz sabdfl: what's the basis for updating stable with new translations, as opposed to doing the translations as part of the freeze process, as GNOME does? 06:29 elmo Mithrandir: that has the "mirror admins lynch you" problem all over again 06:29 Mithrandir elmo: yes, that might be a slight problem. 06:29 sabdfl mdz: reality 06:29 mdz pitti: there is a big difference between a packaging system which downloads packages, and a package which downloads .po files 06:29 Mithrandir pitti: we end up having to reinvent dpkg to grab the translations off $MEDIA 06:29 pitti mdz: we can include the current language snapshots on the CD too if we want 06:30 mdz pitti: you open yourself to a lot of site-specific problems 06:30 mdz outbound access policy, proxy authentication, ... 06:30 pitti mdz: but we can use debs to hold the actual updates 06:30 mdz pitti: and have the package copy them from the CD when it's installed? 06:31 mdz then suddenly you need to implement this in apt 06:31 pitti mdz: the point is not to avoid debs, but to avoid building and pushing them to our servers regularly 06:31 pitti mdz: as I said, you can assemble the debs either on the fly or use premade ones 06:31 Mithrandir would having a separate repository for translations be complete crack? 06:31 mdz if it's a problem for them to be pushed to the mirrors regularly, then may I make the bold suggestion that we, well, don't push them to the mirrors regularly? 06:31 pitti mdz: create them on the fly in rosetta? 06:32 pitti mdz: when an user wants to update his translations? 06:32 elmo mdz: or how about this for a bold suggestion: don't insist on sticking with a format that's so inflexible we can neither split them up, nor ship them in one big lump, and that way get to ship them as often as we want 06:32 mdz pitti: what would the system look like which would decide whether to create a .deb or use an existing one? which piece of the package management system would it inhabit? 06:33 mdz what is the proposed alternative transport format? a few million .po files? 06:33 smurfix_ mdz: million? 06:33 pitti mdz: ^ that is actually the most flexible one 06:33 pitti mdz: the user needs to download exactly the po files he needs, nothing else 06:33 elmo per-source or per-binary package lumps of .po files is an alternative too 06:33 pitti mdz: but still, having deb as transport format is not a bad idea on itself 06:34 mdz smurfix: O(packages*languages) 06:34 mdz which is O(thousands*hundreds) at least 06:34 pitti mdz: the bad idea is not the deb format, but to include the debs into our normal archive structure 06:34 mdz elmo: how exactly is a per-source or per-binary lump of .po files better or worse than per-source or per-binary .debs? 06:34 pitti mdz: re your first question: the deb would be recreated if the user presses "update language" 06:35 pitti mdz: of course with network access 06:35 elmo mdz: by definition, they wouldn't be in the pool or bloating the Packages files 06:35 pitti mdz: alternatively the user can install an updated deb from somewhere else 06:35 mdz elmo: instead, they would be in a separate "pool" with its own index file and bloat that? 06:35 smurfix_ mdz: end users won't see that many files and mirrors can use more scalable distribution methods if necessary. 06:35 elmo mdz: if you use .debs, yes :P 06:36 mdz elmo: or anything else 06:36 pitti mdz: if you download po files direcly, there is no need for index files at all 06:36 mdz storing the .po files individually doesn't scale at all 06:36 pitti mdz: why not? 06:36 mdz aggregation is a must 06:36 mdz pitti: how will you determine which ones to download? 06:36 pitti mdz: timestamps? 06:37 mdz pitti: you will do thousands and thousands of HTTP HEAD requests? 06:37 Keybuk sorry guys, I'm going to have to bow out now 06:37 mdz or download an index with every .po file in it? 06:37 pitti mdz: not necessarily 06:37 Keybuk have to pack for both London, Mataro and Christmas-week tonight 06:37 pitti mdz: you can tell rosetta "give me all translations newer than date x" 06:37 pitti mdz: rosetta would give you a tarball-like answer 06:38 pitti mdz: the timestamps can be sent out by rosetta as well 06:38 pitti mdz: so we don't need to rely on clocks 06:38 pitti mdz: or peek on po files 06:38 mvo_ pitti: I'm not sure if this will not hit rosetta way too hard if a lot of users use our system 06:38 pitti mdz: we just store the last timestamp/index mark/whatever of the last update 06:38 smurfix_ if http head doesn't scale, then don't use it ... sounds simple. :-P 06:38 pitti mvo_: if users start to download translations daily, it doesn't matter if we hit rosetta or archive 06:39 Mithrandir pitti: rosetta isn't mirrored. 06:39 mdz smurfix: the point is that we don't have a choice in whether to aggregate .po files into larger blobs 06:39 mdz which seems to be the idea that everyone is attacking 06:39 pitti Mithrandir: then let's mirror it :-) 06:39 elmo mdz: no, what I'm saying is that putting these in .debs in the archive is not a sane solution, given the requirements 06:39 pitti mdz: but the aggregation would just be temporary as a download armor 06:40 pitti mdz: s/armor/transport format/ 06:40 Mithrandir elmo: do you think putting them somewhere which is not mirrored would be as bad? 06:40 pitti Mithrandir: if we don't have the bandwidth to support daily translation downloads, then there is no solution at all, I think 06:41 Mithrandir if so, the language-pack tool could just be a small shell script which wraps apt similar to d-i's build process. 06:41 mdz elmo: how do you propose that we provide them as a portable, CD-friendly, installable, removable chunk of data, then? 06:41 pitti same transport format as for download, I think? 06:41 Mithrandir pitti: putting plain files on CDs is painful, .debs are easy. :) 06:41 pitti all translations later than 0 06:41 elmo mdz: whatever you want dude, I'm not trying to impose a solution on you or pretend to be clever enough to have even close to a good one. I'm just trying to tell you what is and isn't possible and/or sane from a archive/mirror perspective 06:42 pitti Mithrandir: so wrap it up in a deb for my sake 06:42 mdz elmo: the message I'm getting is that we need something which provides .deb-like semantics without sending mirrors running away screaming 06:43 mdz even if we accept that we need a delta-update mechanism 06:43 pitti mdz: maybe we should think a bit about this particular problem before continuing 06:43 mdz we need an initial blob that we ship on CDs 06:43 mdz pitti: aw, it hasn't even been two hours yet :-) 06:43 smurfix_ the transport blob on cd would just have a few files you don't want *shrug* 06:44 pitti mdz: the cd can wrap up rosetta's transport format into a deb 06:44 mdz how do we mirror translations, if not as .debs? 06:44 pitti mdz: we should rather mirror rosetta's database 06:44 smurfix_ mdz: a replicated datavase? 06:44 Kamion Mithrandir: .debs not in the main archive are no easier than random files, TBH 06:44 mdz smurfix: say what? 06:44 smurfix_ database 06:45 Kamion Mithrandir: at least not significantly 06:45 Mithrandir Kamion: they are. 06:45 mdz smurfix: I understood that, but what do you mean? 06:45 mdz smurfix: "run a copy of this application"? 06:45 Mithrandir Kamion: just duplicate the code to do security patches on the CD. 06:45 Kamion Mithrandir: it's pretty trivial to get debian-cd to copy random files onto a CD 06:45 Kamion Mithrandir: I know how to do it 06:45 Kamion Mithrandir: the question is doing anything useful with them at the other end 06:45 mdz anything which is not a flat file is not going to be mirrored to an extent even close to the current archive 06:46 mdz let's explore that a bit 06:46 mdz so we have some random data on the CD 06:46 mdz well, not so random 06:46 mdz translation data 06:46 mdz which we need to install on the system 06:46 pitti mdz: but with any aggregation you will loose the flexibility of individually updating po files 06:46 Kamion mdz: I'm not necessarily saying it's a good idea, just that it's not impossible 06:47 elmo mdz: it doesn't necessarily need to be - the hit on translations is going to be an order of magnitude less than archive for some time tocome, and if we come up with something that doesn't involve databases, we will get mirrors for it in time 06:47 Mithrandir pitti: yes, you will. Some overhead is fine. 06:47 pitti mdz: the blob could be put into something similar to /var/cache/apt/archives, and l-s-XX could use that instead of downloading 06:47 mdz elmo: something that doesn't involve databases == flat file, no/ 06:47 mdz ? 06:47 smurfix_ mdz: it's one possible solution. Whether it's actually feasible for mirrors to run mysql-or-whatever is a different question, which I'm not qualified to answer. 06:47 mdz pitti: so d-i would do this copying? 06:47 elmo mdz: no, I mean doesn't invovle replicating databases.. 06:47 pitti mdz: it probably has 06:48 pitti mdz: similar to archive-copier 06:48 mdz smurfix: I can say, even in near-complete ignorance of the existing mirrors, that it is not feasible 06:48 elmo mdz: i.e. anything that can be mirrored via http, ftp and/or rsync 06:48 pitti mdz: if l-s-XX sees that something is already in the cache, it doesn't need to contact rosetta 06:48 mdz elmo: agreed 06:48 pitti mdz: just an idea... 06:48 mdz pitti: what about the case where I have installed the system, and then later want to install an additional language? 06:49 pitti mdz: without downloading? 06:49 mdz pitti: right, say I have an Ubuntu DVD which contains translations for many languages 06:49 Kamion I would like to avoid piling more and more stuff into d-i which we don't have an interface for changing in the real system 06:50 pitti mdz: same technology as with the initial install, I think, but we need an easy interface for that 06:50 Kamion d-i should certainly take care of installing your primary language, but it's not going to solve the whole problem for you 06:50 pitti Kamion: in fact d-i shouldn't bother about these blobs at all 06:50 pitti Kamion: d-i should just install l-s-XX and be fine 06:50 Kamion pitti: mkay 06:51 pitti the lang packs should have the knowledge of getting their blobs 06:51 Kamion pitti: but I think you'll find it needs to bother for net-less installs 06:51 pitti Kamion: at least this seems to be more consistent 06:51 Kamion pitti: at least at the archive-copier level or similar 06:51 pitti right 06:51 mdz pitti: the language pack will not be able to get data from the CD 06:51 pitti not? 06:51 Kamion pitti: the CD's ejected 06:51 mdz I would object to reimplementing apt-cdrom in the language pack 06:51 pitti mdz: me too 06:52 pitti maybe archive-copier should be extended then 06:52 pitti I don't have a very good solution without thinking about it for at least some hours 06:52 mdz the reason that I'm hesitant to give up on .debs entirely is that so many of these problems simply go away for .debs 06:52 pitti it's all ad-hoc solutions 06:52 pitti mdz: then let's find a solution which uses debs as transport format 06:52 mdz we even have gnome-app-install as a nice interface for add/remove 06:53 pitti mdz: use debs as wrapper, but don't publish them onto our normal archive 06:53 mdz I'm at a disadvantage, having missed the previous meeting 06:53 mdz was a consensus reached there that .debs really aren't the way to go? 06:53 elmo no 06:53 pitti mdz: well, at the previous meeting we argued about extracting po files during build and shipping the debs in our archive 06:53 pitti mdz: no 06:53 mdz if we move away from .debs, we trade the delta-update problem for a bunch of other problems which aren't necessarily less 06:54 pitti mdz: wouldn't that be cured by allowing debs to be built on the fly? 06:54 pitti mdz: allowing, not requiring 06:54 pitti mdz: initial translations would be on the CD as normal debs 06:54 mdz pitti: I'm not clear on exactly what the debs-on-the-fly solution would look like, can you describe it? 06:55 pitti but upgrades would assemble a new deb based on existing and downloaded translations 06:55 pitti this new deb is then installed normally as a new revision of the old one 06:55 pitti something along that line 06:55 mdz what piece of software would be responsible for building the .debs? 06:55 pitti the langpack, Isuppose 06:55 mdz the language pack deb? 06:55 pitti it would depend on dpkg-dev 06:56 pitti it should be relatively easy to pack together a bunch of po files with a boilerplate control directory 06:56 mdz if you're thinking of invoking dpkg --install in postinst or something like that, I think there are solid technical reasons not to do that 06:57 pitti mdz: no, not necessarily in postinst 06:57 Kamion can't be done in postinsts 06:57 mdz definitely can't be done now, and good reasons not to fix it 06:57 pitti mdz: of course we have to delay that somehow 06:57 mdz pitti: dpkg post-hooks or some such? 06:57 pitti mdz: I don't know instantly 06:57 pitti mdz: but I feel that there surely is a sane solution for that 06:58 pitti mdz: btw, creating the deb is not done in the postinst 06:58 pitti mdz: it is done in sudo update-language-pack or so 06:58 pitti mdz: and this could install the deb 06:58 mdz pitti: it seems to me that you end up solving most of the .deb delta update problem in the process 06:58 mdz in which case we should just solve the .deb delta update problem and be done with it 06:58 pitti mdz: we just have to find a way to call update-lang-pack 06:59 pitti outside of langpack postinst, that is 06:59 pitti but we solve the mirror problem, the download redundancy and pretty much all other problems AFAICS 07:00 pitti actually, when I think about it 07:00 mdz true, the ideal solution for .deb delta updates in apt wouldn't necessarily work for rsync mirrors 07:00 pitti the translations don't need to be updated in sync with pacackage updates 07:00 pitti the user could click onto a button "update translations" which would just call update-language-... 07:01 pitti especialyl in stable releases this is a fine thing 07:01 pitti since you don't normally upgrade pacakges anyway 07:01 mdz this is similar to how the OAV signature stuff works, for example 07:01 pitti (modulo security udpates) 07:02 pitti Proposal: I will think about it once more and post my result to the list 07:02 mdz yes, we need to close this meeting 07:02 pitti unless anybody has a great new idea 07:02 mdz interest is waning :-) 07:02 pitti I'll sum up on the list 07:03 mdz let's give it some more thought and revisit in Mataro 07:03 pitti oh right, a BOF! 07:03 Mithrandir BOFs are good. 07:03 mdz we will have translators, etc. there as well 07:03 pitti I will work out the current plan for Mataro 07:03 pitti mdz: right, I already saw the planned bof on the wiki 07:04 mdz ok 07:04 mdz thanks, everyone 07:04 mdz meeting adjourned 07:04 pitti byebye Generated by irclog2html.pl 2.1 by Jeff Waugh - find it at freshmeat.net!
MeetingLog/Ubuntu/2004-12-11 (last edited 2008-08-06 16:37:49 by localhost)