metalink

Differences between revisions 1 and 2
Revision 1 as of 2007-04-19 15:02:23
Size: 4886
Editor: 59
Comment:
Revision 2 as of 2007-04-19 15:19:56
Size: 4923
Editor: 59
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
 * '''Created''': [[Date(19-04-2007T20:35IST)]]  * '''Created''': [[Date(2007-04-19T21:00:54Z)]]
Line 13: Line 13:
As of right now,
As of right now, whenever a release comes there is lot of strain on the ubuntu servers as well as on the torrents. The idea is to do intelligent load-balancing while at the same time giving high speed download speed to the user without a single point of failure.
Line 17: Line 16:
  * Bob is the maintainer for the boot process for Ubuntu. In the Dapper cycle, he would like to work on getting the boot time down to two seconds from boot manager to GDM screen. He creates an entry for the specification in Launchpad, proposes it for the UBZ sprint, and starts writing out a braindump of it in the Ubuntu wiki. Magnus, who is in charge of UBZ scheduling, thinks it sounds fishy but approves it to make sure that the change is discussed and documented properly. He marks it as priority Medium because he isn't sure Bob will have time free for implementing it during Dapper.   * Bob has a 100 mbps line but he is disappointed at first when the download is slow and then due to some issue the mirror finally breaks down. -1 experience from him.
Line 19: Line 18:
  * Pedro works on Malone, in Launchpad. Before UBZ, he remembers that the dependency handling in the bug tracker is really not optimal. He writes out a Summary and Rationale in a Launchpad wiki page, registers it as a specification in Launchpad, and suggests it for UBZ. Monica, Launchpad manageress, thinks that this is really not the time to be talking about it and rejects the application for UBZ. He then indicates it for the next conference, UBB, and marks its priority is Low.   * Rachel is downloading through the torrent but finds there aren't enough no. of seeders hence she is not able to max out her speeds.
Line 21: Line 20:
  * Jason is an Ubuntu and Rosetta user. He has noticed that changes made to translations are making their way into language packs but not to the upstream versions, and adds a specification that describes a way for getting upstream to use language packs. Monica also has a plan for this but hadn't described it in a spec, so she adds it to the UBZ spec list, and adds Carlos, Rosetta maintainer, as drafter for it.   * Conanical server breaks down as no. of people leeching from the server goes up. Loss of time for everybody. Maybe strain on other parts of the site also (guessing).
Line 25: Line 24:
This specification covers feature specifications for Ubuntu and Launchpad. It is not meant as a more general specification format. Any computer connected to the net and wants to download a ubuntu .iso
Line 29: Line 28:
A specification should be built with the following considerations: People who are on high-speed bandwidth are either not able to max out their download speed or if they do max out they stress the server quite a bit. Instead of it if there is a .metalink file also then the user would be able to download his file in segments from different servers, as well as torrent , thereby reducing time to complete while at the same there is no single point of failure. Also he is able to download from the mirror which is closer to him.
Line 31: Line 30:
  * The person implementing it may not be the person writing it. It should be clear enough for someone to be able to read it and have a clear path towards implementing it. If it doesn't, it needs more detail. ==Feature list of .metalink==
Line 33: Line 32:
  * That the use cases covered in the specification should be practical situations, not contrived issues. Features [back to top]
Line 35: Line 34:
  * Limitations and issues discovered during the creation of a specification should be clearly pointed out so that they can be dealt with explicitly.     * Standard format for listing multiple mirrors for redundancy and/or automatic accelerated (segmented) downloads.
    * Combines FTP and HTTP with Peer-to-peer (P2P, shared bandwidth).
    * Simplifies advanced download features, layout, and user experience.
    * Automatic checksum verification; No separate MD5/SHA-1 file or manual process for integrity verification.
    * Automatic error recovery and repair of corrupted downloads.
    * No Single Point of Failure (SPOF) like FTP or HTTP URLs. More fault tolerant.
    * Stores more descriptive and useful information for software distribution.
    * Backward compatible with regular hyperlinks.
    * Automatic load balancing distributes traffic so individual servers are under less strain.
    * No long confusing list of possibly outdated Mirrors and P2P links.
    * Automated creating, editing, and validating is easy with standard text processing tools.
    * In a format browsers and other tools can already process (even if they might not make use of it yet).
    * Makes the download process simpler for users (automatic selection of language, Operating System, location, etc).
    * Uniquely identifies files, so even if all references to it in the Metalink stop working, the same exact file can be found in other places.
    * Share a library of files between filesharing applications with a small .metalink.
    * Congratulations, you're the first person to read this far.
    * Can finish P2P downloads even if files are no longer shared.
    * For FTP/HTTP, an updated client is needed, but not a separate client like some types of P2P.
    * NO changes or separate services needed on the server side, like a "tracker" etc. Users can create Metalinks for files anyone is hosting and share them.
    * Useful for automatic updating programs when new versions are released.
Line 37: Line 55:
  * If you don't know enough to be able to competently write a spec, you should either get help or research the problem further. Avoid spending time making up a solution: base yourself on your peers' opinions and prior work.

Specific issues related to particular sections are described further below.

=== Summary ===

The summary should not attempt to say '''why''' the spec is being defined, just '''what''' is being specified.

=== Rationale ===

This should be the description of '''why''' this spec is being defined.

=== Scope and Use Cases ===

While not always required, but in many cases they bring much better clarity to the scope and scale of the specification than could be obtained by talking in abstract terms.

=== Implementation Plan ===

This section is usually broken down into subsections, such as the packages being affected, data and system migration where necessary, user interface requirements and pictures (photographs of drawings on paper work well).
Line 75: Line 74:
CategorySpec CategorySpec CategoryMarketing

* Launchpad Entry: foo

Summary

  • Officially support .metalink format also for downloading .iso packages. Give links of the same on the most often .iso packages like ubuntu, kubuntu, xubuntu & edubuntu etc.

Rationale

As of right now, whenever a release comes there is lot of strain on the ubuntu servers as well as on the torrents. The idea is to do intelligent load-balancing while at the same time giving high speed download speed to the user without a single point of failure.

Use Cases

  • Bob has a 100 mbps line but he is disappointed at first when the download is slow and then due to some issue the mirror finally breaks down. -1 experience from him.
  • Rachel is downloading through the torrent but finds there aren't enough no. of seeders hence she is not able to max out her speeds.
  • Conanical server breaks down as no. of people leeching from the server goes up. Loss of time for everybody. Maybe strain on other parts of the site also (guessing).

Scope

Any computer connected to the net and wants to download a ubuntu .iso

Design

People who are on high-speed bandwidth are either not able to max out their download speed or if they do max out they stress the server quite a bit. Instead of it if there is a .metalink file also then the user would be able to download his file in segments from different servers, as well as torrent , thereby reducing time to complete while at the same there is no single point of failure. Also he is able to download from the mirror which is closer to him.

==Feature list of .metalink==

Features [back to top]

  • Standard format for listing multiple mirrors for redundancy and/or automatic accelerated (segmented) downloads.
  • Combines FTP and HTTP with Peer-to-peer (P2P, shared bandwidth).
  • Simplifies advanced download features, layout, and user experience.
  • Automatic checksum verification; No separate MD5/SHA-1 file or manual process for integrity verification.
  • Automatic error recovery and repair of corrupted downloads.
  • No Single Point of Failure (SPOF) like FTP or HTTP URLs. More fault tolerant.
  • Stores more descriptive and useful information for software distribution.
  • Backward compatible with regular hyperlinks.
  • Automatic load balancing distributes traffic so individual servers are under less strain.
  • No long confusing list of possibly outdated Mirrors and P2P links.
  • Automated creating, editing, and validating is easy with standard text processing tools.
  • In a format browsers and other tools can already process (even if they might not make use of it yet).
  • Makes the download process simpler for users (automatic selection of language, Operating System, location, etc).
  • Uniquely identifies files, so even if all references to it in the Metalink stop working, the same exact file can be found in other places.
  • Share a library of files between filesharing applications with a small .metalink.
  • Congratulations, you're the first person to read this far.
  • Can finish P2P downloads even if files are no longer shared.
  • For FTP/HTTP, an updated client is needed, but not a separate client like some types of P2P.
  • NO changes or separate services needed on the server side, like a "tracker" etc. Users can create Metalinks for files anyone is hosting and share them.
  • Useful for automatic updating programs when new versions are released.

Implementation

To implement a specification, the assignee should observe the use cases carefully, and follow the design specified. He should make note of places in which he has strayed from the design section, adding rationale describing why this happened. This is important so that next iterations of this specification (and new specifications that touch upon this subject) can use the specification as a reference.

The implementation is very dependent on the type of feature to be implemented. Refer to the team leader for further suggestions and guidance on this topic.

Outstanding Issues

The specification process requires experienced people to drive it. More documentation on the process should be produced.

The drafting of a specification requires English skills and a very good understanding of the problem. It must also describe things to an extent that someone else could implement. This is a difficult set of conditions to ensure throughout all the specifications added.

There is a lot of difficulty in gardening obsolete, unwanted and abandoned specifications in the Wiki.

BoF agenda and discussion

We'll have a first public session on this on the first Monday in UBZ.


CategorySpec CategoryMarketing

metalink (last edited 2008-08-06 16:21:29 by localhost)