EarlyUserspace
|
Size: 1882
Comment:
|
← Revision 41 as of 2008-08-06 16:19:00 ⇥
Size: 7397
Comment: converted to 1.6 markup
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from UbuntuDownUnder/BOFs/UbuntuDevelopment/EarlyUserspace = UbuntuDevel/EarlyUserspace = |
## page was renamed from EarlyUserspace = Early Userspace = |
| Line 8: | Line 6: |
| * Created: [[Date(2005-04-24T00:19:19Z)]] by MattZimmerman[[BR]] * Priority: LowPriority[[BR]] * People: NeedsLead, NeedsSecond[[BR]] * Contributors: JaneSilber[[BR]] * Interested: JeffBailey [[BR]] * Status: UduBof, DistroSpecification[[BR]] * Branch: [[BR]] * Malone Bug: [[BR]] * Packages: [[BR]] * Depends: [[BR]] * UduSessions: 1[[BR]] |
* Created: <<Date(2005-04-24T00:19:19Z)>> by MattZimmerman<<BR>> * Priority: MediumPriority<<BR>> * People: JeffBaileyLead, TollefFogHeenSecond<<BR>> * Contributors: JaneSilber<<BR>> * Interested: AndresSalomon, PaulSladen<<BR>> * Status: BreezyGoal, EditedSpecification, DistroSpec, JeffBaileyQueue, MattZimmermanQueue * Branch: <<BR>> * Malone Bug: <<BR>> * Packages: <<BR>> * Depends: <<BR>> * UduSessions: 1(0)<<BR>> |
| Line 22: | Line 20: |
| Develop a plan for a new early userspace system for Breezy which will increase robustness and allow for more flexible deployment of Ubuntu. Discussion must cover: * Transition plan for initrd->initramfs * hotplug or hotplug-ng? * Finding the root filesystem * Generating a unique id? * Mounting by label/uuid * Falling back to existing behaviour * Precalculating boot device and driver requirements * Debuggability * More verbose output in recovery mode * Do something smarter than panic if things go wrong. Dump debug info? * Availability of command-line recovery tools |
Early User``Space is defined as the time from when the kernel hands control over to userspace applications until the time that the full system init is called. The kernel traditionally has included a lot of functionality that can be easily done in userspace. These include: * DHCP * NFS Root * devfs * Partition detection * Raid (md) detection * root filesystem mounting * Device detection. Many of these items are called using the `prepare_namepace()` call in the kernel. When using an initramfs solution, the `prepare_namespace()` call is skipped, and a normal user binary is called to complete the setup which is responsible for chaining to the full system init. |
| Line 40: | Line 42: |
| Upstream is moving towards this solution. Some features, such as devfs are scheduled for removal in a very short period of time. Writing kernel code is difficult, since each feature causes bloat that is included in every generic Ubuntu kernel and also bugs in each new function can cause a kernel compromise. Debugging these functions is hard because the code is seldom used, and kgdb is suboptimal. The current initrd-based solution is problematic because it relies on black magic to migrate kernel threads from the initial root to the new root (`pivot_root` syscall). The initramfs solves this in a more elegant fashion. Right now there are three boot cases: * Live CD * the installer * regular system boot. The first two are unified: the Live CD and the Installer. Actual system bootup is hardcoded, which occasionally results in a system that is installed but cannot boot. Hotplug-ng is the hotplug of choice because it is written by the author of udev and klibc for the purposes of early userspace. |
|
| Line 42: | Line 57: |
| * Perform sufficient hardware detection in early userspace to support root filesystems on disk, USB or network * Transparently locate and mount the proper root filesystem, even if it is found on a different logical device than at the time of kernel configuration * Be easier to debug than existing early userspace * Fall back gracefully to precalculated process (a la mkinitrd) with a warning if the process fails. Keep breakage to a minimum; booting is important! |
* Jim wants NFSRoot (and LTSP) for his thin client terminals and would prefer if the initramfs was the same across all the machines. * Matt wants to boot his machine. * Jeff wants to be able to boot with a root file system on a firewire or USB thumbdrive when his hard drive has crashed in order to rescue his data. * Fabio wants to test changes to the early boot sequence because he is a developer of said system. * Mark frequently changes hardware in his machine. He wants the system to Just Work, even when he has made major changes to the system such as moving a hard drive from one system to another. Technical use cases: * root on plain partition (on whatever physical medium) [done] * root on lvm volume * root on software raidN * root on evms * root on lvm/raid combination * root on nfs [done] * root on iscsi * root on hardware raid |
| Line 50: | Line 87: |
| Breezy Targets: * Update klibc and udev to recent version from current packages (http://people.ubuntu.com/~jbailey/) and upload to Breezy. * Create archives of kernel modules for inclusion in initramfs * Tweak kernel packaging to help mkinitramfs. `kernel-package` needs to grow a feature where you can specify which mkinitrd/mkramfs program to call. * Add ability to concatenate CPIO files to grub version 1. This will make it possible to use multiple CPIO archives on i386/amd64. Note that mkinitramfs will still be able to generate "legacy" (monolithic) initramfses and this would probably be the default until bootloaders generally support this case. * Package `hotplug-ng` and add to initramfs * Expand `initramfs-tools` with various hooks for evms and lvm Post-Breezy Targets: * User space partition detection and partition label/UUID handling. |
|
| Line 52: | Line 107: |
| This process unifies the boot process with that of the installer. Unfortunately, systems that were customized after mkinitrd was originally installed may find their system no longer bootable due to changes in the boot order. |
|
| Line 54: | Line 111: |
| * `udev` * `hotplug-ng` * `klibc` * `linux-*` * `evms` * `lvm` * `usplash` * `grub` == Modules to include in the initrd broken down by package == === base === * md * raid0 * raid1 * raid5 * raid6 * ehci-hcd * ohci-hcd * uhci-hcd * usbhid * usb-storage * ext2 * ext3 * isofs * nfs * reiserfs * xfs === ethernet === * 3x59x * 8139cp * 8139too * 8390 * b44 * bmac * bnx2 * defxx * dl2k * e1000 * e100 * epic100 * eql * fealnx * famachi * hp100 * mace * mv643xx_eth * natsemi * ne2k-pci * netconsole * ns83820 * pcnet32 * r8169 * s2io * sis900 * skge * slhc * starfire * sundance * sungem * sungem_phy * sunhme * tg3 * tlan * de2104x * de4x5 * dmfe * tulip * winbond-840 * xircom_cb * xircom_tulip_cb * typhon * via-rhine * via-velocity * yellowfin === ide === * ide-cd * ide-disk * aec62xx * cmd64x * generic * hpt34x * hpt366 * ns87415 * pdc202xx_new * pdc202xx_old * sc1200 * siimage * slc82c105 * trm290 * via82cxxx === scsi === * 3w-9xxx * 3w-xxxx * a100u2x * aacraid * ahci * aic79xx * aic7xxx * atp870u * BusLogic * ch * dc395x * dmx3191d * dpt_i2o * eata * fdomain * initio * ipr * ips * lpfc * mac53c94 * megaraid * megaraid_mbox * megaraid_mm * mesh * nsp32 * osst * qla1280 * qla2100 * qla2200 * qla2300 * qla2322 * qla2xxx * qla6312 * qlogicfas408 * qlogicfc * sata_promise * sata_qstor * sata_sil * sata_sis * sata_svw * sata_sx4 * sata_uli * sata_via * sata_vsc * scsi_mod * scsi_transport_fc * scsi_transport_iscsi * scsi_transport_spi * sd_mod * sym53c8xx * tmscsim === uncategorized === * pcmcia_core (for root filesystem on e.g. external USB2 hard drive connected via PCMCIA USB2 adapter) * yenta_socket (for root filesystem on e.g. external USB2 hard drive connected via PCMCIA USB2 adapter) |
|
| Line 56: | Line 397: |
| Should be able to drop into rescue shell on error, or by choice. Must be usplash-friendly. On catastrophic failure, enough debug information should be provided to allow an adequate bug report to be filed. |
|
| Line 58: | Line 403: |
| === UDU BOF Agenda === === UDU Pre-Work === |
* grub version 1 is no longer actively developed upstream, so updates will be Ubuntu specific (although Debian will likely accept them). There is a version 2 which is a total rewrite and which may be viable in the Breezy+1 timeframe. * Coordination with the Debian Kernel Team. * Partition Detection is deferred until post-Breezy to ensure that the system is stable and generally functional. Because it's not core functionality, it may be a good candidate for a bounty. ---- CategoryUdu CategorySpec |
Early Userspace
Status
Created: 2005-04-24 by MattZimmerman
Priority: MediumPriority
People: JeffBaileyLead, TollefFogHeenSecond
Contributors: JaneSilber
Interested: AndresSalomon, PaulSladen
Status: BreezyGoal, EditedSpecification, DistroSpec, JeffBaileyQueue, MattZimmermanQueue
Branch:
Malone Bug:
Packages:
Depends:
UduSessions: 1(0)
Introduction
Early UserSpace is defined as the time from when the kernel hands control over to userspace applications until the time that the full system init is called.
The kernel traditionally has included a lot of functionality that can be easily done in userspace. These include:
- DHCP
- NFS Root
- devfs
- Partition detection
- Raid (md) detection
- root filesystem mounting
- Device detection.
Many of these items are called using the prepare_namepace() call in the kernel. When using an initramfs solution, the prepare_namespace() call is skipped, and a normal user binary is called to complete the setup which is responsible for chaining to the full system init.
Rationale
Upstream is moving towards this solution. Some features, such as devfs are scheduled for removal in a very short period of time. Writing kernel code is difficult, since each feature causes bloat that is included in every generic Ubuntu kernel and also bugs in each new function can cause a kernel compromise. Debugging these functions is hard because the code is seldom used, and kgdb is suboptimal.
The current initrd-based solution is problematic because it relies on black magic to migrate kernel threads from the initial root to the new root (pivot_root syscall). The initramfs solves this in a more elegant fashion.
Right now there are three boot cases:
- Live CD
- the installer
- regular system boot.
The first two are unified: the Live CD and the Installer. Actual system bootup is hardcoded, which occasionally results in a system that is installed but cannot boot.
Hotplug-ng is the hotplug of choice because it is written by the author of udev and klibc for the purposes of early userspace.
Scope and Use Cases
- Jim wants NFSRoot (and LTSP) for his thin client terminals and would prefer if the initramfs was the same across all the machines.
- Matt wants to boot his machine.
- Jeff wants to be able to boot with a root file system on a firewire or USB thumbdrive when his hard drive has crashed in order to rescue his data.
- Fabio wants to test changes to the early boot sequence because he is a developer of said system.
- Mark frequently changes hardware in his machine. He wants the system to Just Work, even when he has made major changes to the system such as moving a hard drive from one system to another.
Technical use cases:
- root on plain partition (on whatever physical medium) [done]
- root on lvm volume
- root on software raidN
- root on evms
- root on lvm/raid combination
- root on nfs [done]
- root on iscsi
- root on hardware raid
Implementation Plan
Breezy Targets:
Update klibc and udev to recent version from current packages (http://people.ubuntu.com/~jbailey/) and upload to Breezy.
- Create archives of kernel modules for inclusion in initramfs
Tweak kernel packaging to help mkinitramfs. kernel-package needs to grow a feature where you can specify which mkinitrd/mkramfs program to call.
- Add ability to concatenate CPIO files to grub version 1. This will make it possible to use multiple CPIO archives on i386/amd64. Note that mkinitramfs will still be able to generate "legacy" (monolithic) initramfses and this would probably be the default until bootloaders generally support this case.
Package hotplug-ng and add to initramfs
Expand initramfs-tools with various hooks for evms and lvm
Post-Breezy Targets:
- User space partition detection and partition label/UUID handling.
Data Preservation and Migration
This process unifies the boot process with that of the installer. Unfortunately, systems that were customized after mkinitrd was originally installed may find their system no longer bootable due to changes in the boot order.
Packages Affected
udev
hotplug-ng
klibc
linux-*
evms
lvm
usplash
grub
Modules to include in the initrd broken down by package
base
* md
* raid0
* raid1
* raid5
* raid6
* ehci-hcd
* ohci-hcd
* uhci-hcd
* usbhid
* usb-storage
* ext2
* ext3
* isofs
* nfs
* reiserfs
* xfs
ethernet
* 3x59x
* 8139cp
* 8139too
* 8390
* b44
* bmac
* bnx2
* defxx
* dl2k
* e1000
* e100
* epic100
* eql
* fealnx
* famachi
* hp100
* mace
* mv643xx_eth
* natsemi
* ne2k-pci
* netconsole
* ns83820
* pcnet32
* r8169
* s2io
* sis900
* skge
* slhc
* starfire
* sundance
* sungem
* sungem_phy
* sunhme
* tg3
* tlan
* de2104x
* de4x5
* dmfe
* tulip
* winbond-840
* xircom_cb
* xircom_tulip_cb
* typhon
* via-rhine
* via-velocity
* yellowfin
ide
* ide-cd
* ide-disk
* aec62xx
* cmd64x
* generic
* hpt34x
* hpt366
* ns87415
* pdc202xx_new
* pdc202xx_old
* sc1200
* siimage
* slc82c105
* trm290
* via82cxxx
scsi
* 3w-9xxx
* 3w-xxxx
* a100u2x
* aacraid
* ahci
* aic79xx
* aic7xxx
* atp870u
* BusLogic
* ch
* dc395x
* dmx3191d
* dpt_i2o
* eata
* fdomain
* initio
* ipr
* ips
* lpfc
* mac53c94
* megaraid
* megaraid_mbox
* megaraid_mm
* mesh
* nsp32
* osst
* qla1280
* qla2100
* qla2200
* qla2300
* qla2322
* qla2xxx
* qla6312
* qlogicfas408
* qlogicfc
* sata_promise
* sata_qstor
* sata_sil
* sata_sis
* sata_svw
* sata_sx4
* sata_uli
* sata_via
* sata_vsc
* scsi_mod
* scsi_transport_fc
* scsi_transport_iscsi
* scsi_transport_spi
* sd_mod
* sym53c8xx
* tmscsim
uncategorized
* pcmcia_core (for root filesystem on e.g. external USB2 hard drive connected via PCMCIA USB2 adapter)
* yenta_socket (for root filesystem on e.g. external USB2 hard drive connected via PCMCIA USB2 adapter)
User Interface Requirements
Should be able to drop into rescue shell on error, or by choice. Must be usplash-friendly.
On catastrophic failure, enough debug information should be provided to allow an adequate bug report to be filed.
Outstanding Issues
- grub version 1 is no longer actively developed upstream, so updates will be Ubuntu specific (although Debian will likely accept them). There is a version 2 which is a total rewrite and which may be viable in the Breezy+1 timeframe.
- Coordination with the Debian Kernel Team.
- Partition Detection is deferred until post-Breezy to ensure that the system is stable and generally functional. Because it's not core functionality, it may be a good candidate for a bounty.
UbuntuDownUnder/BOFs/EarlyUserspace (last edited 2008-08-06 16:19:00 by localhost)