Originally all hardware drivers in Linux were a part of the kernel. This meant for any hardware, the user had to "roll their own" kernel to work with their hardware. The kernel module changed all that. Now we can ship modules for all known hardware and load what we need... even have most of it auto detected. Many modules are hardware drivers, but there are also a lot of higher level modules that are more general too. For example, a wireless lan interface might have one module that talks directly to the hardware but 3 or 4 other modules that talk to that module.
So what is the problem then?
- Some modules handle hardware that interferes with audio.
- Some modules handle hardware who's irq interferes with the audio interface.
An example of the first case is the ath9k module for the wireless transmitter used in some netbooks. No matter how low the priority of the wireless irq handler, this generates xruns once a minute when it scans the wireless area. Turning the wireless transceiver off makes things worse, not better, with xruns every 5 seconds. The only solution is to accept either very high latency audio or remove this module while using low latency audio. The module can be removed using the modprobe -r modulename command. When audio work is finished, the module can be reloaded again with the modprobe command. When reloading a kernel module, some other software may need to be restarted. It is up to the user to research what. In the case of the wireless module, wireless networking just started working again with no user input. Linux is pretty automated these days so try just reloading first.
In the second case a USB port may use the same irq as the sound interface and no amount of irq prioritizing helps. http://wiki.linuxmusicians.com/doku.php?id=system_configuration#unbinding_devices has a great write up on this.