irqtune changes the IRQ priority of devices to allow devices that require high priority and fast service (e.g. serial ports, modems) to have it.
With irqtune, a 3X speedup of serial/modem throughput is possible.
You probably need irqtune, if you are experiencing any of the following:
When the PC boots Linux, the timer is given, by default, the highest IRQ priority in the system (it's IRQ 0 and thus, priority 0). On a standard configuration, the serial ports are priority 11 and 12!!! This means that 10 other devices have higher priority.
When multiple devices are in contention to interrupt the CPU, their priority decides which interrupts will occur in what order.
After an arbitrary period of having interrupts disabled (e.g after a cli), at the point where they're reenabled (sti). This can happen in several places:
In the default configuration, the serial ISR will usually lose as it's priority 11.
irqtune gives priority 0 to the whatever device we specify. If we specify a serial device, irqtune guarantees that the serial ISR gets control whenever a contention occurs.
Serial devices are somewhat unique. Even though they have one of the slowest data rates (relative to a disk), they are the largest consumer of interrupts and are extremely sensitive to interrupt latency (the time from when a device yanks the IRQ line until its ISR is executed).
Unlikely doesn't mean never. There are places were the contention period must occur, no matter how we program the CPU.
Variations in CPU speed, RAM size, RAM speed, cache size, disk speed, disk rotational position, number and type of other devices, system workload, etc, etc etc, all contribute to variations of order and timing of internal OS events.
Unlikely on one system may mean 1/1000th % chance. On another system, it may mean happens 50 times/second. Beyond a certain point, it all comes down to measurement.
Tight, accurate, repeatable measurement is the key to system tuning. If we can't measure it, we can't tune it. Once we can measure these things, we can then try various combinations until we achieve our desired results. It can often be pointless to guess at performance.
In fact, the interrupt disable windows themselves are used to prevent data corruption caused by various parts of the kernel that try to update a data structure simultaneously and get corrupted. Many of these windows are put there in the unlikely case that such corruption would occur. In this context, unlikely is never considered a reason to forego interrupt locking. Eliminating a necessary interrupt lock window could result in kernel panics, RAM corruption, disk data corruption, etc.
Not really.
In actual practice, most devices don't even notice the difference. Most other devices (e.g. disks, tape, ethernet) are DMA devices. The DMA does most of the work, thus greatly reducing their need for interrupts. If the device allows a request queue, it may function autonomously on several requests, producing only one interrupt for the entire batch.
Furthermore, serial interrupt services are, themselves, very fast. They slam their data as quickly as possible and get out ASAP. No fancy calculations, just the minimum, mindless data transfer. Almost everything else is handled later, in the bottom-half with interrupts enabled. In fact, a serial ISR may have to re-interrupt it's own bottom-half several times.
Those devices that do experience some slight slowdown are more likely to have long interrupt disable windows themselves. Having several smaller cli/sti windows is much better than one large cli/sti window--It's just harder to program.
Well, actually a prioritized system behaves like a balanced system--most of the time. This occurs when all devices have short interrupt lockout windows and short ISR execution times. The priority mechanism is like a safety valve--it only really matters when some device or combination of devices has held interrupts locked for an extended period of time.
When a disk ISR gets delayed, that's all that happens, a slight delay--disks and tapes are not real-time devices. When a serial ISR gets delayed, data is destroyed--serial devices are real-time devices that dictate the cadence of the entire system.
It may sound cruel, but you just can't be fair.
And speaking of sound, if the sound card gets delayed, we hear an annoying pop right in the middle of our favorite piece of music.
These are real-time devices that can not tolerate excessive delays. If we must defer some less time critical devices to meet the minimum real-time criteria of devices, then that's a bargain.
Ultimately, it's a bit of a compromise. Which is better:
No. It's actually an old idea. I've been doing device drivers since 1977 and Unix kernel work since 1981. I've personally written 8 serial drivers have used this many times commercially. Giving the serial device the highest priority is actually standard practice in many systems. With a 4Mhz CPU, these problems used to occur at 1200 baud :-)
cd /usr/local
gzip -d < irqtune.tgz | tar xvf -
cd /usr/local/irqtune
make install
/sbin/irqtune
/sbin/irqtune_mod.o
/sbin/irqtune_npr.o
As a note to purists, the installation directory is completely arbitrary. /sbin is short, sweet, and easy to type.
The standard convention would be to install irqtune as /sbin/irqtune
and the .o files as, say, /usr/lib/irqtune/*.o.
irqtune uses a slightly different convention. It wants all installed
files to be in the same directory.
This works fine because argv[0] will
point to the .o files. This also allows:
If placing .o files in /sbin is deemed to be an
anathema, we have two options:
make SBIN=/whatever install
make INSTALL=sh install
Yes. If the shell/stub installation method is used:
irqtune will work even if we don't have the kernel source loaded. It uses insmod to load the patch, invoke it, and then unload it. The IRQ priority changes will last so long as the kernel is booted.
irqtune takes two arguments optional arguments:
The default is 3 14 which will work for many standard configurations. More on this later.
usage: irqtune [options] [master] [slave]
version: 0.5
options:
-e -- show verbose errors
-h -- display help
-n -- nogo mode (just show what would happen)
-o -- reset to original values (0/8)
-v -- display irqtune version
-F<file> -- use <file> instead of /proc/interrupts
-q -- suppress priority table printing
-s -- sort table by priority
-u -- force module unload
-x -- show inactive devices in table
arguments:
master -- high priority IRQ on PIC master (DEFAULT: 3)
slave -- high priority IRQ on PIC slave (DEFAULT: 14)
Yes. Just add a /sbin/irqtune line to this file and we're in business. You may also issue another irqtune command at any time.
irqtune is 99.44% kernel revision independent. It is built using ELF binaries, so your insmod must understand them. Also, make sure that you have the correct insmod for your kernel. But that's really about it. There are some slight caveats--these are discussed later.
Well, I'd recommend that you upgrade your kernel as ELF binaries are cool :-)
But if you insist, you'll just have to recompile irqtune. Just be sure
that /usr/src/linux/include is installed. The exact procedure for building
a.out binaries can vary with compiler revision, so be sure to check
your documentation on this (You may need to add a parameter or two).
make warp9
INSTALLATION:
install install prebuilt binaries in /sbin
custom create custom installation -- Normally _not_ required -- Try
install first (EQUIVALENT: kvers sbin install)
uninstall remove prebuilt binaries from /sbin
INSTALLATION OVERRIDES:
SBIN SBIN=/whatever specify installation directory (DEFAULT: /sbin)
INSTALL INSTALL=simple simple installation (DEFAULT: simple)
INSTALL=sh install as shell stub
REBUILDING FROM SOURCE:
kvers create kernel_version from local system (NOTE: this is normally
_not_ required because we use the insmod -f). You can rebuild
_without_ /usr/src/linux/include
sbin rebuild binaries from source (full or partial)
FORCE FULL REBUILD (requires /usr/src/linux/include):
clean remove .o files to force full recompile)
warp9 rebuild all (EQUIVALENT: clean kvers sbin install)
NOTE: For brevity, we've combined the non-sorted and sorted output in these examples.
SORTED BY IRQ: SORTED BY PRIORITY: I00/P00: 8578913 timer I00/P00: 8578913 timer I01/P01: 109547 keyboard I01/P01: 109547 keyboard I02/P02: 0 + cascade I02/P02: 0 + cascade I03/P10: 86470 + serial I11/P05: 197648 + sermux I04/P11: 197648 + serial I12/P06: 17968 + eth I11/P05: 197648 + sermux I13/P07: 1 math error I12/P06: 17968 + eth I14/P08: 93123 + Ux4F I13/P07: 1 math error I03/P10: 86470 + serial I14/P08: 93123 + Ux4F I04/P11: 197648 + serial
NOTE: /proc/interrupts, and therefore irqtune, only reports on active devices. So to scope out the serial IRQ's, ideally, you'd have X Windows up with your serial mouse and be connected via PPP to the net.
The leftmost number is the IRQ number. The next number is the priority. The rightmost column is the internal device name (not to be confused with /dev names). In the above case, the two serial ports are on IRQ 3 and IRQ 4. Just use the lower number, in this case 3:
SORTED BY IRQ: SORTED BY PRIORITY: I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I11/P12: 197648 + sermux I14/P07: 93123 + Ux4F I12/P13: 17968 + eth I11/P12: 197648 + sermux I13/P14: 1 math error I12/P13: 17968 + eth I14/P07: 93123 + Ux4F I13/P14: 1 math error
Glad you asked. There are actually two interrupt controllers, a master and a slave. The slave is cascaded to the master via its IRQ 2. The master controls IRQ's 0-7 and the slave controls IRQ's 8-15.
You actually may select two high IRQ priorities, one for the master and one for the slave. irqtune defaults the slave to IRQ 14, which is normally the disk controller.
In fact, cascade is sort of a "zero width" device as it does not contribute to interrupt latency. Setting the cascade to top priority on the master has an interesting effect which we'll see shortly.
In this case, we might want to use:
SORTED BY IRQ: SORTED BY PRIORITY: I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I11/P14: 197648 + sermux I12/P07: 17968 + eth I12/P07: 17968 + eth I13/P08: 1 math error I13/P08: 1 math error I14/P09: 93123 + Ux4F I14/P09: 93123 + Ux4F I11/P14: 197648 + sermux
This is a bit tricky because now we've got a serial device on the slave controller. It would be much better to put all serial cards on the master controller. Things would stay much simpler.
In this case we would want to use:
SORTED BY IRQ: SORTED BY PRIORITY: I00/P13: 8578913 timer I02/P00: 0 + cascade I01/P14: 109547 keyboard I11/P00: 197648 + sermux I02/P00: 0 + cascade I12/P01: 17968 + eth I03/P08: 86470 + serial I13/P02: 1 math error I04/P09: 197648 + serial I14/P03: 93123 + Ux4F I11/P00: 197648 + sermux I03/P08: 86470 + serial I12/P01: 17968 + eth I04/P09: 197648 + serial I13/P02: 1 math error I00/P13: 8578913 timer I14/P03: 93123 + Ux4F I01/P14: 109547 keyboard
Yes, this is shorthand way of saying 2 11. You can make a slave device top priority, but we get no options for the master IRQ. It will always be 2, the cascade device. Remember, the cascade device contributes no latency delay by itself.
Well, we boosted the priority of the serial multiplexer at the expense of the regular serial ports. The only way to allow all serial ports equally high priority is to group them on consecutive IRQ's and set the high priority for the lowest of those IRQ's.
We can't.
We're limited by the architecture of the PC and its interrupt controllers.
We must change the IRQ of a device by physical restrapping--we can't
do it by reprogramming the priority alone.
We'll go back to the earlier example. We'll wave a magic wand and Poof!--assume we just restrapped the serial multiplexer to IRQ 5:
SORTED BY IRQ: SORTED BY PRIORITY: I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I05/P02: 197648 + sermux I03/P00: 86470 + serial I00/P05: 8578913 timer I04/P01: 197648 + serial I01/P06: 109547 keyboard I05/P02: 197648 + sermux I02/P07: 0 + cascade I12/P13: 17968 + eth I14/P07: 93123 + Ux4F I13/P14: 1 math error I12/P13: 17968 + eth I14/P07: 93123 + Ux4F I13/P14: 1 math error
Sure. Waving the wand again, we restrap the ethernet card to IRQ 6.
SORTED BY IRQ: SORTED BY PRIORITY: I00/P05: 8578913 timer I03/P00: 86470 + serial I01/P06: 109547 keyboard I04/P01: 197648 + serial I02/P07: 0 + cascade I05/P02: 197648 + sermux I03/P00: 86470 + serial I06/P03: 17968 + eth I04/P01: 197648 + serial I00/P05: 8578913 timer I05/P02: 197648 + sermux I01/P06: 109547 keyboard I06/P03: 17968 + eth I02/P07: 0 + cascade I13/P14: 1 math error I14/P07: 93123 + Ux4F I14/P07: 93123 + Ux4F I13/P14: 1 math error
Exactly.
Different systems may have highly different criteria for what is optimum. It is, ultimately, a choice that each system administrator must make based upon the specific requirements for the particular system in question. We can only provide tools to do the job, but the final choice is ultimately decided on a case-by-case basis. There is no one size fits all solution.
Under the 2.0.X kernels, use of IRQ sharing will defeat IRQ priority because the serial port ISR's are installed as slow rather than fast interrupts (e.g. they don't use the SA_INTERRUPT flag).
Change the following line:
#define IRQ_T(info) ((info->flags & ASYNC_SHARE_IRQ) ? SA_SHIRQ : SA_INTERRUPT)and replace it with:
#define IRQ_T(info) (((info->flags & ASYNC_SHARE_IRQ) ? SA_SHIRQ : 0) | SA_INTERRUPT)
Under earlier kernels this is not a problem because the serial ISR was always installed with SA_INTERRUPT.
When irqtune was first released, some experimental changes were made to the kernel to solve the IRQ priority problem by use of a round-robin, balanced priority system that was incompatible with irqtune. These changes were ultimately removed but irqtune will not work with kernel revisions 2.0.15 to 2.0.18.
The only kernel symbol that irqtune's kernel module (irqtune_mod.o) uses is printk (to print a confirmation message to syslog). Some kernels that use MODVERSIONS have some difficulty loading irqtune_mod.o.
The irqtune_npr.o module is exactly the same as irqtune_mod.o except that it does not use printk. irqtune first trys to load irqtune_mod.o and falls back to irqtune_npr.o if it detects a loader error. Since irqtune pre-checks all parameters before attempting to load the kernel module, the confirmation message is a nicety but not a necessity.
See the PPP man page and PPP-Howto for best information, but some recommended
options:
AT-like modems have a special character to escape from data mode to command mode. To avoid confusion here, we'll call this modem escape character a guard character.
The default guard character is '+' (decimal 43, hex 2B). Normally, 3 such characters are required within a special timing sequence.
Although it is unlikely, it is still possible that some PPP packets could
generate the guard sequence inadvertantly. To prevent this, we may want
to inhibit the generation of the guard character in a data sequence. To
do this,
we would add the additional PPP option:
Since '+' is a common ASCII character (PPP escaped characters generate two characters), we may wish to use a less common value for the guard character. For example, a less common value might be (decimal 200, hex C8). We would add an ATS2=200 command to our modem dialer script and change the PPP escape option to C8.
This is only necessary for the IDE driver. The SCSI driver has short disable windows by default. This will shorten the IDE interrupt disable windows.
Beware: Without this option, IDE disk activity will almost certainly cause serial data dropouts.
This reduces the amount of bottom-half processing the system has to do at the expense of larger packets being sent. This may be helpful on slower CPU's or heavily loaded configurations.
Reducing the MRU/MTU to a minimum (296) reduces the bottom-half processing and flip-buffer latency at the expense of adding extra overhead bytes due to the reduced packet size. The optimal value will vary from configuration to configuration.
Beware: Start with 296 as the optimal may not be 1500.
The flip-buffer is a double buffer mechanism in the serial/tty drivers through which all data must pass. It has a fixed size of only 512 bytes. MRU/MTU greater than the flip-buffer size may create an internal race condition that may cause dropouts on slower CPU's or heavily loaded configurations.
Although irqtune will work surprisingly well with just about any kernel revision, the low level IRQ handlers and device drivers have been vastly improved in the 2.0.X kernels. This will only improve irqtune's effect.
Well, first off, if PPP/SLIP was dying mysteriously, it will probably be more reliable.
Repeat this using irqtune and note the transfer times again.
NOTE: IRQTUNE just won't quit--if you want to test in the original mode again, reboot the system first.
It's a matter of probability. Performance measurement is as much art as science.