Comparing Linux 2.4 and Linux 2.6 Kernels:
Many customers ask for support of Linux Kernel version 2.6 for their embedded projects. This is easy enough to understand: after all, 2.6 is the latest and greatest version of the Linux kernel, and who wants to base a new project on old stuff? They are surprised when we tell them that we don't recommend to use Linux 2.6 for most embedded systems.
Here is a comparison of 2.4 and 2.6 on real embedded hardware that explains why:
Build and Test Environment, Kernel Versions and Test Hardware:
The ELDK 3.1 toolchain was used to build the Linux kernels and to provide the necessary test environment (NFS root filesystem, ramdisk images).
The following versions of the Linux Kernel were used in the tests:
The following hardware was used for the tests:
- Sandpoint 8240 (MPC8240 at 132 MHz CPU clock, 64 MB RAM)
- TQM860L (MPC860-T at 50 MHz CPU clock, 16 MB RAM)
Building the Kernels:
2.4.25:
2.6.11.7:
Results:
|
Sandpoint
|
2.4.25
|
2.6.11.7
|
Delta
|
|
real
|
77.7s
|
101.6s
|
+ 31 %
|
|
user
|
268.2s
|
348.6s
|
+ 30 %
|
|
sys
|
28.3s
|
39.4s
|
+ 39 %
|
|
Compressed Kernel
|
887.9kB
|
1123.5kB
|
+ 27 %
|
|
Uncompressed kernel
|
2086.4kB
|
2513.6kB
|
+ 20 %
|
|
TQM860L
|
2.4.25
|
2.6.11.7
|
Delta
|
|
real
|
50.3s
|
72.2s
|
+ 43 %
|
|
user
|
175.3s
|
251.5s
|
+ 43 %
|
|
sys
|
19.6s
|
29.4s
|
+ 50 %
|
|
Compressed Kernel
|
495.8kB
|
706.8kB
|
+ 43 %
|
|
Uncompressed kernel
|
1189.6kB
|
1602.8kB
|
+ 35 %
|
Booting Linux:
The kernel images were booted in the same environment with root filesystem over NFS in an idle network. The timings were recorded in the following log files:
Results:
|
Time from begin of loading the kernel image into RAM ...
|
2.4.25
|
2.6.11.7
|
Delta
|
|
... until "Linux version" message
|
|
TQM860L
|
1.9s
|
2.0s
|
+ 6 %
|
|
Sandpoint
|
13.0s
|
15.0s
|
+ 15 %
|
|
... until "Freeing unused kernel memory" message (= enter user space)
|
|
TQM860L
|
3.7s
|
3.8s
|
+ 1 %
|
|
Sandpoint
|
23.0s
|
22.0s
|
- 4 %
|
|
... until "login:" message (= full multi-user mode)
|
|
TQM860L
|
49.7s
|
58.0s
|
+ 17 %
|
|
Sandpoint
|
56.2s
|
59.6s
|
+ 6 %
|
The slight advantage for the 2.6 when booting is at least partially due to the fact that the configurations were slightly different: in 2.4, the SCSI driver was enabled, which was not the case in 2.6
Running with ramdisk based root file system:
The systems were booted using the ramdisk images provided by the ELDK. The main purpose was to find out how much memory remained free for application code. A simple memory allocator (mem_eat.c) was used to test for free (= allocateable by an application process) memory.
2.4.25:
- Sandpoint Board:
- TQM860L Board:
... Memory: 13396k available (892k kernel code, 252k data, 52k init, 0k highmem) ... # free total used free shared buffers Mem: 15032 7112 7920 0 108 Swap: 0 0 0 Total: 15032 7112 7920 # ./mem_eat ... Got total = 9240576 = 9024kB __alloc_pages: 0-order allocation failed (gfp=0x1d2/0) VM: killing process mem_eat
2.6.11.7:
- Sandpoint Board:
- TQM860L Board:
... Memory: 12980k available (1240k kernel code, 288k data, 88k init, 0k highmem) ... # free total used free shared buffers Mem: 14676 6572 8104 0 4092 Swap: 0 0 0 Total: 14676 6572 8104 # ./mem_eat ... Got total = 8519680 = 8320kB oom-killer: gfp_mask=0xd2 ...
Results:
|
TQM860L
|
2.4.25
|
2.6.11.7
|
Delta
|
|
Kernel Code + Data
|
1144kB
|
1528kB
|
+ 34 %
|
|
total "free"
|
15032kB
|
14676kB
|
- 2 % = - 356kB
|
|
Available for malloc()
|
9024kB
|
8320kB
|
- 8 % = - 704kB
|
Running the lmbench benchmark:
To get some more extensive performance data the lmbench benchmark (version 3.0-a4) was run on the systems). The results give a pretty clear and somewhat disappointing picture:
- Even in tests that should not depend on the OS environment at all (like basic integer or float operations) the 2.6 kernel 1...3% slower thanthe 2.4 kernel. This is probably casued by the bigger kernel size which, especially in combination with the small caches on embedded processors, results in some loss of performance.
- Context switching which is essential for the efficient operation of a multi-tasking OS is on average 55% slower (range: 10...94%).
- Local communication latencies are significantly slower, too: 80% for pipes, 79% for UNIX domain sockets, 34% for UDP, 12% for RPC over UDP, 25% for TCP, 12% for RPC over TCP, and 25% for TCP connections.
- File system and VM system latencies go up dramatically, too: 76% for file creation, 17% for file deletion, 32% for the mmap() latency, 6% for page faults and 2% for select() on 100 file descriptors.
- Local communication bandwidth suffers hard, too, but the results differ depending on the processor used:
- Pipes become slightly (3%) faster on the MPC860, but reach only 49% on the MPC8240
- UNIX domain sockets are 11 slower on the MPC8240, but same speed on MPC860
- TCP is 2% faster on the MPC8240, but 31% slower on the MPC860
- File re-reads drop by 6%
Note that some tests did not run under 2.6 on the MPC860 system because the memory was exhausted (the test system had only 16 MB RAM).
Management Summary:
Using the 2.6 kernel on embedded systems implicates the following disadvantages:
- Slow to build: 2.6 takes 30...40% longer to compile
- Big memory footprint in flash: the 2.6 compressed kernel image is 30...40% bigger
- Big memory footprint in RAM: the 2.6 kernel needs 30...40% more RAM; the available RAM size for applications is 700kB smaller
- Slow to boot: 2.6 takes 5...15% longer to boot into multi-user mode
- Slow to run: context switches up to 96% slower, local communication latencies up to 80% slower, file system latencies up to 76% slower, local communication bandwidth less than 50% in some cases.
There may be a few cases where the use of Linux kernel version 2.6 makes sense even for embedded systems (typically on "bigger" systems with more powerful processors), but if memory footprint or system performance are important you probably want to stick with a 2.4 kernel for now.
-- WolfgangDenk - 24 Apr 2005
