FreeBSD 7.0 和其他BSD系统的性能比较
时间：2008-03-18 来源：linux论坛
In May 2007 I ran some benchmarks of Dragonfly 1.8 to evaluateprogress of its SMP implementation, which was the original focus ofthe project when it launched in 2003 and is still widely believed tobe an area in which they had made concrete progress. This was part ofa larger cross-OS multiprocessor performance evaluation comparingimprovements in FreeBSD to Linux, NetBSD and other operating systems.
The 2007 results showed essentially no performance increase frommultiple processors on dragonfly 1.8, in contrast to the performanceof FreeBSD 7.0 which scaled to 8 CPUs on the benchmark.
Recently Dragonfly 1.12 was released, and the question was raised on the dragonfly-users mailing list of how well the OS performs after afurther year of development. I performed several benchmarks to studythis question.
In this round of testing I compared Dragonfly 1.12, FreeBSD 4.11 andFreeBSD 7.0, running on the same 8-core Xeon hardware. On Dragonflythe GENERIC kernel configuration was used except for enabling SMP andAPIC_IO (for the SMP tests), and removing I486_CPU. Under FreeBSD theGENERIC kernel was used except for enabling the SCHED_ULE scheduler on7.0, removing I486_CPU and enabling SMP when appropriate. The testapplications were compiled from ports/pkgsrc and the sameversions and configuration options used for each OS.
MySQLThis is a good general test of kernel performance and parallelism, aswell as performance of the thread library. MySQL performance(together with PostgreSQL performance) has been a driving force inFreeBSD, Linux and NetBSD SMP development over the past year:screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/NmcKuiXFbep9Slt58Ad1BR7LZnk3UHWr.png');}" onmousewheel="return imgzoom(this);" alt="" />MySQL configuration is the same as in my previous test and is alsodocumented here
Here are the results:
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/8glDemvA1cYSbaIEnOiVU2sdPjR4WCxt.png');}" onmousewheel="return imgzoom(this);" alt="" />Dragonfly 1.12 achieves peak SMP performance of only 15% better thanUP performance, and drops to about 50% below UP performance at higherloads. Enabling SMP has a 20% performance overhead on this benchmark.
UP mode is faster than 4.11 when using the libthread_xu library. Withlibc_r (not graphed) performance is identical to 4.11 in both UP andSMP mode, so the UP performance increase is most likely due to thethread library.
Note: I am using mysql 5.0.51 in the current tests, which hasdifferent performance characteristics than the older 5.0.37 testedlast year, so the current data cannot directly be compared to theprevious dragonfly 1.8 graphs to evaluate whether a small amount ofprogress was made since 1.8. However, there does not appear to be anysignificant performance improvement from dragonfly 1.8 to 1.12.
FreeBSD 7.0 scales to 8 CPUs on this benchmark. Peak performance is6.5 times higher than peak dragonfly performance, and 9.0 times higherthan FreeBSD 4.11 performance. UP performance is consistent with SMPperformance with a single thread. 7.0 UP is 45% faster than 4.11 UPand 10% faster than dragonfly UP.
Note that while these benchmarks are on a test system with 8 CPUcores, the results also provide information about performance onsystems with fewer than 8 cores, such as dual core systems. If thesystem does not show appreciable performance gain when 2 threads areactive and most CPUs are idle, it is unlikely to perform much betterwhen the system only has 2 CPUs. I could not test this directlybecause I don't know how to disable CPUs at boot time/run time indragonfly.
For example, this graph shows FreeBSD 7.0 running postgresql on thesame system with 1, 2, 4 or 8 CPU cores active, as well as comparingthe UP and SMP kernel running with 1 CPU active
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/fbCKu3UIdz2T7rHPGlOk8F9phRWc1MeS.png');}" onmousewheel="return imgzoom(this);" alt="" />The performance seen with 8 CPUs also scales down to 1, 2 and 4 CPUs.This also shows that there is negligible overhead from running theFreeBSD 7.0 SMP kernel on a UP system on this workload.
Filesystem I/OCurrently the major focus of the dragonfly project is the developmentof a new filesystem, so it's interesting to see how well the dragonflyfilesystem layer performs. I created a 500MB memory filesystem (MFS)and used sysbench to perform multi-threaded random write I/O. Itseems that MFS cannot create file systems larger than 500MB, which wasa limiting factor on dragonfly and 4.11.
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/xCI53grmTKPaEo9sMUfehw60NptQWByl.png');}" onmousewheel="return imgzoom(this);" alt="" />Dragonfly 1.12 UP performance is about 30% lower than FreeBSD 4.11 UPperformance. Enabling SMP does not impose an overhead on this test,but there is no performance benefit seen from multiple threads, andinstead performance drops as low as FreeBSD 4.11 SMP performance athigh loads.
FreeBSD 7.0 does not support the MFS file system; the nearestalternative is the tmpfs filesystem which was used for this test.
The sysbench file I/O benchmark apparently has a race condition thatcause it to abort under high I/O load (enabling debugging shows thatsysbench is sometimes generating I/O requests that are out of range ofthe specified test parameters such as file size and number of files,so this appears to be a sysbench bug). This was only a factor on 7.0but the benchmarks were averaged over 5 trials to reduce error.
FreeBSD 7 scales to 3 simultaneous writers and peak SMP performance isa factor of 4 times higher than dragonfly peak performance and 2.6times higher than freebsd 4 performance. FreeBSD 7 peak performancemay be limited by memory bandwidth rather than kernel scalinglimitations, although those come into play at higher loads.
FreeBSD 7.0 has a 20% overhead from SMP on this test compared to UP,which may be because the lockmgr primitive has not yet been optimizedfor SMP mode (this work is in progress). However, SMP performanceexceeds UP with a second writer, and SMP remains 17% faster than UP athigh loads.
UP performance on FreeBSD 7 is 2.6 times higher than dragonfly UPperformance and 1.8 times higher than freebsd 4 UP performance. It isunknown how much of this difference is due to the different design ofthe tmpfs filesystem.
NetworkingNetworking is the major subsystem that has received SMP developmentwork in dragonfly, although it was never completed and still does notallow concurrency in network processing (FreeBSD has a parallelizednetwork stack, and further performance work is ongoing in FreeBSD8.0).
I did not directly test network performance as e.g. a DNS or webserver. I can do this if someone is interested.
NFS performance on this hardware was anomalously low in both 4.11 anddragonfly, averaging only about 300KB/sec on an intel gigabit ethernetNIC. This did not impact the other benchmarks because they were notperforming NFS I/O.
StabilityThe assertion is often made by dragonfly project supporters thatdragonfly has "much better" stability than FreeBSD. It is not clearby what metric this is being objectively evaluated (if at all). Adirect measurement of stability is desirable.
One measure of system stability is the ability to function correctlyunder extreme overload conditions. This tends to provoke raceconditions and other exceptions at a higher frequency than at thelight loads encountered on desktop systems.
To simulate system overload I ran the stress2 benchmark suite on the8-core xeon. This is a suite of test applications that impose massiveoverload on the system under various concurrent workloads. FreeBSD isable to run this test suite indefinitely without errors.
The first problem was encountered while trying to unpack the stress2archive to NFS:
# ls
stress2.tgz
# tar xvf stress2.tgz
tar: Error opening archive: Failed to open 'stress2.tgz': No such file or directory
# ls -l
ls: stress2.tgz: No such file or directory
total 0
This looks like it might be a bug with the dragonfly name cache.
After starting the stress suit, the system panicked in under 4minutes. Unfortunately I was unable to obtain details of the panicbecause the serial console was not working.
Obviously one panic does not demonstrate wide-ranging systeminstability, but it does point to a possible selection bias amongstthe project supporters, who may not be looking hard enough for thestability problems that exist.
SummaryAs with the dragonfly 1.8 kernel, the dragonfly 1.12 kernel does notscale to a second CPU on the benchmarks performed, and the limited SMPimplementation can cause a large performance loss at higher loads.There is sometimes a large performance overhead from enabling SMPcompared to UP, and performance was sometimes worse than that of 4.11.In all cases measured, FreeBSD 7.0 performs significantly better thanboth FreeBSD 4.11 and dragonfly 1.12 in both SMP and UPconfigurations.
The 2007 results showed essentially no performance increase frommultiple processors on dragonfly 1.8, in contrast to the performanceof FreeBSD 7.0 which scaled to 8 CPUs on the benchmark.
Recently Dragonfly 1.12 was released, and the question was raised on the dragonfly-users mailing list of how well the OS performs after afurther year of development. I performed several benchmarks to studythis question.
In this round of testing I compared Dragonfly 1.12, FreeBSD 4.11 andFreeBSD 7.0, running on the same 8-core Xeon hardware. On Dragonflythe GENERIC kernel configuration was used except for enabling SMP andAPIC_IO (for the SMP tests), and removing I486_CPU. Under FreeBSD theGENERIC kernel was used except for enabling the SCHED_ULE scheduler on7.0, removing I486_CPU and enabling SMP when appropriate. The testapplications were compiled from ports/pkgsrc and the sameversions and configuration options used for each OS.
MySQLThis is a good general test of kernel performance and parallelism, aswell as performance of the thread library. MySQL performance(together with PostgreSQL performance) has been a driving force inFreeBSD, Linux and NetBSD SMP development over the past year:screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/NmcKuiXFbep9Slt58Ad1BR7LZnk3UHWr.png');}" onmousewheel="return imgzoom(this);" alt="" />MySQL configuration is the same as in my previous test and is alsodocumented here
Here are the results:
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/8glDemvA1cYSbaIEnOiVU2sdPjR4WCxt.png');}" onmousewheel="return imgzoom(this);" alt="" />Dragonfly 1.12 achieves peak SMP performance of only 15% better thanUP performance, and drops to about 50% below UP performance at higherloads. Enabling SMP has a 20% performance overhead on this benchmark.
UP mode is faster than 4.11 when using the libthread_xu library. Withlibc_r (not graphed) performance is identical to 4.11 in both UP andSMP mode, so the UP performance increase is most likely due to thethread library.
Note: I am using mysql 5.0.51 in the current tests, which hasdifferent performance characteristics than the older 5.0.37 testedlast year, so the current data cannot directly be compared to theprevious dragonfly 1.8 graphs to evaluate whether a small amount ofprogress was made since 1.8. However, there does not appear to be anysignificant performance improvement from dragonfly 1.8 to 1.12.
FreeBSD 7.0 scales to 8 CPUs on this benchmark. Peak performance is6.5 times higher than peak dragonfly performance, and 9.0 times higherthan FreeBSD 4.11 performance. UP performance is consistent with SMPperformance with a single thread. 7.0 UP is 45% faster than 4.11 UPand 10% faster than dragonfly UP.
Note that while these benchmarks are on a test system with 8 CPUcores, the results also provide information about performance onsystems with fewer than 8 cores, such as dual core systems. If thesystem does not show appreciable performance gain when 2 threads areactive and most CPUs are idle, it is unlikely to perform much betterwhen the system only has 2 CPUs. I could not test this directlybecause I don't know how to disable CPUs at boot time/run time indragonfly.
For example, this graph shows FreeBSD 7.0 running postgresql on thesame system with 1, 2, 4 or 8 CPU cores active, as well as comparingthe UP and SMP kernel running with 1 CPU active
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/fbCKu3UIdz2T7rHPGlOk8F9phRWc1MeS.png');}" onmousewheel="return imgzoom(this);" alt="" />The performance seen with 8 CPUs also scales down to 1, 2 and 4 CPUs.This also shows that there is negligible overhead from running theFreeBSD 7.0 SMP kernel on a UP system on this workload.
Filesystem I/OCurrently the major focus of the dragonfly project is the developmentof a new filesystem, so it's interesting to see how well the dragonflyfilesystem layer performs. I created a 500MB memory filesystem (MFS)and used sysbench to perform multi-threaded random write I/O. Itseems that MFS cannot create file systems larger than 500MB, which wasa limiting factor on dragonfly and 4.11.
screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onmouseover="if(this.width>screen.width*0.7) {this.resized=true; this.width=screen.width*0.7; this.style.cursor='hand'; this.alt='Click here to open new window\nCTRL+Mouse wheel to zoom in/out';}" onclick="if(!this.resized) {return true;} else {window.open('http://images.lampchina.net/news/10/xCI53grmTKPaEo9sMUfehw60NptQWByl.png');}" onmousewheel="return imgzoom(this);" alt="" />Dragonfly 1.12 UP performance is about 30% lower than FreeBSD 4.11 UPperformance. Enabling SMP does not impose an overhead on this test,but there is no performance benefit seen from multiple threads, andinstead performance drops as low as FreeBSD 4.11 SMP performance athigh loads.
FreeBSD 7.0 does not support the MFS file system; the nearestalternative is the tmpfs filesystem which was used for this test.
The sysbench file I/O benchmark apparently has a race condition thatcause it to abort under high I/O load (enabling debugging shows thatsysbench is sometimes generating I/O requests that are out of range ofthe specified test parameters such as file size and number of files,so this appears to be a sysbench bug). This was only a factor on 7.0but the benchmarks were averaged over 5 trials to reduce error.
FreeBSD 7 scales to 3 simultaneous writers and peak SMP performance isa factor of 4 times higher than dragonfly peak performance and 2.6times higher than freebsd 4 performance. FreeBSD 7 peak performancemay be limited by memory bandwidth rather than kernel scalinglimitations, although those come into play at higher loads.
FreeBSD 7.0 has a 20% overhead from SMP on this test compared to UP,which may be because the lockmgr primitive has not yet been optimizedfor SMP mode (this work is in progress). However, SMP performanceexceeds UP with a second writer, and SMP remains 17% faster than UP athigh loads.
UP performance on FreeBSD 7 is 2.6 times higher than dragonfly UPperformance and 1.8 times higher than freebsd 4 UP performance. It isunknown how much of this difference is due to the different design ofthe tmpfs filesystem.
NetworkingNetworking is the major subsystem that has received SMP developmentwork in dragonfly, although it was never completed and still does notallow concurrency in network processing (FreeBSD has a parallelizednetwork stack, and further performance work is ongoing in FreeBSD8.0).
I did not directly test network performance as e.g. a DNS or webserver. I can do this if someone is interested.
NFS performance on this hardware was anomalously low in both 4.11 anddragonfly, averaging only about 300KB/sec on an intel gigabit ethernetNIC. This did not impact the other benchmarks because they were notperforming NFS I/O.
StabilityThe assertion is often made by dragonfly project supporters thatdragonfly has "much better" stability than FreeBSD. It is not clearby what metric this is being objectively evaluated (if at all). Adirect measurement of stability is desirable.
One measure of system stability is the ability to function correctlyunder extreme overload conditions. This tends to provoke raceconditions and other exceptions at a higher frequency than at thelight loads encountered on desktop systems.
To simulate system overload I ran the stress2 benchmark suite on the8-core xeon. This is a suite of test applications that impose massiveoverload on the system under various concurrent workloads. FreeBSD isable to run this test suite indefinitely without errors.
The first problem was encountered while trying to unpack the stress2archive to NFS:
# ls
stress2.tgz
# tar xvf stress2.tgz
tar: Error opening archive: Failed to open 'stress2.tgz': No such file or directory
# ls -l
ls: stress2.tgz: No such file or directory
total 0
This looks like it might be a bug with the dragonfly name cache.
After starting the stress suit, the system panicked in under 4minutes. Unfortunately I was unable to obtain details of the panicbecause the serial console was not working.
Obviously one panic does not demonstrate wide-ranging systeminstability, but it does point to a possible selection bias amongstthe project supporters, who may not be looking hard enough for thestability problems that exist.
SummaryAs with the dragonfly 1.8 kernel, the dragonfly 1.12 kernel does notscale to a second CPU on the benchmarks performed, and the limited SMPimplementation can cause a large performance loss at higher loads.There is sometimes a large performance overhead from enabling SMPcompared to UP, and performance was sometimes worse than that of 4.11.In all cases measured, FreeBSD 7.0 performs significantly better thanboth FreeBSD 4.11 and dragonfly 1.12 in both SMP and UPconfigurations.
相关阅读 更多 +
