| Select which kind of scale you want to use: |
|
): mark a successful result, which means this test has neither significant performance regression nor significant standard error
): mark a failing result, which means this test shows a significant performance regression (more than 10%)
): mark a failing result (see above) with a comment explaining this degradation.
or 
): mark a failing or successful result with a significant standard error (more than 3%)
): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)All 5 scenarios |
RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB) | RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB) | RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB) | Win XP Sun 1.4.2_10 (2 GHz 512 MB) | Win XP Sun 1.4.2_10 (3 GHz 2 GB) |
| OpenCheatSheetTest#testOpenCompositeCheatSheet() |
+1.6% [±0.6]
|
+9.5% [±1.8]
|
+3.8% [±0.5]
| n/a |
- 3.0% [±2.3]
|
| OpenCheatSheetTest#testOpenSimpleCheatSheet() |
- 2.4% [±1.2]
|
+14.6% [±8.0]
|
- 7.0% [±1.1]
| n/a |
- 1.9% [±2.9]
|
| BuildHtmlSearchIndex#testHtmlSearchIndexCreate() |
- 2.2% [±2.2]
|
- 5.0% [±1.8]
|
- 6.2% [±1.5]
| n/a |
- 5.8% [±1.2]
|
| IndexAssemblePerformanceTest#testIndexAssemble() |
+12.4% [±21.5]
|
+5.6% [±4.3]
|
+4.4% [±4.5]
| n/a |
+4.1% [±4.3]
|
| TocAssemblePerformanceTest#testTocAssemble() |
- 2.6% [±0.8]
|
- 0.9% [±0.9]
|
- 1.7% [±0.6]
| n/a |
- 4.3% [±1.0]
|