Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 7 scenarios |
SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM) | RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM) |
*OpenAntEditorTest#testOpenAntEditor1() (vs. R-3.4-200806172000_200911301800) | - 3.5% [±30.8] | Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 0.7s is not within [0%, 110'%] of 0.5s - 41.6% [±35.4] | Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.7s is not within [0%, 110'%] of 0.62s - 12.4% [±38.9] |
*OpenAntEditorTest#testOpenAntEditor2() (vs. R-3.4-200806172000_200911301800) | - 5.7% [±2.1] | Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 0.59s is not within [0%, 110'%] of 0.33s - 77.6% [±10.4] | - 4.8% [±1.1] |
*OpenAntEditorTest#testOpenAntEditorNoFolding() (vs. R-3.4-200806172000_200911301800) | - 4.3% [±1.4] | Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 0.44s is not within [0%, 110'%] of 0.22s - 95.5% [±2.3] | - 5.6% [±1.1] |
*SeparateVMTests#testBuild() (vs. R-3.4-200806172000_200911301800) | - 9.4% [±1.9] | - 4.6% [±1.5] | Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 5.2s is not within [0%, 110'%] of 4.7s - 10.2% [±0.2] |
*SeparateVMTests#testBuildMinusDebug() (vs. R-3.4-200806172000_200911301800) | - 5.7% [±0.9] | - 6.4% [±0.3] | - 9.9% [±0.2] |
*SeparateVMTests#testBuildNoConsole() (vs. R-3.4-200806172000_200911301800) | +0.3% [±0.7] | +0.3% [±0.2] | - 7.0% [±0.1] |
*SeparateVMTests#testBuildWithLotsOfLinks() (vs. R-3.4-200806172000_200911301800) | - 3.4% [±0.8] | - 3.0% [±0.3] | - 7.8% [±0.1] |