Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 10 scenarios |
SLED 10 Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) | RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) |
*AnnotationIncrementalBuildTests#AnnotationIncrementalBuildTests with External Dependency check() (vs. N20101204-2000) | - 1.1% [±3.8] | +4.4% [±3.6] | +11.5% [±5.9] | +0.0% [±6.4] |
AnnotationIncrementalBuildTests#testIncrementalAnnot() | - 2.1% [±3.1] | +4.4% [±3.3] | - 4.2% [±7.8] | - 9.2% [±6.7] |
ApiDescriptionTests#testCleanVisit() | Performance criteria not met when compared to 'R-3.6-201006080911_201110071800': Elapsed Process= 1.5s is not within [0%, 110'%] of 1.4s - 11.8% [±11.1] | - 2.1% [±7.5] | Performance criteria not met when compared to 'R-3.6-201006080911_201107151800': Elapsed Process= 3.5s is not within [0%, 110'%] of 1.3s - 171.6% [±4.2] | Performance criteria not met when compared to 'R-3.6-201006080911_201110071800': Elapsed Process= 3.5s is not within [0%, 110'%] of 1.3s - 171.0% [±5.0] |
*EnumIncrementalBuildTests#EnumIncrementalBuildTests with External Dependency check() (vs. N20101204-2000) | - 1.5% [±3.5] | +1.0% [±3.1] | +3.5% [±6.1] | - 7.6% [±5.7] |
EnumIncrementalBuildTests#testIncremantalEnum() | - 2.0% [±3.1] | - 2.5% [±2.9] | Performance criteria not met when compared to 'R-3.6-201006080911_201107151800': Elapsed Process= 0.12s is not within [0%, 110'%] of 0.1s - 11.7% [±6.6] | - 3.7% [±5.8] |
*FullSourceBuildTests#FullSourceBuildTests with External Dependency check() (vs. N20101204-2000) | - 6.8% [±0.2] | - 8.6% [±0.2] | +70.8% [±0.1] | +3.3% [±0.1] |
FullSourceBuildTests#testCleanFullBuild() | - 2.6% [±1.6] | - 2.6% [±1.4] | - 2.5% [±0.9] | - 2.6% [±1.1] |
FullSourceBuildTests#testFullBuild() | - 2.0% [±2.0] | - 2.0% [±1.5] | - 2.6% [±1.0] | - 2.3% [±1.3] |
*IncrementalBuildTests#IncrementalBuildTests with External Dependency checks() (vs. N20101204-2000) | - 5.6% [±2.2] | - 4.0% [±2.4] | +3.7% [±3.9] | - 5.9% [±3.4] |
IncrementalBuildTests#testIncrementalBuildAll() | - 9.7% [±2.1] | +2.7% [±2.1] | +2.2% [±3.2] | +0.9% [±3.3] |