Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 7 scenarios |
SLED 10 Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) | RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.6.0_17 (2 x 3.00GHz - 3GB RAM) |
OpenManifestEditorPerfTest#testEditorOpenMF() | Performance criteria not met when compared to 'R-3.6-201006080911_201110071800': Elapsed Process= 0.22s is not within [0%, 110'%] of 0.2s - 10.3% [±1.4] | Performance criteria not met when compared to 'R-3.6-201006080911_201110071800': Elapsed Process= 0.22s is not within [0%, 110'%] of 0.2s - 11.9% [±1.4] | Performance criteria not met when compared to 'R-3.6-201006080911_201107151800': Elapsed Process= 0.16s is not within [0%, 110'%] of 0.14s - 11.9% [±1.6] | +5.7% [±3.7] |
OpenManifestEditorPerfTest#testEditorOpenXML() | - 1.6% [±1.0] | - 1.9% [±0.9] | - 1.9% [±0.9] | +6.9% [±1.9] |
SchemaLoaderPerfTest#testSchemaAbbreviated() | +1.2% [±0.2] | +1.2% [±0.2] | +1.6% [±0.1] | +1.6% [±0.1] |
SchemaLoaderPerfTest#testSchemaUnabbreviated() | - 8.2% [±0.1] | - 8.7% [±0.1] | Performance criteria not met when compared to 'R-3.6-201006080911_201107151800': Elapsed Process= 0.25s is not within [0%, 110'%] of 0.22s - 11.8% [±0.2] | Performance criteria not met when compared to 'R-3.6-201006080911_201110071800': Elapsed Process= 0.24s is not within [0%, 110'%] of 0.22s - 13.4% [±0.2] |
SchemaTraversePerfTest#testSchemaTraverse() | +0.0% [±0.2] | +2.5% [±0.2] | +1.6% [±0.2] | +1.6% [±0.2] |
*TargetPlatformPerfTest#testResolveTargetDefinition() (vs. M20111019-0800) | +0.9% [±1.2] | +0.7% [±1.1] | - 1.2% [±1.0] | - 1.8% [±1.1] |
*TargetPlatformPerfTest#testSearchModelRegistry() (vs. M20111019-0800) | +3.1% [±0.4] | +0.7% [±0.4] | +1.8% [±0.3] | - 1.2% [±0.4] |