Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 8 scenarios |
SUSE Linux Enterprise Server 12 (x86_64) |
*OpenManifestEditorPerfTest#testEditorOpenMF() (vs. R-4.10-201812060815-201901031800) | Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 0.56s is not within [0%, 110'%] of 0.48s - 16.1% [±3.9] |
*OpenManifestEditorPerfTest#testEditorOpenXML() (vs. R-4.10-201812060815-201901031800) | Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 0.9s is not within [0%, 110'%] of 0.74s - 21.7% [±3.7] |
*PDEModelManagerPerfTest#testModelManagerLoad() (vs. R-4.10-201812060815-201901031800) | Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 5.3s is not within [0%, 110'%] of 2.8s - 92.6% [n/a] |
*SchemaLoaderPerfTest#testSchemaAbbreviated() (vs. R-4.10-201812060815-201901031800) | +5.3% [±5.3] |
*SchemaLoaderPerfTest#testSchemaUnabbreviated() (vs. R-4.10-201812060815-201901031800) | +3.9% [±5.8] |
*SchemaTraversePerfTest#testSchemaTraverse() (vs. R-4.10-201812060815-201901031800) | +6.2% [±4.3] |
*TargetPlatformPerfTest#testResolveTargetDefinition() (vs. R-4.10-201812060815-201901031800) | - 9.8% [±2.3] |
*TargetPlatformPerfTest#testSearchModelRegistry() (vs. R-4.10-201812060815-201901031800) | - 6.0% [±4.4] |