Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 11 scenarios |
SUSE Linux Enterprise Server 12 (x86_64) |
*AnnotationIncrementalBuildTests#AnnotationIncrementalBuildTests with External Dependency check() (vs. I20181001-0920) | - 9.9% [±0.0] |
*AnnotationIncrementalBuildTests#testIncrementalAnnot() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 0.18s is not within [0%, 110'%] of 0.15s - 18.1% [±0.0] |
*ApiDescriptionTests#testCleanVisit() (vs. I20181001-0920) | +34.8% [±1.1] |
*EnumIncrementalBuildTests#EnumIncrementalBuildTests with External Dependency check() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 0.21s is not within [0%, 110'%] of 0.18s - 20.9% [±0.0] |
*EnumIncrementalBuildTests#testIncremantalEnum() (vs. I20181001-0920) | - 0.8% [±0.0] |
*FullSourceBuildTests#FullSourceBuildTests with External Dependency check() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 4.1s is not within [0%, 110'%] of 3.4s - 22.7% [±0.3] |
*FullSourceBuildTests#testCleanFullBuild() (vs. I20181001-0920) | +7.3% [±0.4] |
*FullSourceBuildTests#testFullBuild() (vs. I20181001-0920) | +11.2% [±0.4] |
*IncrementalBuildTests#IncrementalBuildTests with External Dependency checks() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 0.42s is not within [0%, 110'%] of 0.37s - 14.5% [±0.0] |
*IncrementalBuildTests#testIncrementalBuildAll() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 0.49s is not within [0%, 110'%] of 0.44s - 11.4% [±0.0] |
*UseScanTests#testUseScan() (vs. I20181001-0920) | Performance criteria not met when compared to 'I20181001-0920': Elapsed Process= 57.8s is not within [0%, 110'%] of 19.6s - 194.9% [±7.9] |