Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 8 scenarios |
SUSE Linux Enterprise Server 12 (x86_64) |
*BreakpointManagerPerfTests#testStartup100Breakpoints() (vs. R-4.10-201812060815-201901031800) | +0.0% [±1.6] |
*BreakpointManagerPerfTests#testStartup200Breakpoints() (vs. R-4.10-201812060815-201901031800) | Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 2ms is not within [0%, 110'%] of 2ms - 25.0% [±1.0] |
*BreakpointManagerPerfTests#testStartup50Breakpoints() (vs. R-4.10-201812060815-201901031800) | +50.0% [±2.7] |
*PerfConsoleTests#testProcessConsolePlainOutput10000Lines() (vs. R-4.10-201812060815-201901031800) | Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 1.3s is not within [0%, 110'%] of 1.2s - 11.0% [±13.7] |
*PerfConsoleTests#testProcessConsoleStackTraceOutput10000Lines() (vs. R-4.10-201812060815-201901031800) | +10.4% [±12.5] |
*PerfConsoleTests#testProcessConsoleWrappedOutput10000Lines() (vs. R-4.10-201812060815-201901031800) | - 4.1% [±9.3] |
*PerfContextualLaunchMenu#testContextualLaunchMenu() (vs. R-4.10-201812060815-201901031800) | +17.1% [±3.0] |
*PerfVariableDetailTests#testToStringDetails() (vs. R-4.10-201812060815-201901031800) | +2.8% [±3.6] |