Select which kind of scale you want to use: |
NaN
) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time
)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time
)All 17 scenarios |
SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM) | RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM) | Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM) |
SyncTests#testSync100NoUI()AddedFiles | - 9.1% [±3.6] | +34.5% [±11.2] | +11.0% [±6.0] |
SyncTests#testSync100NoUI()ModifiedFiles | +20.0% [±4.1] | +46.8% [±11.4] | +12.6% [±7.8] |
SyncTests#testSync100NoUI()RemovedFiles | +14.9% [±5.5] | +25.0% [±6.7] | +4.3% [±7.7] |
WorkflowTests#testBigWorkflowNoUI()Checkout | +21.3% [±8.2] | +18.4% [±9.9] | +29.4% [±11.4] |
WorkflowTests#testBigWorkflowNoUI()Commit1 | +18.3% [±19.3] | +19.0% [±19.0] | +4.9% [±9.5] |
WorkflowTests#testBigWorkflowNoUI()Commit2 | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 2.2s is not within [0%, 110'%] of 1.9s - 17.2% [±13.2] | +1.0% [±9.1] | - 3.3% [±6.2] |
WorkflowTests#testBigWorkflowNoUI()Commit3 | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 2.1s is not within [0%, 110'%] of 1.8s - 15.4% [±14.0] | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 2.1s is not within [0%, 110'%] of 1.7s - 24.1% [±10.5] | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 3s is not within [0%, 110'%] of 2.4s - 22.0% [±10.1] |
WorkflowTests#testBigWorkflowNoUI()Commit4 | - 2.5% [±9.2] | - 6.2% [±9.5] | +10.9% [±8.1] |
WorkflowTests#testBigWorkflowNoUI()Replace1 | +73.4% [±16.2] | +60.6% [±22.5] | +32.9% [±22.1] |
WorkflowTests#testBigWorkflowNoUI()Replace2 | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 6.3s is not within [0%, 110'%] of 3.8s - 66.9% [±15.9] | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 6s is not within [0%, 110'%] of 5.1s - 19.2% [±10.5] | Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 6s is not within [0%, 110'%] of 5.1s - 19.3% [±13.1] |
WorkflowTests#testBigWorkflowNoUI()Replace3 | +40.0% [±21.3] | +32.1% [±14.9] | +30.5% [±12.3] |
WorkflowTests#testBigWorkflowNoUI()Share | +12.5% [±10.7] | +23.6% [±11.9] | +42.6% [±12.8] |
WorkflowTests#testBigWorkflowNoUI()Tag1 | +53.7% [±22.2] | +47.4% [±21.8] | +42.4% [±19.2] |
WorkflowTests#testBigWorkflowNoUI()Update1 | +15.5% [±8.7] | +21.6% [±15.5] | +6.1% [±5.2] |
WorkflowTests#testBigWorkflowNoUI()Update2 | +34.5% [±13.3] | +22.6% [±8.5] | +6.0% [±5.3] |
WorkflowTests#testBigWorkflowNoUI()Update3 | +39.6% [±16.1] | +15.0% [±8.5] | +11.9% [±5.5] |
WorkflowTests#testBigWorkflowNoUI()Update4 | +21.4% [±7.3] | +7.9% [±9.4] | +12.7% [±4.2] |