): mark a successful result, which means this test has neither significant performance regression nor significant standard error
): mark a failing result, which means this test shows a significant performance regression (more than 10%)
): mark a failing result (see above) with a comment explaining this degradation.
or
): mark a failing or successful result with a significant standard error (more than 3%)
): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)(build_test_time - baseline_test_time) / baseline_test_time)sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)All 8 scenarios |
wtpbuild |
| BasicPluginTests#testInitializePlugin() |
+0.0% [n/a]
|
| FormatTest#testFormat() |
+56.8% [±80.3]
|
| FormatTestWithReconcilerOnly#testFormat() |
+61.3% [±78.8]
|
| OpenEditorTest#testOpenEditor() |
Performance criteria not met when compared to '{config=wtpbuild, build=R20070502-1353, jvm=ibm1.5.0}':
Elapsed Process: 1.15s is not within [0%, 110%] of 964ms
- 19.3% [±9.7]
|
| OpenEditorWithFoldingTest#testOpenEditorLarge() |
+37.2% [±10.5]
|
| OpenEditorWithFoldingTest#testOpenEditorMedium() |
+16.7% [±25.2]
|
| OpenEditorWithFoldingTest#testOpenEditorSmall() |
Performance criteria not met when compared to '{config=wtpbuild, build=R20070502-1353, jvm=ibm1.5.0}':
Elapsed Process: 578ms is not within [0%, 110%] of 381ms
- 51.7% [±18.9]
|
| ScrollTest#testScrolling() |
- 4.3% [n/a]
|