Performance of org.eclipse.ui: M20080221-1800 relative to 3.2 (200801281721)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 6 scenarios	RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)	RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)	RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)	Win XP Sun 1.4.2_10 (2 GHz 512 MB)	Win XP Sun 1.4.2_10 (3 GHz 2 GB)
EmptyWorkbenchPerfTest#testOpen() [close]	Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 67ms is not within [0%, 110%] of 30ms - 123.3% [±8.1]	Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 48ms is not within [0%, 110%] of 23ms - 108.7% [±14.8]	Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 39ms is not within [0%, 110%] of 17ms - 129.4% [±18.2]	Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 31ms is not within [0%, 110%] of 25ms - 24.0% [±21.2]	- 8.7% [±15.7]
EmptyWorkbenchPerfTest#testOpen() [open]	+4.0% [±2.6]	+6.2% [±4.5]	+9.5% [±2.8]	+23.5% [±4.1]	+20.7% [±3.5]
EmptyWorkbenchPerfTest#testRestore() [close]	Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 76ms is not within [0%, 110%] of 38ms - 100.0% [±9.6]	Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 111ms is not within [0%, 110%] of 29ms - 282.8% [±215.0]	Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 37ms is not within [0%, 110%] of 22ms - 68.2% [±5.3]	- 8.8% [±13.6]	Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 34ms is not within [0%, 110%] of 29ms - 17.2% [±9.0]
EmptyWorkbenchPerfTest#testRestore() [open]	+9.0% [±2.8]	Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 531ms is not within [0%, 110%] of 378ms - 40.5% [±28.5]	Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 371ms is not within [0%, 110%] of 306ms - 21.2% [±5.4]	+15.9% [±5.1]	+23.7% [±3.4]
PlatformUIPerfTest#testCreateAndDisposeDisplayX100()	+20.0% [±4.9]	+25.6% [±5.0]	+34.2% [±6.0]	+29.0% [±6.9]	+37.2% [±6.0]
PlatformUIPerfTest#testRunAndShutdownWorkbench()	Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200801281721, jvm=sun}': Elapsed Process: 1.58s is not within [0%, 110%] of 1.22s - 30.1% [±67.0]	+13.3% [±62.4]	- 9.1% [±66.8]	- 2.0% [±80.9]	+4.8% [±65.4]