Performance of org.eclipse.jdt.ui: I20150603-2000 relative to R-4.4-201406061215

SUSE Linux Enterprise Server 11 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 65 scenarios	SUSE Linux Enterprise Server 11 (x86_64)
CleanUpPerfTest#testAllCleanUps()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 8.5s is not within [0%, 110'%] of 5.2s - 63.0% [±4.5]
CleanUpPerfTest#testCodeFormatCleanUp()	+4.6% [±7.9]
CleanUpPerfTest#testCodeStyleCleanUp()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 2.3s is not within [0%, 110'%] of 1.7s - 38.2% [±7.9]
CleanUpPerfTest#testControlStatementsCleanUp()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.7s is not within [0%, 110'%] of 0.62s - 12.2% [±9.7]
CleanUpPerfTest#testConvertLoopCleanUp()	+6.2% [±4.6]
CleanUpPerfTest#testExpressionsCleanUp()	- 6.1% [±8.4]
CleanUpPerfTest#testJava50CleanUp()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.73s is not within [0%, 110'%] of 0.6s - 21.0% [±7.4]
CleanUpPerfTest#testNullCleanUp()	- 4.8% [±8.2]
CleanUpPerfTest#testOrganizeImports()	+5.1% [±9.4]
CleanUpPerfTest#testSortMembersCleanUp()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1.1s is not within [0%, 110'%] of 0.81s - 39.3% [±7.8]
CleanUpPerfTest#testStringCleanUp()	+4.7% [±8.8]
CleanUpPerfTest#testUnnecessaryCodeCleanUp()	+16.9% [±7.9]
CleanUpPerfTest#testUnusedCodeCleanUp()	+6.1% [±7.0]
CleanUpPerfTest#testVariableDeclarationCleanUp()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1.1s is not within [0%, 110'%] of 0.82s - 37.8% [±5.1]
OpenTypePerfTest#testColdException()	+47.3% [n/a]
OpenTypePerfTest#testWarmException()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.35s is not within [0%, 110'%] of 0.32s - 10.1% [n/a]
OpenTypePerfTest#testWarmException10()	- 2.5% [±5.0]
OpenTypePerfTest#testWarmOpenSWT()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.34s is not within [0%, 110'%] of 0.31s - 10.8% [n/a]
OpenTypePerfTest#testWarmOpenSWTHistory10()	- 7.5% [±2.6]
OpenTypePerfTest#testWarmS10()	+1.0% [±6.3]
OrganizeImportsPerfTest#testOrganizeImport()	- 2.9% [±14.4]
PackageExplorerColdPerfTest#testOpen()	+1.9% [n/a]
PackageExplorerEmptyPerfTest#testOpen()	+0.0% [n/a]
PackageExplorerPerfTest#testExpand()	- 2.6% [n/a]
PackageExplorerPerfTest#testOpen()	+2.6% [n/a]
PackageExplorerPerfTest#testRefreshClassFolder()	+8.5% [n/a]
PackageExplorerPerfTest#testSelect()	+31.0% [n/a]
PackageExplorerWarmPerfTest#testOpen()	- 2.7% [±4.4]
IntroduceIndirectionPerfAcceptanceTests#testIntroduceIndirection()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 22.3s is not within [0%, 110'%] of 19.7s - 13.5% [n/a]
MoveCompilationUnitPerfTests1#test_1000_10()	- 2.0% [n/a]
MoveCompilationUnitPerfTests1#test_100_10()	- 8.5% [n/a]
MoveCompilationUnitPerfTests1#test_10_10()	- 9.5% [±0.7]
MoveCompilationUnitPerfTests2#test_10_10()	- 9.8% [±4.7]
MoveCompilationUnitPerfTests2#test_10_100()	+6.0% [n/a]
MoveCompilationUnitPerfTests2#test_10_1000()	+2.0% [n/a]
MoveStaticMembersPerfTests1#test_1000_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 36.2s is not within [0%, 110'%] of 25.6s - 41.4% [n/a]
MoveStaticMembersPerfTests1#test_100_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 5.4s is not within [0%, 110'%] of 4.2s - 29.3% [n/a]
MoveStaticMembersPerfTests1#test_10_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.83s is not within [0%, 110'%] of 0.7s - 17.5% [±3.5]
MoveStaticMembersPerfTests2#test_10_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.86s is not within [0%, 110'%] of 0.7s - 22.1% [±6.3]
MoveStaticMembersPerfTests2#test_10_100()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 3.9s is not within [0%, 110'%] of 2.6s - 51.5% [n/a]
MoveStaticMembersPerfTests2#test_10_1000()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 42.8s is not within [0%, 110'%] of 37.8s - 13.2% [n/a]
RenameMethodPerfTests1#test_1000_10()	+0.4% [±6.5]
RenameMethodPerfTests1#test_100_10()	- 3.2% [±7.7]
RenameMethodPerfTests1#test_10_10()	+4.0% [±5.4]
RenameMethodPerfTests2#test_10_10()	+1.0% [±5.6]
RenameMethodPerfTests2#test_10_100()	- 2.6% [±5.9]
RenameMethodPerfTests2#test_10_1000()	+5.6% [±6.7]
RenameMethodWithOverloadPerfTests#test_1000_10()	+4.5% [±8.7]
RenameMethodWithOverloadPerfTests#test_100_10()	- 3.0% [±5.5]
RenameMethodWithOverloadPerfTests#test_10_10()	- 8.9% [±5.4]
RenamePackagePerfTests1#test_1000_10()	- 3.2% [±6.3]
RenamePackagePerfTests1#test_100_10()	- 2.2% [±6.8]
RenamePackagePerfTests1#test_10_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.41s is not within [0%, 110'%] of 0.34s - 18.9% [±14.4]
RenamePackagePerfTests2#test_10_10()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.38s is not within [0%, 110'%] of 0.34s - 11.6% [±5.4]
RenamePackagePerfTests2#test_10_100()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1.3s is not within [0%, 110'%] of 1.1s - 20.1% [±7.5]
RenamePackagePerfTests2#test_10_1000()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 8s is not within [0%, 110'%] of 6.6s - 21.5% [±9.1]
RenameTypePerfAcceptanceTests#testWarm()	+28.1% [n/a]
RenameTypePerfTests1#test_1000_10()	+0.4% [±6.2]
RenameTypePerfTests1#test_100_10()	- 4.2% [±7.3]
RenameTypePerfTests1#test_10_10()	- 4.4% [±5.1]
RenameTypePerfTests2#test_10_10()	+0.6% [±7.8]
RenameTypePerfTests2#test_10_100()	- 7.3% [±8.0]
RenameTypePerfTests2#test_10_1000()	- 3.1% [±7.3]
ExtractInterfaceHeapAcceptanceTests#testExtractControl()	- 6.5% [n/a]
ExtractInterfacePerfAcceptanceTests#testExtractControl()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 32.5s is not within [0%, 110'%] of 29.3s - 10.7% [n/a]