Performance of org.eclipse.jdt.ui: I20190307-0500 relative to R-4.10-20181206081

SUSE Linux Enterprise Server 12 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 65 scenarios	SUSE Linux Enterprise Server 12 (x86_64)
*CleanUpPerfTest#testAllCleanUps() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 10.6s is not within [0%, 110'%] of 8.9s - 19.1% [±4.9]
*CleanUpPerfTest#testCodeFormatCleanUp() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 1.2s is not within [0%, 110'%] of 0.71s - 63.4% [±8.1]
*CleanUpPerfTest#testCodeStyleCleanUp() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 3.4s is not within [0%, 110'%] of 2.4s - 44.0% [±3.7]
*CleanUpPerfTest#testControlStatementsCleanUp() (vs. R-4.10-201812060815-201901031800)	+40.7% [±7.3]
*CleanUpPerfTest#testConvertLoopCleanUp() (vs. R-4.10-201812060815-201901031800)	- 7.4% [±6.1]
*CleanUpPerfTest#testExpressionsCleanUp() (vs. R-4.10-201812060815-201901031800)	+63.9% [±52.9]
*CleanUpPerfTest#testJava50CleanUp() (vs. R-4.10-201812060815-201901031800)	+41.8% [±27.2]
*CleanUpPerfTest#testNullCleanUp() (vs. R-4.10-201812060815-201901031800)	- 7.1% [±7.3]
*CleanUpPerfTest#testOrganizeImports() (vs. R-4.10-201812060815-201901031800)	+24.0% [±70.0]
*CleanUpPerfTest#testSortMembersCleanUp() (vs. R-4.10-201812060815-201901031800)	+12.8% [±26.5]
*CleanUpPerfTest#testStringCleanUp() (vs. R-4.10-201812060815-201901031800)	- 3.5% [±7.0]
*CleanUpPerfTest#testUnnecessaryCodeCleanUp() (vs. R-4.10-201812060815-201901031800)	- 0.9% [±7.2]
*CleanUpPerfTest#testUnusedCodeCleanUp() (vs. R-4.10-201812060815-201901031800)	+6.2% [±10.0]
*CleanUpPerfTest#testVariableDeclarationCleanUp() (vs. R-4.10-201812060815-201901031800)	+21.2% [±20.5]
*OpenTypePerfTest#testColdException() (vs. R-4.10-201812060815-201901031800)	+3.9% [n/a]
*OpenTypePerfTest#testWarmException() (vs. R-4.10-201812060815-201901031800)	+26.0% [n/a]
*OpenTypePerfTest#testWarmException10() (vs. R-4.10-201812060815-201901031800)	+22.8% [±7.2]
*OpenTypePerfTest#testWarmOpenSWT() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 2.3s is not within [0%, 110'%] of 0.88s - 157.2% [n/a]
*OpenTypePerfTest#testWarmOpenSWTHistory10() (vs. R-4.10-201812060815-201901031800)	+24.3% [±6.9]
*OpenTypePerfTest#testWarmS10() (vs. R-4.10-201812060815-201901031800)	+15.2% [±4.4]
*OrganizeImportsPerfTest#testOrganizeImport() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 0.14s is not within [0%, 110'%] of 0.12s - 21.1% [±5.0]
*PackageExplorerColdPerfTest#testOpen() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 77ms is not within [0%, 110'%] of 43ms - 80.2% [n/a]
*PackageExplorerEmptyPerfTest#testOpen() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 1.4s is not within [0%, 110'%] of 0.28s - 418.1% [n/a]
*PackageExplorerPerfTest#testExpand() (vs. R-4.10-201812060815-201901031800)	+6.9% [n/a]
*PackageExplorerPerfTest#testOpen() (vs. R-4.10-201812060815-201901031800)	Performance criteria not met when compared to 'R-4.10-201812060815-201901031800': Elapsed Process= 0.45s is not within [0%, 110'%] of 0.34s - 32.5% [n/a]
*PackageExplorerPerfTest#testRefreshClassFolder() (vs. R-4.10-201812060815-201901031800)	- 6.6% [n/a]
*PackageExplorerPerfTest#testSelect() (vs. R-4.10-201812060815-201901031800)	- 8.8% [n/a]
*PackageExplorerWarmPerfTest#testOpen() (vs. R-4.10-201812060815-201901031800)	+26.9% [±5.8]
*IntroduceIndirectionPerfAcceptanceTests#testIntroduceIndirection() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 40.6s is not within [0%, 110'%] of 26.3s - 54.6% [n/a]
*MoveCompilationUnitPerfTests1#test_1000_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 16.2s is not within [0%, 110'%] of 12.4s - 30.6% [n/a]
*MoveCompilationUnitPerfTests1#test_100_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 2.9s is not within [0%, 110'%] of 2s - 49.7% [n/a]
*MoveCompilationUnitPerfTests1#test_10_10() (vs. R-4.10-201812060815-201901232200)	- 0.2% [±41.9]
*MoveCompilationUnitPerfTests2#test_10_10() (vs. R-4.10-201812060815-201901232200)	+15.5% [±4.5]
*MoveCompilationUnitPerfTests2#test_10_100() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.99s is not within [0%, 110'%] of 0.77s - 28.4% [n/a]
*MoveCompilationUnitPerfTests2#test_10_1000() (vs. R-4.10-201812060815-201901232200)	- 7.0% [n/a]
*MoveStaticMembersPerfTests1#test_1000_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 10mn 15s is not within [0%, 110'%] of 40.4s - 1,421.8% [n/a]
*MoveStaticMembersPerfTests1#test_100_10() (vs. R-4.10-201812060815-201901232200)	+53.2% [n/a]
*MoveStaticMembersPerfTests1#test_10_10() (vs. R-4.10-201812060815-201901232200)	- 2.6% [±2.7]
*MoveStaticMembersPerfTests2#test_10_10() (vs. R-4.10-201812060815-201901232200)	+58.2% [±47.6]
*MoveStaticMembersPerfTests2#test_10_100() (vs. R-4.10-201812060815-201901232200)	- 8.0% [n/a]
*MoveStaticMembersPerfTests2#test_10_1000() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 1mn 42s is not within [0%, 110'%] of 52.1s - 96.4% [n/a]
*RenameMethodPerfTests1#test_1000_10() (vs. R-4.10-201812060815-201901232200)	+24.6% [±17.5]
*RenameMethodPerfTests1#test_100_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 4.1s is not within [0%, 110'%] of 2.3s - 76.3% [±40.5]
*RenameMethodPerfTests1#test_10_10() (vs. R-4.10-201812060815-201901232200)	+1.9% [±6.8]
*RenameMethodPerfTests2#test_10_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.76s is not within [0%, 110'%] of 0.47s - 60.2% [±38.7]
*RenameMethodPerfTests2#test_10_100() (vs. R-4.10-201812060815-201901232200)	+6.7% [±31.1]
*RenameMethodPerfTests2#test_10_1000() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 7.3s is not within [0%, 110'%] of 0.75s - 869.5% [±61.4]
*RenameMethodWithOverloadPerfTests#test_1000_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 26.6s is not within [0%, 110'%] of 17.2s - 54.9% [±14.5]
*RenameMethodWithOverloadPerfTests#test_100_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 15.1s is not within [0%, 110'%] of 3.5s - 331.3% [±25.3]
*RenameMethodWithOverloadPerfTests#test_10_10() (vs. R-4.10-201812060815-201901232200)	+0.2% [±20.3]
*RenamePackagePerfTests1#test_1000_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 16.3s is not within [0%, 110'%] of 12.5s - 30.5% [±19.0]
*RenamePackagePerfTests1#test_100_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 2.1s is not within [0%, 110'%] of 1.4s - 49.4% [±12.2]
*RenamePackagePerfTests1#test_10_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 9.6s is not within [0%, 110'%] of 0.45s - 2,013.9% [±147.0]
*RenamePackagePerfTests2#test_10_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.87s is not within [0%, 110'%] of 0.54s - 60.4% [±26.6]
*RenamePackagePerfTests2#test_10_100() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 11.2s is not within [0%, 110'%] of 3.5s - 221.5% [±23.4]
*RenamePackagePerfTests2#test_10_1000() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 34s is not within [0%, 110'%] of 18.3s - 85.3% [±12.4]
*RenameTypePerfAcceptanceTests#testWarm() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 4.5s is not within [0%, 110'%] of 3.9s - 13.6% [n/a]
*RenameTypePerfTests1#test_1000_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 19.9s is not within [0%, 110'%] of 10.7s - 85.1% [±10.6]
*RenameTypePerfTests1#test_100_10() (vs. R-4.10-201812060815-201901232200)	+56.8% [±39.8]
*RenameTypePerfTests1#test_10_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.82s is not within [0%, 110'%] of 0.29s - 186.7% [±14.1]
*RenameTypePerfTests2#test_10_10() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.31s is not within [0%, 110'%] of 0.28s - 10.6% [±5.1]
*RenameTypePerfTests2#test_10_100() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 0.99s is not within [0%, 110'%] of 0.37s - 165.8% [±26.6]
*RenameTypePerfTests2#test_10_1000() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 1.3s is not within [0%, 110'%] of 1.1s - 16.2% [±8.6]
*ExtractInterfaceHeapAcceptanceTests#testExtractControl() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 1mn 56s is not within [0%, 110'%] of 44.7s - 159.9% [n/a]
*ExtractInterfacePerfAcceptanceTests#testExtractControl() (vs. R-4.10-201812060815-201901232200)	Performance criteria not met when compared to 'R-4.10-201812060815-201901232200': Elapsed Process= 4mn 37s is not within [0%, 110'%] of 55.4s - 400.7% [n/a]