Performance of org.eclipse.core: I20150603-2000 relative to R-4.4-201406061215

SUSE Linux Enterprise Server 11 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 53 scenarios	SUSE Linux Enterprise Server 11 (x86_64)
BenchFileStore#testStoreExists()	+4.0% [±3.6]
BenchFileStore#testStoreIsReadOnly()	+1.8% [±1.7]
BenchFileStore#testStoreLastModified()	+7.8% [±7.1]
BenchMiscWorkspace#testGetProject()	+0.9% [±18.5]
BenchMiscWorkspace#testNoOp()	- 1.6% [±3.5]
BenchWorkspace#testCountResources()	+49.4% [±5.5]
BenchWorkspace#testCountResourcesDuringOperation()	+3.6% [±18.9]
BenchWorkspace#testFindMaxProblemSeverity()	+8.6% [±3.4]
BuilderPerformanceTest#testManualBuildWithAutobuildOn()	+0.0% [±12.8]
FileSystemPerformanceTest#testPutFileInfo()	- 1.5% [±4.2]
LocalHistoryPerformanceTest#testAddState()	- 3.9% [±6.5]
LocalHistoryPerformanceTest#testBug28603()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 42ms is not within [0%, 110'%] of 31ms - 35.2% [±6.7]
LocalHistoryPerformanceTest#testClearHistory100x4()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 37ms is not within [0%, 110'%] of 33ms - 10.2% [±29.7]
LocalHistoryPerformanceTest#testClearHistory20x20()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 13ms is not within [0%, 110'%] of 10ms - 24.2% [±17.2]
LocalHistoryPerformanceTest#testClearHistory4x100()	+1.8% [±17.4]
LocalHistoryPerformanceTest#testCopyHistory100x4()	- 7.8% [±7.0]
LocalHistoryPerformanceTest#testCopyHistory20x20()	- 7.6% [±1.2]
LocalHistoryPerformanceTest#testCopyHistory4x100()	- 5.7% [±5.9]
LocalHistoryPerformanceTest#testGetDeletedMembers100x4()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 32ms is not within [0%, 110'%] of 26ms - 20.4% [±22.6]
LocalHistoryPerformanceTest#testGetDeletedMembers20x20()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 10ms is not within [0%, 110'%] of 8ms - 18.8% [±1.0]
LocalHistoryPerformanceTest#testGetDeletedMembers4x100()	+30.0% [±8.8]
LocalHistoryPerformanceTest#testGetHistory()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 15ms is not within [0%, 110'%] of 11ms - 28.1% [n/a]
LocalHistoryPerformanceTest#testHistoryCleanUp100x4()	+0.5% [±3.5]
LocalHistoryPerformanceTest#testHistoryCleanUp20x20()	- 2.3% [±7.0]
MarkerPerformanceTest#testSetAttributes1()	+2.1% [n/a]
MarkerPerformanceTest#testSetAttributes2()	+5.4% [n/a]
PropertyManagerPerformanceTest#testGetProperty100x4()	- 7.4% [±11.2]
PropertyManagerPerformanceTest#testGetProperty20x20()	- 2.2% [±0.4]
PropertyManagerPerformanceTest#testGetProperty4x100()	- 3.3% [±0.5]
PropertyManagerPerformanceTest#testSetProperty100x4()	+10.7% [±9.2]
PropertyManagerPerformanceTest#testSetProperty20x20()	+19.1% [±8.6]
PropertyManagerPerformanceTest#testSetProperty4x100()	+16.0% [±1.4]
WorkspacePerformanceTest#testCloseOpenProject()	+13.9% [±2.9]
WorkspacePerformanceTest#testCreateResources()	- 2.0% [±0.8]
WorkspacePerformanceTest#testDeleteProject()	+17.1% [±14.2]
WorkspacePerformanceTest#testFolderCopy()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 26.5s is not within [0%, 110'%] of 18.8s - 41.0% [±50.6]
WorkspacePerformanceTest#testFolderMove()	+11.8% [±5.7]
WorkspacePerformanceTest#testLoadSnapshot()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.26s is not within [0%, 110'%] of 0.23s - 11.2% [±17.2]
WorkspacePerformanceTest#testRefreshProject()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.31s is not within [0%, 110'%] of 0.26s - 15.5% [±18.7]
BenchPath#testHash()	- 3.4% [±4.7]
BenchPath#testPathCreation()	+5.4% [±3.2]
BenchPath#testToOSString()	+1.2% [±6.1]
BenchPath#testToString()	+7.6% [±6.7]
ContentTypePerformanceTest#testLoadCatalog()	- 1.8% [n/a]
ContentTypePerformanceTest#testNameMatching()	+20.1% [±25.1]
PreferencePerformanceTest#testGetStringCommonPrefixKeys()	+8.2% [±3.7]
PreferencePerformanceTest#testGetStringMisses()	+6.9% [±4.9]
PreferencePerformanceTest#testGetStringSequentialKeys()	- 1.5% [±8.7]
PreferencePerformanceTest#testGetStringUniqueKeys()	+4.0% [±4.0]
PreferencePerformanceTest#testPutStringKeys()	- 1.8% [±3.6]
PreferencePerformanceTest#testRemoveStringKeys()	- 2.1% [±9.9]
StartupTest.testApplicationStartup	+0.5% [±0.6]
UIStartupTest.testUIApplicationStartup	- 9.2% [±2.6]