Performance of org.eclipse.core: M20161124-1400 relative to R (201606061100)

SUSE Linux Enterprise Server 11 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 54 scenarios	SUSE Linux Enterprise Server 11 (x86_64)
BenchFileStore#testStoreExists()	+1.5% [±5.3]
BenchFileStore#testStoreIsReadOnly()	+1.1% [±2.3]
BenchFileStore#testStoreLastModified()	+23.0% [±5.8]
BenchMiscWorkspace#testGetProject()	- 3.3% [±18.7]
BenchMiscWorkspace#testNoOp()	+1.6% [±5.1]
BenchWorkspace#testCountResources()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 39ms is not within [0%, 110'%] of 22ms - 71.8% [±18.4]
BenchWorkspace#testCountResourcesDuringOperation()	+27.1% [±13.8]
BenchWorkspace#testFindMaxProblemSeverity()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 2.6s is not within [0%, 110'%] of 2.2s - 17.0% [±2.1]
BuilderPerformanceTest#testManualBuildWithAutobuildOn()	+1.4% [±15.6]
FileSystemPerformanceTest#testPutFileInfo()	+1.4% [±4.4]
LocalHistoryPerformanceTest#testAddState()	+3.5% [±5.7]
LocalHistoryPerformanceTest#testBug28603()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 42ms is not within [0%, 110'%] of 34ms - 22.5% [±6.4]
LocalHistoryPerformanceTest#testClearHistory100x4()	+54.2% [±15.5]
LocalHistoryPerformanceTest#testClearHistory20x20()	+51.9% [±18.3]
LocalHistoryPerformanceTest#testClearHistory4x100()	+35.0% [±5.7]
LocalHistoryPerformanceTest#testCopyHistory100x4()	+0.0% [±2.6]
LocalHistoryPerformanceTest#testCopyHistory20x20()	- 6.8% [±5.1]
LocalHistoryPerformanceTest#testCopyHistory4x100()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 54ms is not within [0%, 110'%] of 47ms - 14.3% [±3.2]
LocalHistoryPerformanceTest#testGetDeletedMembers100x4()	+13.9% [±15.1]
LocalHistoryPerformanceTest#testGetDeletedMembers20x20()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 12ms is not within [0%, 110'%] of 9ms - 21.4% [±6.0]
LocalHistoryPerformanceTest#testGetDeletedMembers4x100()	- 6.2% [±2.5]
LocalHistoryPerformanceTest#testGetHistory()	Performance criteria not met when compared to 'R-4.6-201606061100': Elapsed Process= 14ms is not within [0%, 110'%] of 12ms - 10.2% [n/a]
LocalHistoryPerformanceTest#testHistoryCleanUp100x4()	- 2.4% [±6.9]
LocalHistoryPerformanceTest#testHistoryCleanUp20x20()	+1.1% [±8.7]
MarkerPerformanceTest#testSetAttributes1()	+5.4% [n/a]
MarkerPerformanceTest#testSetAttributes2()	+4.5% [n/a]
PropertyManagerPerformanceTest#testGetProperty100x4()	- 1.6% [±8.7]
PropertyManagerPerformanceTest#testGetProperty20x20()	+1.7% [±0.5]
PropertyManagerPerformanceTest#testGetProperty4x100()	- 2.6% [±0.3]
PropertyManagerPerformanceTest#testSetProperty100x4()	+8.1% [±9.2]
PropertyManagerPerformanceTest#testSetProperty20x20()	+32.7% [±6.9]
PropertyManagerPerformanceTest#testSetProperty4x100()	+16.3% [±1.7]
WorkspacePerformanceTest#testCloseOpenProject()	- 1.5% [±7.2]
WorkspacePerformanceTest#testCreateResources()	+2.2% [±0.4]
WorkspacePerformanceTest#testDeleteProject()	+8.7% [±0.9]
WorkspacePerformanceTest#testFolderCopy()	+31.4% [±9.5]
WorkspacePerformanceTest#testFolderMove()	+10.0% [±7.5]
WorkspacePerformanceTest#testLoadSnapshot()	- 5.7% [±14.9]
WorkspacePerformanceTest#testRefreshProject()	+5.1% [±2.7]
BenchPath#testHash()	+0.2% [±1.2]
BenchPath#testPathCreation()	- 1.9% [±3.8]
BenchPath#testToOSString()	+0.9% [±4.3]
BenchPath#testToString()	+1.4% [±3.5]
ContentTypePerformanceTest#testIsKindOf()	+13.8% [±24.4]
ContentTypePerformanceTest#testLoadCatalog()	- 5.8% [±1.8]
ContentTypePerformanceTest#testNameMatching()	+9.6% [±18.6]
PreferencePerformanceTest#testGetStringCommonPrefixKeys()	+0.5% [±4.1]
PreferencePerformanceTest#testGetStringMisses()	- 3.4% [±2.4]
PreferencePerformanceTest#testGetStringSequentialKeys()	+0.2% [±2.6]
PreferencePerformanceTest#testGetStringUniqueKeys()	- 2.4% [±3.6]
PreferencePerformanceTest#testPutStringKeys()	- 1.2% [±3.5]
PreferencePerformanceTest#testRemoveStringKeys()	+1.9% [±3.0]
StartupTest.testApplicationStartup	+3.3% [±0.8]
UIStartupTest.testUIApplicationStartup	+0.8% [±1.6]