Performance of org.eclipse.core: I20070625-1500 relative to 3.2

RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)

Win XP Sun 1.4.2_10 (3 GHz 2 GB)

RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)

Win XP Sun 1.4.2_10 (2 GHz 512 MB)

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)


Scenario Status

The following table gives a complete but compact view of performance results for the component.
Each line of the table shows the results for one scenario on all machines.

The name of the scenario is in bold when its results are also displayed in the fingerprints
and starts with an '*' when the scenario has no results in the last baseline run.

Here are information displayed for each test (ie. in each cell): Hints:

All 51 scenarios

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)
RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)
RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)
Win XP Sun 1.4.2_10 (2 GHz 512 MB)
Win XP Sun 1.4.2_10 (3 GHz 2 GB)
BenchFileStore#testStoreExists() - 1.7% [±2.5] - 1.0% [±2.2] - 1.1% [±2.1] +0.7% [±0.9] +1.5% [±0.6]
BenchFileStore#testStoreIsReadOnly() +0.9% [±1.1] +0.0% [±1.4] - 1.1% [±0.6] +1.2% [±1.0] +1.7% [±0.7]
BenchFileStore#testStoreLastModified() - 0.8% [±1.0] - 1.0% [±1.7] - 2.2% [±0.6] +1.4% [±0.3] +2.0% [±0.4]
*BenchMiscWorkspace#testGetProject() (vs. I20060801-0900) - 0.8% [±1.1] - 0.8% [±0.8] +0.4% [±0.4] - 5.0% [±0.6] +2.9% [±0.3]
BenchMiscWorkspace#testNoOp() - 1.1% [±1.1] +63.1% [±0.3] +67.2% [±0.3] +58.0% [±1.6] +70.4% [±0.3]
BenchWorkspace#testCountResources() +0.0% [±1.0] +0.0% [±3.2] +0.0% [±3.2] +0.4% [±4.4] - 4.0% [±3.3]
BenchWorkspace#testCountResourcesDuringOperation() +3.0% [±0.6] - 0.4% [±0.7] - 3.1% [±1.0] - 1.4% [±0.7] +0.7% [±0.6]
*BenchWorkspace#testFindMaxProblemSeverity() (vs. N20070315-0010) +3.9% [±0.7] - 0.7% [±0.2] - 2.2% [±0.3] - 4.7% [±1.3] +0.1% [±0.1]
LocalHistoryPerformanceTest#testAddState() +33.0% [±65.2] +68.2% [±30.3] +70.6% [±30.6] +37.2% [±26.7] +47.5% [±20.8]
LocalHistoryPerformanceTest#testBug28603() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 85ms is not within [0%, 110%] of 42ms - 102.4% [±107.0] +6.7% [±5.5] +3.8% [±5.2] +8.5% [±6.2] +12.9% [±4.2]
LocalHistoryPerformanceTest#testClearHistory100x4() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 127ms is not within [0%, 110%] of 32ms - 296.9% [±292.2] +93.0% [±90.0] +13.6% [±29.7] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 35ms is not within [0%, 110%] of 31ms - 12.9% [±24.7] +0.0% [±27.5]
LocalHistoryPerformanceTest#testClearHistory20x20() - 10.0% [±9.9] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 9ms is not within [0%, 110%] of 8ms - 12.5% [±7.2] +0.0% [±4.4] +0.0% [±2.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 15ms is not within [0%, 110%] of 11ms - 36.4% [±34.2]
LocalHistoryPerformanceTest#testClearHistory4x100() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 16ms is not within [0%, 110%] of 9ms - 77.8% [±63.1] +12.5% [±7.9] Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 10ms is not within [0%, 110%] of 7ms - 42.9% [±21.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 23ms is not within [0%, 110%] of 16ms - 43.8% [±49.0] +0.0% [±50.4]
LocalHistoryPerformanceTest#testCopyHistory100x4() - 9.1% [±28.5] +24.6% [±23.0] +0.2% [±7.3] - 2.9% [±6.7] - 0.9% [±3.0]
LocalHistoryPerformanceTest#testCopyHistory20x20() +44.0% [±35.3] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 264ms is not within [0%, 110%] of 151ms - 74.8% [±93.8] +18.0% [±18.9] +15.3% [±7.5] +2.5% [±7.1]
LocalHistoryPerformanceTest#testCopyHistory4x100() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 87ms is not within [0%, 110%] of 57ms - 52.6% [±43.8] - 2.9% [±53.1] +37.5% [±40.8] +12.1% [±14.6] +16.4% [±20.2]
LocalHistoryPerformanceTest#testGetDeletedMembers100x4() +3.8% [±23.8] +12.2% [±6.8] +14.6% [±7.7] +2.1% [±1.1] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 54ms is not within [0%, 110%] of 47ms - 14.9% [±16.0]
LocalHistoryPerformanceTest#testGetDeletedMembers20x20() +0.0% [±2.9] +0.0% [±4.4] - 7.7% [±3.8] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 31ms is not within [0%, 110%] of 23ms - 34.8% [±37.0] +25.8% [±56.3]
LocalHistoryPerformanceTest#testGetDeletedMembers4x100() +42.9% [±67.3] +38.9% [±36.2] +41.2% [±12.1] +0.0% [±47.7] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 23ms is not within [0%, 110%] of 15ms - 53.3% [±50.1]
LocalHistoryPerformanceTest#testGetHistory() +0.0% [n/a] +12.5% [n/a] +7.1% [n/a] +6.2% [n/a] [n/a]
LocalHistoryPerformanceTest#testHistoryCleanUp100x4() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 738ms is not within [0%, 110%] of 443ms - 66.6% [±64.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.33s is not within [0%, 110%] of 354ms - 275.4% [±269.3] - 4.0% [±7.1] +3.6% [±73.0] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 2.06s is not within [0%, 110%] of 646ms - 218.7% [±216.7]
LocalHistoryPerformanceTest#testHistoryCleanUp20x20() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 722ms is not within [0%, 110%] of 444ms - 62.6% [±56.6] +3.6% [±8.5] - 8.0% [±2.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.83s is not within [0%, 110%] of 762ms - 139.9% [±126.8] +16.2% [±75.6]
MarkerPerformanceTest#testSetAttributes1() - 3.4% [n/a] +2.7% [n/a] +3.6% [n/a] +0.9% [n/a] +3.6% [n/a]
MarkerPerformanceTest#testSetAttributes2() +2.3% [n/a] +1.3% [n/a] +3.4% [n/a] +1.6% [n/a] +3.5% [n/a]
PropertyManagerPerformanceTest#testGetProperty100x4() +0.8% [±1.2] - 0.1% [±1.3] - 0.6% [±0.7] +0.0% [±0.9] - 0.6% [±0.6]
PropertyManagerPerformanceTest#testGetProperty20x20() +2.8% [±2.3] - 7.7% [±4.8] - 0.2% [±0.1] - 3.5% [±0.2] - 3.8% [±0.1]
PropertyManagerPerformanceTest#testGetProperty4x100() +2.6% [±0.4] - 0.6% [±0.1] - 0.2% [±0.1] - 4.0% [±0.1] - 4.0% [±0.0]
PropertyManagerPerformanceTest#testSetProperty100x4() +0.6% [±0.9] - 2.2% [±2.1] - 3.4% [±2.6] - 0.7% [±1.5] - 0.2% [±0.8]
PropertyManagerPerformanceTest#testSetProperty20x20() +0.9% [±1.2] +0.0% [±3.2] +0.0% [±0.9] +1.8% [±4.0] - 0.6% [±1.4]
PropertyManagerPerformanceTest#testSetProperty4x100() - 0.6% [±1.8] +0.0% [±4.6] - 5.8% [±2.1] +0.0% [±0.9] - 2.1% [±1.6]
WorkspacePerformanceTest#testCreateResources() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 16.61s is not within [0%, 110%] of 12.89s - 28.9% [±40.3] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 7.81s is not within [0%, 110%] of 7.03s - 11.2% [±22.6] +2.7% [±1.9] +1.7% [±9.1] +10.1% [±20.2]
WorkspacePerformanceTest#testDeleteProject() +1.0% [±19.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 16.67s is not within [0%, 110%] of 14.08s - 18.4% [±31.0] +17.1% [±25.4] - 4.0% [±4.1] +1.6% [±1.2]
WorkspacePerformanceTest#testFolderCopy() +5.2% [±12.8] +14.5% [±13.5] +2.4% [±15.0] - 5.6% [±3.7] - 0.7% [±11.9]
WorkspacePerformanceTest#testFolderMove() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 4.65s is not within [0%, 110%] of 3.48s - 33.6% [±66.1] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.41s is not within [0%, 110%] of 1.23s - 14.5% [±12.2] - 6.9% [±7.9] +12.2% [±7.0] - 4.1% [±3.7]
*WorkspacePerformanceTest#testRefreshProject() (vs. N20070215-0010) - 0.8% [±9.8] - 4.5% [±2.7] +7.1% [±9.1] - 1.4% [±1.1] - 0.7% [±0.8]
BenchPath#testHash() - 1.2% [±0.1] +0.9% [±0.3] - 0.8% [±0.2] +1.0% [±0.2] +0.4% [±0.3]
BenchPath#testPathCreation() +2.2% [±0.6] +3.5% [±0.5] - 0.2% [±0.4] +1.3% [±0.5] +2.1% [±0.4]
BenchPath#testToOSString() +2.1% [±0.3] - 0.8% [±0.4] +0.6% [±0.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.21s is not within [0%, 110%] of 870ms - 39.3% [±0.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 867ms is not within [0%, 110%] of 679ms - 27.7% [±0.5]
BenchPath#testToString() +1.6% [±0.3] - 1.2% [±0.5] +0.6% [±0.2] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.22s is not within [0%, 110%] of 854ms - 42.9% [±0.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 881ms is not within [0%, 110%] of 681ms - 29.4% [±0.5]
ContentTypePerformanceTest#testContentMatching() - 4.0% [±2.1] - 9.9% [±2.2] - 6.3% [±1.4] - 8.0% [±2.2] - 6.8% [±1.8]
ContentTypePerformanceTest#testIsKindOf() - 0.5% [±0.9] +0.6% [±0.7] +1.1% [±0.8] - 6.1% [±1.1] - 1.4% [±0.7]
ContentTypePerformanceTest#testLoadCatalog() +11.2% [±0.0] +6.4% [±0.0] +7.6% [±0.0] +10.9% [±0.0] +5.0% [±0.0]
ContentTypePerformanceTest#testNameMatching() - 0.4% [±0.7] +2.2% [±0.4] - 4.0% [±0.6] - 8.4% [±0.8] - 9.1% [±0.5]
PreferencePerformanceTest#testGetStringCommonPrefixKeys() - 6.4% [±0.7] +0.3% [±0.6] +2.6% [±0.9] +0.3% [±1.9] - 2.9% [±2.0]
PreferencePerformanceTest#testGetStringMisses() +4.8% [±2.3] +12.1% [±0.5] +0.4% [±0.2] - 7.9% [±1.4] - 6.6% [±0.6]
PreferencePerformanceTest#testGetStringSequentialKeys() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.75s is not within [0%, 110%] of 1.4s - 24.8% [±0.4] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.1s is not within [0%, 110%] of 930ms - 18.2% [±0.5] +3.6% [±0.9] +10.8% [±0.3] +13.1% [±0.9]
PreferencePerformanceTest#testGetStringUniqueKeys() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 1.59s is not within [0%, 110%] of 1.22s - 30.7% [±0.6] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 987ms is not within [0%, 110%] of 813ms - 21.4% [±0.7] +3.9% [±0.8] +12.3% [±1.5] +16.1% [±1.2]
PreferencePerformanceTest#testPutStringKeys() - 5.5% [±0.5] - 5.1% [±1.1] +1.1% [±1.6] - 4.2% [±0.4] - 4.5% [±0.9]
PreferencePerformanceTest#testRemoveStringKeys() - 4.9% [±0.4] - 6.8% [±0.5] +5.4% [±0.5] - 3.9% [±1.4] - 4.8% [±1.4]
StartupTest.testApplicationStartup Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 4.86s is not within [0%, 105%] of 2.21s - Explanation comment: Performance decrease caused by the JAR signing. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=180219 for details. - 120.0% [±0.0] +9.8% [±0.0] +7.3% [±0.0] - 3.7% [±0.0] - 0.2% [±0.0]
UIStartupTest.testUIApplicationStartup Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 17.89s is not within [50%, 105%] of 16.21s - 10.4% [±0.0] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 11.47s is not within [50%, 105%] of 10.07s - 13.9% [±0.0] Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 9.22s is not within [50%, 105%] of 8.63s - 6.8% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 11.96s is not within [50%, 105%] of 10.36s - 15.4% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200706111606, jvm=sun}': Elapsed Process: 7.64s is not within [50%, 105%] of 6.67s - 14.4% [±0.0]