Performance of org.eclipse.core: M20071023-1652 relative to 3.2

RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)

Win XP Sun 1.4.2_10 (3 GHz 2 GB)

RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)

Win XP Sun 1.4.2_10 (2 GHz 512 MB)

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)


Scenario Status

The following table gives a complete but compact view of performance results for the component.
Each line of the table shows the results for one scenario on all machines.

The name of the scenario is in bold when its results are also displayed in the fingerprints
and starts with an '*' when the scenario has no results in the last baseline run.

Here are information displayed for each test (ie. in each cell): Hints:

All 51 scenarios

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)
RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)
RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)
Win XP Sun 1.4.2_10 (2 GHz 512 MB)
Win XP Sun 1.4.2_10 (3 GHz 2 GB)
BenchFileStore#testStoreExists() +0.0% [±2.8] - 2.0% [±2.4] +0.0% [±2.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 996ms is not within [0%, 110%] of 591ms - 68.5% [±17.6] +1.5% [±0.5]
BenchFileStore#testStoreIsReadOnly() +0.0% [±1.0] - 1.0% [±1.6] +0.0% [±0.6] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.07s is not within [0%, 110%] of 594ms - 79.8% [±36.1] +1.1% [±0.6]
BenchFileStore#testStoreLastModified() +0.9% [±0.9] +1.0% [±1.2] +0.0% [±0.6] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.16s is not within [0%, 110%] of 589ms - 97.1% [±20.4] +0.9% [±0.3]
*BenchMiscWorkspace#testGetProject() (vs. I20060801-0900) - 18.7% [±0.5] +3.5% [±0.6] +1.9% [±0.4] - 800.2% [±312.6] - 0.9% [±0.3]
BenchMiscWorkspace#testNoOp() +5.3% [±1.4] +64.0% [±0.3] +67.6% [±0.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 46.52s is not within [0%, 110%] of 12.34s - 277.0% [±112.1] +70.3% [±0.4]
BenchWorkspace#testCountResources() - 0.4% [±0.9] - 3.4% [±2.9] +1.7% [±3.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 387ms is not within [0%, 110%] of 240ms - 61.3% [±43.1] - 2.7% [±3.0]
BenchWorkspace#testCountResourcesDuringOperation() +0.8% [±1.6] - 0.2% [±0.8] - 3.0% [±1.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2.9s is not within [0%, 110%] of 1.59s - 82.4% [±45.4] - 0.5% [±0.6]
*BenchWorkspace#testFindMaxProblemSeverity() (vs. N20070315-0010) +4.1% [±0.6] - 0.6% [±0.2] - 2.9% [±0.2] - 125.5% [±32.4] - 0.6% [±0.2]
LocalHistoryPerformanceTest#testAddState() +65.7% [±62.7] +68.0% [±29.0] +70.6% [±30.6] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 848ms is not within [0%, 110%] of 245ms - 246.1% [±287.4] +46.7% [±21.6]
LocalHistoryPerformanceTest#testBug28603() +4.8% [±6.5] +6.7% [±6.4] +7.7% [±4.4] +6.6% [±28.0] +11.3% [±4.3]
LocalHistoryPerformanceTest#testClearHistory100x4() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 129ms is not within [0%, 110%] of 35ms - 268.6% [±286.4] +33.3% [±33.6] +13.6% [±32.4] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 27ms is not within [0%, 110%] of 23ms - 17.4% [±46.9] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 27ms is not within [0%, 110%] of 23ms - 17.4% [±25.4]
LocalHistoryPerformanceTest#testClearHistory20x20() +0.0% [±5.7] +0.0% [±4.8] +0.0% [±4.8] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 15ms is not within [0%, 110%] of 12ms - 25.0% [±33.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 11ms is not within [0%, 110%] of 4ms - 175.0% [±138.6]
LocalHistoryPerformanceTest#testClearHistory4x100() +35.7% [±31.2] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 8ms is not within [0%, 110%] of 7ms - 14.3% [±6.8] +0.0% [±5.5] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 39ms is not within [0%, 110%] of 8ms - 387.5% [±177.5] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 15ms is not within [0%, 110%] of 8ms - 87.5% [±57.8]
LocalHistoryPerformanceTest#testCopyHistory100x4() +4.7% [±23.5] +23.8% [±6.9] +11.5% [±9.1] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 11.1s is not within [0%, 110%] of 1.96s - 466.9% [±173.5] +0.2% [±3.0]
LocalHistoryPerformanceTest#testCopyHistory20x20() +50.3% [±36.7] +63.7% [±48.9] +22.8% [±20.2] +6.4% [±8.2] +5.6% [±7.0]
LocalHistoryPerformanceTest#testCopyHistory4x100() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 63ms is not within [0%, 110%] of 55ms - 14.5% [±13.5] +35.7% [±38.1] +37.5% [±40.7] - 2.3% [±19.8] +18.6% [±19.1]
LocalHistoryPerformanceTest#testGetDeletedMembers100x4() +9.8% [±16.8] +10.4% [±7.9] +12.5% [±7.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 109ms is not within [0%, 110%] of 70ms - 55.7% [±75.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 54ms is not within [0%, 110%] of 47ms - 14.9% [±16.0]
LocalHistoryPerformanceTest#testGetDeletedMembers20x20() - 8.3% [±49.2] +0.0% [±4.4] +0.0% [±3.6] +25.8% [±27.5] +25.8% [±24.2]
LocalHistoryPerformanceTest#testGetDeletedMembers4x100() +44.0% [±46.2] +58.6% [±63.8] +31.2% [±9.4] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 70ms is not within [0%, 110%] of 23ms - 204.3% [±175.2] +30.4% [±32.6]
LocalHistoryPerformanceTest#testGetHistory() +5.0% [n/a] +0.0% [n/a] +14.3% [n/a] +6.2% [n/a] +0.0% [n/a]
LocalHistoryPerformanceTest#testHistoryCleanUp100x4() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 688ms is not within [0%, 110%] of 444ms - 55.0% [±50.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 619ms is not within [0%, 110%] of 441ms - 40.4% [±64.5] - 3.2% [±7.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 8.54s is not within [0%, 110%] of 2.43s - 251.2% [±146.9] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 956ms is not within [0%, 110%] of 634ms - 50.8% [±44.4]
LocalHistoryPerformanceTest#testHistoryCleanUp20x20() - 8.6% [±1.7] - 4.5% [±61.5] - 10.0% [±3.9] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2.96s is not within [0%, 110%] of 1.84s - 61.2% [±69.4] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.99s is not within [0%, 110%] of 799ms - 148.7% [±109.9]
MarkerPerformanceTest#testSetAttributes1() - 1.2% [n/a] +5.6% [n/a] +8.2% [n/a] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 49.81s is not within [0%, 110%] of 6.48s - 668.1% [n/a] +4.1% [n/a]
MarkerPerformanceTest#testSetAttributes2() - 3.6% [n/a] +3.3% [n/a] +5.9% [n/a] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 9.03s is not within [0%, 110%] of 6.73s - 34.1% [n/a] +2.9% [n/a]
PropertyManagerPerformanceTest#testGetProperty100x4() - 0.8% [±1.1] +0.0% [±1.2] - 1.4% [±0.8] - 2.0% [±1.3] - 0.1% [±0.6]
PropertyManagerPerformanceTest#testGetProperty20x20() - 3.0% [±1.2] +6.0% [±16.3] - 0.7% [±0.1] - 2.8% [±0.2] - 4.1% [±0.1]
PropertyManagerPerformanceTest#testGetProperty4x100() - 1.9% [±0.0] - 0.6% [±0.1] - 0.5% [±0.1] - 3.1% [±0.0] - 4.5% [±0.0]
PropertyManagerPerformanceTest#testSetProperty100x4() - 2.8% [±3.3] +0.0% [±2.4] +1.1% [±2.1] - 2.4% [±1.8] +0.7% [±1.5]
PropertyManagerPerformanceTest#testSetProperty20x20() - 0.4% [±0.9] +3.9% [±5.6] +0.6% [±1.4] - 1.9% [±4.4] +0.6% [±1.2]
PropertyManagerPerformanceTest#testSetProperty4x100() +0.0% [±1.7] +0.6% [±4.1] - 2.1% [±2.5] +2.8% [±3.2] - 0.9% [±2.5]
WorkspacePerformanceTest#testCreateResources() +19.9% [±33.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 13s is not within [0%, 110%] of 11.32s - 14.8% [±55.2] +1.5% [±25.7] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.05m is not within [0%, 110%] of 25.84s - 144.1% [±71.0] +6.5% [±18.4]
WorkspacePerformanceTest#testDeleteProject() +28.8% [±24.1] +8.6% [±22.3] +9.0% [±21.1] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 51.05s is not within [0%, 110%] of 9.69s - 426.6% [±127.7] - 1.2% [±2.2]
WorkspacePerformanceTest#testFolderCopy() +9.9% [±16.6] +5.3% [±6.4] +16.7% [±14.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 4.75m is not within [0%, 110%] of 1.48m - 221.3% [±59.5] +3.2% [±12.0]
WorkspacePerformanceTest#testFolderMove() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 5.52s is not within [0%, 110%] of 3.23s - 71.0% [±76.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.42s is not within [0%, 110%] of 1.26s - 12.6% [±13.9] - 9.7% [±10.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 9.04s is not within [0%, 110%] of 2.4s - 276.7% [±57.7] +11.8% [±10.8]
*WorkspacePerformanceTest#testRefreshProject() (vs. N20070215-0010) - 6.6% [±17.8] - 5.9% [±3.1] +6.3% [±9.2] - 1,347.8% [±869.2] - 0.7% [±0.8]
BenchPath#testHash() - 0.9% [±0.1] - 1.3% [±0.3] - 5.1% [±0.2] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 7.41s is not within [0%, 110%] of 2.12s - 248.8% [±50.6] +0.7% [±0.2]
BenchPath#testPathCreation() +2.4% [±0.6] +1.7% [±0.5] +0.0% [±0.4] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 3.19s is not within [0%, 110%] of 1.49s - 114.4% [±30.1] +1.8% [±0.4]
BenchPath#testToOSString() +1.0% [±0.4] - 1.0% [±0.5] - 1.6% [±0.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 3.42s is not within [0%, 110%] of 868ms - 293.4% [±80.2] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 867ms is not within [0%, 110%] of 717ms - 20.9% [±0.4]
BenchPath#testToString() +1.2% [±0.2] - 1.6% [±0.5] - 1.8% [±0.2] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.28s is not within [0%, 110%] of 853ms - 49.9% [±1.5] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 881ms is not within [0%, 110%] of 721ms - 22.2% [±0.5]
ContentTypePerformanceTest#testContentMatching() - 2.0% [±2.1] - 8.2% [±1.8] - 1.6% [±1.4] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.19s is not within [0%, 110%] of 1.03s - 15.4% [±2.9] - 6.1% [±1.9]
ContentTypePerformanceTest#testIsKindOf() +0.9% [±0.9] +0.6% [±0.6] +0.7% [±0.8] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.44s is not within [0%, 110%] of 817ms - 76.5% [±16.2] - 1.4% [±0.6]
ContentTypePerformanceTest#testLoadCatalog() +14.6% [±0.0] +9.0% [±0.0] +10.3% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 165ms is not within [0%, 110%] of 128ms - 29.5% [±0.0] +9.9% [±0.0]
ContentTypePerformanceTest#testNameMatching() +4.3% [±0.6] +0.0% [±0.5] - 5.3% [±0.5] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2s is not within [0%, 110%] of 1.45s - 37.6% [±4.6] - 4.7% [±0.5]
PreferencePerformanceTest#testGetStringCommonPrefixKeys() +3.2% [±0.7] +0.2% [±1.7] +2.5% [±1.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.83s is not within [0%, 110%] of 1.51s - 21.2% [±2.8] +2.2% [±2.1]
PreferencePerformanceTest#testGetStringMisses() +19.4% [±1.1] +16.7% [±0.4] - 2.5% [±0.3] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.37s is not within [0%, 110%] of 1.01s - 34.9% [±7.4] - 6.4% [±0.4]
PreferencePerformanceTest#testGetStringSequentialKeys() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.54s is not within [0%, 110%] of 1.4s - 10.8% [±0.5] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.06s is not within [0%, 110%] of 934ms - 13.5% [±0.9] +0.6% [±0.8] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2.02s is not within [0%, 110%] of 1.64s - 22.6% [±19.2] +12.6% [±1.0]
PreferencePerformanceTest#testGetStringUniqueKeys() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 1.54s is not within [0%, 110%] of 1.22s - 26.6% [±9.7] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 946ms is not within [0%, 110%] of 816ms - 15.9% [±1.1] +1.7% [±0.8] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2.39s is not within [0%, 110%] of 1.5s - 59.2% [±25.6] +16.8% [±1.4]
PreferencePerformanceTest#testPutStringKeys() - 3.5% [±2.0] - 3.9% [±0.9] - 8.5% [±1.7] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 7.1s is not within [0%, 110%] of 2.22s - 220.0% [±96.4] +4.4% [±0.4]
PreferencePerformanceTest#testRemoveStringKeys() +0.0% [±0.4] - 1.7% [±0.8] +0.0% [±0.4] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 113ms is not within [0%, 110%] of 100ms - 13.0% [±1.5] - 6.8% [±1.8]
StartupTest.testApplicationStartup Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 4.64s is not within [0%, 105%] of 2.22s - Explanation comment: Performance decrease caused by the JAR signing. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=180219 for details. - 108.8% [±0.0] +8.4% [±0.0] +16.1% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 17.32s is not within [0%, 105%] of 2.02s - Explanation comment: Performance decrease caused by the JAR signing. See https://bugs.eclipse.org/bugs/show_bug.cgi?id=180219 for details. - 757.1% [±0.0] - 1.6% [±0.0]
UIStartupTest.testUIApplicationStartup Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 17.53s is not within [50%, 105%] of 16.2s - 8.2% [±0.0] Performance criteria not met when compared to '{config=eclipseperflnx2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 11.64s is not within [50%, 105%] of 10.6s - 9.8% [±0.0] Performance criteria not met when compared to '{config=eclipseperflnx3_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 9.16s is not within [50%, 105%] of 8.54s - 7.3% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 37.44s is not within [50%, 105%] of 10.26s - 264.8% [±0.0] Performance criteria not met when compared to '{config=eclipseperfwin2_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 7.56s is not within [50%, 105%] of 6.65s - 13.8% [±0.0]