Performance of org.eclipse.jdt.core: M20071023-1652 relative to 3.2

RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)

Win XP Sun 1.4.2_10 (3 GHz 2 GB)

RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)

Win XP Sun 1.4.2_10 (2 GHz 512 MB)

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)


Scenario Status

The following table gives a complete but compact view of performance results for the component.
Each line of the table shows the results for one scenario on all machines.

The name of the scenario is in bold when its results are also displayed in the fingerprints
and starts with an '*' when the scenario has no results in the last baseline run.

Here are information displayed for each test (ie. in each cell): Hints:

All 56 scenarios

RHEL 3.0 Sun 1.4.2_10 (2 GHz 512 MB)
RHEL 3.0 Sun 1.4.2_10 (3 GHz 2 GB)
RHEL 4.0 Sun 1.4.2_10 (3 GHz 2.5 GB)
Win XP Sun 1.4.2_10 (2 GHz 512 MB)
Win XP Sun 1.4.2_10 (3 GHz 2 GB)
FullSourceWorkspaceASTTests#testDomAstCreationProjectJLS3() +10.7% [±0.3] +13.2% [±0.9] +11.5% [±0.1] +8.9% [±0.2] +12.3% [±0.2]
FullSourceWorkspaceASTTests#testPerfDomAstCreationJLS2() +16.1% [±2.3] +18.2% [±1.7] +15.7% [±2.0] +15.0% [±1.8] +17.5% [±1.6]
FullSourceWorkspaceASTTests#testPerfDomAstCreationJLS3() +18.2% [±0.3] +18.8% [±0.3] +16.3% [±0.2] +15.3% [±0.2] +16.7% [±0.1]
FullSourceWorkspaceASTTests#testWkspDomAstCreationJLS2() +4.1% [n/a] +9.1% [n/a] +7.4% [n/a] Performance criteria not met when compared to '{config=eclipseperfwin1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 2.29m is not within [0%, 110%] of 2.05m - 11.6% [n/a] +6.5% [n/a]
FullSourceWorkspaceBuildTests#testBatchCompilerNoWarning() - 0.3% [±1.4] +1.3% [±1.4] +2.8% [±0.7] +1.8% [±0.5] - 4.1% [±1.5]
FullSourceWorkspaceBuildTests#testCompileJDTCoreProjectAllWarnings() +1.0% [±1.0] +1.5% [±1.7] +1.3% [±0.1] +0.2% [±0.3] +1.3% [±0.6]
FullSourceWorkspaceBuildTests#testCompileJDTCoreProjectDefault() +0.6% [±1.7] +4.7% [±1.2] +3.4% [±0.2] +2.6% [±0.3] +1.7% [±0.6]
FullSourceWorkspaceBuildTests#testCompileJDTCoreProjectJavadoc() +3.1% [±1.4] +3.4% [±1.5] +3.3% [±0.2] +2.0% [±0.2] +2.4% [±0.6]
FullSourceWorkspaceBuildTests#testCompileJDTCoreProjectNoWarning() +0.3% [±1.7] +2.9% [±0.5] +2.6% [±0.1] +2.1% [±0.4] +0.8% [±1.4]
FullSourceWorkspaceBuildTests#testCompileSWTProjectDefault() - 2.4% [±1.9] +1.6% [±1.2] +2.7% [±0.1] +0.9% [±0.2] +1.2% [±0.5]
FullSourceWorkspaceBuildTests#testFullBuildDefault() - 0.6% [n/a] +0.8% [n/a] - 0.8% [n/a] +3.8% [n/a] - 0.4% [n/a]
FullSourceWorkspaceBuildTests#testFullBuildProjectAllWarnings() - 4.6% [±2.9] - 0.8% [±0.4] - 3.3% [±2.6] - 1.6% [±0.4] - 1.9% [±1.3]
FullSourceWorkspaceBuildTests#testFullBuildProjectDefault() - 0.2% [±0.5] +0.6% [±0.6] +1.0% [±0.2] +4.5% [±3.9] - 1.3% [±1.4]
FullSourceWorkspaceBuildTests#testFullBuildProjectNoWarning() - 0.5% [±0.4] +1.1% [±0.6] +1.1% [±0.2] +6.5% [±5.6] +0.4% [±1.4]
FullSourceWorkspaceBuildTests#testParser() - 2.0% [±0.6] +4.6% [±0.6] +1.6% [±0.2] +0.7% [±0.2] +3.0% [±1.2]
FullSourceWorkspaceBuildTests#testScanner() Performance criteria not met when compared to '{config=eclipseperflnx1_R3.3, build=3.2_200606291905_200707261759, jvm=sun}': Elapsed Process: 7.52s is not within [0%, 110%] of 5.7s - 32.0% [±2.2] - 6.7% [±0.1] - 2.7% [±0.2] - 4.3% [±0.3] +0.4% [±0.1]
FullSourceWorkspaceCompletionTests#testPerfCompleteEmptyName() - 1.7% [±0.5] - 0.3% [±0.4] - 3.2% [±0.4] - 2.0% [±0.4] - 3.8% [±0.3]
FullSourceWorkspaceCompletionTests#testPerfCompleteEmptyNameWithoutMethods() - 1.0% [±0.4] - 2.4% [±1.1] - 4.6% [±1.1] - 3.3% [±0.4] - 4.2% [±0.4]
FullSourceWorkspaceCompletionTests#testPerfCompleteEmptyNameWithoutTypes() +0.7% [±0.4] +0.0% [±0.5] +4.3% [±4.3] - 1.5% [±0.3] - 7.8% [±4.1]
FullSourceWorkspaceCompletionTests#testPerfCompleteMemberAccess() +4.8% [±0.5] +5.7% [±0.4] +5.0% [±0.4] +3.7% [±0.4] +1.9% [±0.4]
FullSourceWorkspaceCompletionTests#testPerfCompleteMethodDeclaration() +3.6% [±1.3] +4.3% [±1.2] +2.4% [±1.3] +4.5% [±2.6] +0.8% [±1.1]
FullSourceWorkspaceCompletionTests#testPerfCompleteName() +6.5% [±0.2] +26.3% [±1.3] +17.6% [±0.2] +6.4% [±1.8] +24.7% [±1.3]
FullSourceWorkspaceCompletionTests#testPerfCompleteNameWithoutMethods() +6.6% [±0.4] +26.0% [±0.2] +19.7% [±0.3] +9.7% [±0.4] +24.7% [±0.2]
FullSourceWorkspaceCompletionTests#testPerfCompleteNameWithoutTypes() +0.1% [±0.6] - 0.6% [±0.6] - 9.0% [±3.9] - 2.1% [±0.5] - 2.4% [±0.5]
FullSourceWorkspaceCompletionTests#testPerfCompleteTypeReference() +7.2% [±3.8] +26.1% [±0.3] +20.8% [±0.2] +10.6% [±0.2] +24.7% [±0.1]
FullSourceWorkspaceFormatterTests#testFormatDefault() +41.4% [±0.9] +38.3% [±0.8] +41.5% [±0.8] +44.0% [±0.7] +39.7% [±0.6]
FullSourceWorkspaceFormatterTests#testFormatDefaultBigFile() +65.3% [±0.1] +59.4% [±0.1] +65.5% [±0.0] +67.8% [±0.1] +60.9% [±0.2]
FullSourceWorkspaceModelTests#testCloseProjects() +29.0% [±7.2] +28.1% [±5.8] +21.3% [±10.3] +18.3% [±6.4] +20.9% [±3.0]
FullSourceWorkspaceModelTests#testCreateJavaElement() +86.5% [±0.5] +86.5% [±0.3] +90.6% [±0.3] +84.7% [±0.2] +85.0% [±0.2]
FullSourceWorkspaceModelTests#testFindType() +93.3% [±0.1] +93.5% [±0.2] +93.6% [±0.1] +94.4% [±0.0] +94.1% [±0.0]
FullSourceWorkspaceModelTests#testInitJDTPlugin() +9.0% [±4.6] +16.2% [±1.0] +11.7% [±0.6] +15.8% [±7.7] +16.1% [±0.3]
FullSourceWorkspaceModelTests#testNameLookupFindKnownSecondaryType() - 1.8% [±0.9] - 0.4% [±0.8] - 1.3% [±0.7] - 0.9% [±0.6] +0.2% [±0.7]
FullSourceWorkspaceModelTests#testNameLookupFindKnownType() +2.5% [±1.6] +2.7% [±1.6] +0.5% [±1.2] +2.5% [±1.2] +7.3% [±1.1]
FullSourceWorkspaceModelTests#testNameLookupFindUnknownType() +2.0% [±2.0] +0.0% [±1.0] - 1.5% [±0.7] +2.0% [±0.7] +0.0% [±0.4]
FullSourceWorkspaceModelTests#testPerfReconcile() +3.3% [±0.5] +7.0% [±0.6] +5.4% [±0.2] +4.9% [±0.4] +5.8% [±0.3]
FullSourceWorkspaceModelTests#testPerfReconcileBigFileWithSyntaxError() - 0.3% [±0.6] +0.2% [±0.9] +1.1% [±0.3] +0.0% [±0.6] +0.0% [±0.4]
FullSourceWorkspaceModelTests#testPerfSearchAllTypeNamesAndReconcile() +5.9% [±0.3] +11.7% [±0.6] +9.4% [±0.2] +5.4% [±0.0] +10.0% [±0.2]
FullSourceWorkspaceModelTests#testProjectFindKnownMemberType() - 1.2% [±0.6] +0.0% [±1.2] +2.1% [±0.7] +0.0% [±0.1] - 4.3% [±1.0]
FullSourceWorkspaceModelTests#testProjectFindKnownSecondaryType() - 1.7% [±0.7] - 1.1% [±0.8] +2.7% [±0.5] - 7.1% [±0.5] - 4.2% [±0.4]
FullSourceWorkspaceModelTests#testProjectFindKnownType() +2.7% [±0.9] +2.4% [±0.9] +1.3% [±0.9] +1.4% [±1.0] +0.2% [±0.4]
FullSourceWorkspaceModelTests#testProjectFindUnknownType() - 3.1% [±0.5] - 1.6% [±0.6] +0.0% [±0.5] - 7.0% [±0.3] - 6.4% [±0.6]
FullSourceWorkspaceModelTests#testSeekPackageFragments() +0.0% [±1.8] - 7.4% [±2.4] - 0.6% [±1.6] - 4.2% [±2.9] - 1.7% [±2.5]
FullSourceWorkspaceSearchTests#testIndexing() +5.7% [n/a] +16.1% [n/a] +11.3% [n/a] +3.5% [n/a] +12.4% [n/a]
FullSourceWorkspaceSearchTests#testIndexingOneProject() +2.9% [±0.4] +9.5% [±0.5] +9.5% [±0.5] +2.9% [±0.3] +7.7% [±0.1]
FullSourceWorkspaceSearchTests#testNewSearchAllTypeNames() +32.8% [±0.4] +54.9% [±2.1] +47.2% [±0.8] +31.8% [±0.1] +55.5% [±0.4]
*FullSourceWorkspaceSearchTests#testSearchAllTypeNameMatches() (vs. I20070310-1638) +0.2% [±0.3] +1.3% [±1.0] - 2.3% [±0.2] - 2.9% [±0.0] - 3.3% [±0.0]
FullSourceWorkspaceSearchTests#testSearchAllTypeNames() +16.7% [±0.7] +33.1% [±0.4] +27.1% [±0.2] +15.1% [±0.3] +31.9% [±0.1]
FullSourceWorkspaceSearchTests#testSearchBinaryMethod() +3.0% [±0.2] +12.4% [±1.7] +8.8% [±0.2] +3.9% [±0.3] +9.9% [±0.4]
FullSourceWorkspaceSearchTests#testSearchConstructor() - 3.8% [±0.6] +0.6% [±0.6] +0.5% [±0.5] - 0.4% [±0.1] +0.1% [±0.2]
FullSourceWorkspaceSearchTests#testSearchField() +0.2% [±0.3] +4.1% [±0.8] +3.9% [±0.2] +1.1% [±0.1] +4.3% [±0.2]
FullSourceWorkspaceSearchTests#testSearchMethod() +1.4% [±0.7] +5.7% [±0.9] +6.6% [±0.5] +2.0% [±0.4] +5.0% [±0.4]
FullSourceWorkspaceSearchTests#testSearchPackageDeclarations() +97.0% [±0.1] +96.5% [±0.7] +96.5% [±0.3] +100.0% [±0.8] +100.0% [±0.1]
FullSourceWorkspaceSearchTests#testSearchPackageDeclarationsWorkspace() +62.8% [±0.1] +72.0% [±0.3] +72.3% [±0.1] +52.0% [±0.2] +63.7% [±0.1]
FullSourceWorkspaceSearchTests#testSearchType() +0.2% [±0.6] +4.5% [±0.7] +3.3% [±0.3] +0.6% [±0.3] +3.5% [±0.3]
FullSourceWorkspaceTypeHierarchyTests#testPerfAllTypes() +13.4% [±0.2] +26.2% [±0.7] +22.5% [±0.3] +12.6% [±0.2] +25.4% [±0.4]
FullSourceWorkspaceTypeHierarchyTests#testPerfClassWithPotentialSubinterfaces() +26.6% [±1.5] +49.8% [±1.5] +39.5% [±0.5] +26.6% [±0.4] +52.5% [±0.5]