Performance of org.eclipse.jdt.text: I20090611-1540 relative to R-3.4-200806172000 (200906121800)

Back to global results

The following fingerprints show results for the most representative tests of the org.eclipse.jdt.text component.

Select which kind of scale you want to use:

Help on fingerprints

Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)

SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)

RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)

Scenario Status

The following table gives a complete but compact view of performance results for the component.
Each line of the table shows the results for one scenario on all machines.

The name of the scenario is in bold when its results are also displayed in the fingerprints
and starts with an '*' when the scenario has no results in the last baseline run.

Here are information displayed for each test (ie. in each cell):

an icon showing whether the test fails or passes and whether it's reliable or not.
The legend for this icon is:
- Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
- Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
- Gray (): mark a failing result (see above) with a comment explaining this degradation.
- Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
- Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
- "n/a": mark a test for with no performance results
the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

Hints:

fly over image of failing tests to see the complete error message
to look at the complete and detailed test results, click on its image

All 113 scenarios	SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)	RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)	Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)
ActivateJavaEditorTest#testActivateEditor()	+73.5% [±0.8]	+68.2% [±0.7]	+80.2% [±0.6]
ActivateTextEditorTest#testActivateEditor()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 2s is not within [0%, 110'%] of 1.7s - 19.7% [±1.4]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 3.9s is not within [0%, 110'%] of 3s - 30.8% [±1.5]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.44s is not within [0%, 110'%] of 0.29s - 52.3% [±1.3]
CodeCompletionPerformanceTest#testApplicationNoParamters()	+1.5% [±1.2]	+2.7% [±1.4]	+3.7% [±1.2]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses()	- 3.8% [±0.8]	- 3.6% [±0.9]	- 2.8% [±1.3]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses2()	+4.8% [±1.1]	+0.6% [±0.8]	+6.8% [±1.5]
CodeCompletionPerformanceTest#testApplicationWithParamterNames()	+3.9% [±1.1]	+2.8% [±0.7]	+5.6% [±1.5]
CodeCompletionPerformanceTest#testCompletionNoParamters()	- 2.9% [±0.7]	- 4.7% [±1.6]	+1.4% [±3.6]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses()	- 5.8% [±0.6]	- 3.1% [±0.6]	- 3.3% [±1.5]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses2()	- 1.5% [±0.4]	- 0.3% [±0.2]	- 2.0% [±0.8]
CodeCompletionPerformanceTest#testCompletionWithParamterNames()	- 4.3% [±1.0]	- 4.1% [±1.0]	- 3.8% [±1.4]
ContentTypeTest#testJavaDirty()	- 3.6% [±4.3]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 60ms is not within [0%, 110'%] of 45ms - 33.3% [±2.8]	+17.9% [±6.9]
ContentTypeTest#testPluginPropertiesDirty()	- 4.8% [±2.8]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 92ms is not within [0%, 110'%] of 70ms - 31.4% [±1.4]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.11s is not within [0%, 110'%] of 87ms - 25.3% [±4.4]
ContentTypeTest#testPluginXMLDirty()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 5.7s is not within [0%, 110'%] of 4.1s - 41.5% [±0.1]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 4.6s is not within [0%, 110'%] of 3.2s - 41.0% [±0.2]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 5.7s is not within [0%, 110'%] of 4.1s - 39.6% [±0.1]
ContentTypeTest#testPropertiesDirty()	- 9.8% [±3.5]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 66ms is not within [0%, 110'%] of 50ms - 32.0% [±2.6]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 81ms is not within [0%, 110'%] of 59ms - 37.3% [±11.2]
ContentTypeTest#testTextDirty()	- 6.0% [±2.3]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 66ms is not within [0%, 110'%] of 47ms - 40.4% [±4.2]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 72ms is not within [0%, 110'%] of 53ms - 35.8% [±10.0]
ConvertLineDelimitersProjectTest#test()	+41.4% [±40.9]	+6.5% [±13.0]	- 4.4% [±35.7]
DocumentLineDifferInitializationTest#testInitializationWithFewChanges()	- 0.5% [±1.5]	+87.5% [±0.0]	+88.6% [±89.6]
DocumentLineDifferInitializationTest#testInitializationWithManyChanges()	+13.6% [±0.0]	- 0.2% [±0.1]	+64.3% [±60.5]
DocumentLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+13.7% [±0.3]	+83.9% [±83.9]	+8.4% [±0.8]
DocumentLineDifferInitializationTest#testInitializationWithNoChanges()	+1.2% [±0.2]	+0.0% [±0.0]	+83.7% [±85.2]
DocumentLineDifferModificationTest#testEditingChanged()	+22.2% [±1.4]	+5.6% [±5.3]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 14ms is not within [0%, 110'%] of 2ms - 600.0% [±133.1]
DocumentLineDifferModificationTest#testEditingUnchanged()	+50.0% [±16.7]	+40.0% [±19.9]	[n/a]
DocumentPerformanceTest#measureDeleteInsert()	+5.3% [±1.8]	+0.0% [±2.0]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 21ms is not within [0%, 110'%] of 19ms - 10.5% [±12.0]
DocumentPerformanceTest#measureGetChar()	+1.4% [±0.7]	- 1.7% [±0.2]	+6.8% [±3.7]
DocumentPerformanceTest#measureGetLength()	+0.0% [±0.3]	+1.4% [±0.2]	+1.2% [±2.4]
DocumentPerformanceTest#measureGetLine()	+0.0% [±1.5]	+0.0% [±1.8]	+13.8% [±7.1]
DocumentPerformanceTest#measureGetNumberOfLines()	+0.0% [±0.2]	+0.0% [±0.1]	+0.0% [±2.9]
DocumentPerformanceTest#measureInsertAtEnd()	+0.0% [±1.9]	+0.0% [±1.6]	- 3.0% [±5.8]
DocumentPerformanceTest#measureInsertAtStart()	+2.7% [±2.4]	+0.0% [±1.1]	+8.0% [±3.1]
DocumentPerformanceTest#measureLineByIndex()	+0.0% [±0.2]	+0.0% [±0.1]	+6.2% [±2.9]
DocumentPerformanceTest#measureLineByOffset()	+1.6% [±0.8]	+0.0% [±0.2]	+2.3% [±2.0]
DocumentPerformanceTest#measureLineDelimiterByIndex()	+0.0% [±0.4]	+0.0% [±0.1]	- 4.0% [±2.6]
DocumentPerformanceTest#measureLineInfoByIndex()	+9.0% [±1.1]	+0.7% [±0.3]	+1.9% [±2.2]
DocumentPerformanceTest#measureLineInfoByOffset()	- 1.4% [±0.9]	- 1.6% [±0.2]	- 1.4% [±1.5]
DocumentPerformanceTest#measureLineLengthByIndex()	+0.0% [±0.2]	+0.0% [±0.2]	+0.0% [±1.8]
DocumentPerformanceTest#measureRandomReplace()	+0.0% [±0.6]	+0.0% [±0.2]	+2.4% [±1.4]
DocumentPerformanceTest#measureRepeatedReplace()	+0.7% [±0.6]	- 0.9% [±0.5]	- 0.7% [±1.8]
DocumentPerformanceTest#measureSet()	- 2.1% [±1.8]	+2.6% [±1.3]	- 2.3% [±3.8]
DocumentPerformanceTest#measureTypingReplaceInLargeFile()	+0.0% [±0.6]	+2.0% [±0.3]	- 2.1% [±1.9]
DocumentPerformanceTest#measureTypingReplaceInSmallFile()	- 0.7% [±0.5]	+0.0% [±0.3]	+0.7% [±1.6]
GapTextStorePerformanceTest#measureDeleteInsert()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 13ms is not within [0%, 110'%] of 11ms - 18.2% [±6.2]	+0.0% [±1.9]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 11ms is not within [0%, 110'%] of 10ms - 10.0% [±22.7]
GapTextStorePerformanceTest#measureGetAll()	+0.0% [±0.1]	+0.0% [±0.1]	- 0.7% [±1.6]
GapTextStorePerformanceTest#measureGetChar()	+0.0% [±0.3]	+0.5% [±0.1]	+0.4% [±0.8]
GapTextStorePerformanceTest#measureGetLine()	+0.7% [±0.6]	- 0.7% [±0.3]	+1.5% [±1.8]
GapTextStorePerformanceTest#measureInsertAtEnd()	+0.0% [±1.5]	- 5.0% [±1.6]	+0.0% [±11.5]
GapTextStorePerformanceTest#measureInsertAtStart()	+0.0% [±3.1]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 11ms is not within [0%, 110'%] of 10ms - 10.0% [±3.3]	+0.0% [±23.3]
GapTextStorePerformanceTest#measureRandomReplace()	+0.0% [±0.3]	- 1.0% [±0.2]	+0.0% [±2.0]
GapTextStorePerformanceTest#measureRepeatedReplace()	+0.0% [±0.7]	+0.0% [±0.5]	+0.0% [±13.0]
GapTextStorePerformanceTest#measureSet()	+0.0% [±0.4]	+2.1% [±0.4]	+2.7% [±6.6]
GapTextStorePerformanceTest#measureTypingReplaceInLargeFile()	- 4.2% [±2.7]	+4.0% [±1.5]	- 4.2% [±10.5]
GapTextStorePerformanceTest#measureTypingReplaceInSmallFile()	+0.0% [±0.7]	- 8.3% [±1.3]	+0.0% [±10.5]
JavaExpandSelectionTest#test()	+5.5% [±0.9]	+0.0% [±0.3]	- 2.5% [±4.8]
JavaFormatterProjectTest#test()	- 8.2% [±5.5]	- 1.6% [±6.9]	+10.8% [±17.2]
JavaFormatterTest#test()	+5.0% [±0.5]	+0.0% [±0.3]	- 2.5% [±5.2]
JavaIndenterTest#testJavaIndenter2()	- 7.7% [±3.8]	- 4.5% [±18.7]	n/a
*JavaMoveLineTest#test() (vs. R-3.4-200806172000_200905291800)	n/a	+79.2% [±21.3]	n/a
JavaReplaceAllWithQuickDiffTest#test()	- 0.5% [±9.1]	+13.7% [±1.1]	n/a
LineTrackerPerformanceTest#testLineByIndex()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.76s is not within [0%, 110'%] of 0.65s - 16.3% [±0.8]	+23.6% [±0.3]	- 0.3% [±0.6]
LineTrackerPerformanceTest#testLineByIndex2()	+0.0% [±0.2]	+0.0% [±0.1]	+4.6% [±1.0]
LineTrackerPerformanceTest#testLineByOffset()	+0.3% [±0.2]	- 0.7% [±0.4]	- 0.3% [±0.6]
LineTrackerPerformanceTest#testLineByOffset2()	- 0.1% [±0.0]	+0.3% [±0.0]	+0.7% [±0.1]
LineTrackerPerformanceTest#testRandomReplace()	+0.3% [±0.1]	+0.8% [±0.1]	+0.3% [±1.0]
LineTrackerPerformanceTest#testReplace()	+0.0% [±0.4]	+0.7% [±0.4]	+0.0% [±1.3]
LineTrackerPerformanceTest#testSet()	- 1.3% [±0.4]	+0.0% [±0.2]	- 2.1% [±1.6]
NonInitialTypingTest#testTypeAMethod()	+58.8% [±25.8]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.14s is not within [0%, 110'%] of 1ms - 14,000.0% [±13,983.4]	n/a
OpenJavaContentAssistTest#test1()-cold	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.26s is not within [0%, 110'%] of 0.23s - 10.7% [±4.6]	+6.7% [±4.3]	+61.0% [±4.0]
OpenJavaContentAssistTest#test1()-warm	+43.3% [±2.8]	+28.6% [±2.2]	+52.0% [±3.7]
OpenJavaEditorStartupTest#testJavaEditorStartup()	+1.8% [n/a]	- 2.8% [n/a]	- 1.4% [n/a]
OpenJavaEditorTest#testOpenEditor3()	+6.2% [±1.8]	+5.1% [±2.2]	+0.4% [±3.7]
OpenJavaEditorTest#testOpenEditor4()	+2.3% [±2.5]	+8.6% [±2.2]	- 0.9% [±3.5]
OpenJavaEditorTest#testOpenEditor5()	+17.3% [±3.2]	+18.9% [±2.6]	+7.5% [±4.1]
OpenJavaEditorTest#testOpenEditor6()	+9.5% [±3.7]	+15.1% [±3.7]	- 1.9% [±4.9]
OpenJavaEditorTest#testOpenJavaEditor1()	+6.1% [±6.2]	+18.0% [±4.2]	+4.8% [±8.6]
OpenJavaEditorTest#testOpenJavaEditor2()	+5.6% [±5.1]	+10.3% [±4.2]	- 2.4% [±7.7]
OpenPreferencePageTest#testOpenPreferencePage()-cold	+21.5% [n/a]	- 5.8% [n/a]	n/a
OpenQuickOutlineTest#testOpenQuickOutline1()-cold	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.29s is not within [0%, 110'%] of 0.24s - 18.1% [±0.7]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.37s is not within [0%, 110'%] of 0.31s - 21.2% [±2.1]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.14s is not within [0%, 110'%] of 0.12s - 10.4% [±2.5]
OpenQuickOutlineTest#testOpenQuickOutline1()-warm	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.29s is not within [0%, 110'%] of 0.25s - 16.3% [±1.7]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.35s is not within [0%, 110'%] of 0.31s - 13.1% [±2.4]	+1.6% [±2.2]
OpenTextEditorStartupTest#testTextEditorStartup()	- 3.9% [n/a]	- 0.9% [n/a]	+0.6% [n/a]
OpenTextEditorTest#testOpenEditor3()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.15s is not within [0%, 110'%] of 0.12s - 19.2% [±0.4]	+7.4% [±1.4]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 94ms is not within [0%, 110'%] of 62ms - 51.6% [±0.4]
OpenTextEditorTest#testOpenTextEditor1()	- 8.5% [±2.5]	+1.5% [±1.4]	- 1.5% [±1.8]
OpenTextEditorTest#testOpenTextEditor2()	- 8.9% [±3.8]	+0.0% [±1.3]	- 1.6% [±7.6]
RevertJavaEditorTest#test2()	+11.7% [±3.8]	+3.4% [±3.6]	+31.1% [±6.3]
RevertJavaEditorTest#testRevertJavaEditor()	+16.9% [±5.1]	+12.8% [±3.3]	+58.0% [±45.4]
RevertTextEditorTest#testRevertTextEditor()	+0.0% [±16.0]	+7.4% [±12.5]	+0.0% [±25.2]
SaveJavaEditorTest#test1()	- 0.6% [±3.5]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.41s is not within [0%, 110'%] of 0.26s - 56.7% [±50.1]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.88s is not within [0%, 110'%] of 0.39s - 125.9% [±84.7]
SaveTextEditorTest#test1()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.49s is not within [0%, 110'%] of 0.43s - 14.4% [±37.8]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.39s is not within [0%, 110'%] of 0.3s - 31.1% [±42.4]	+25.5% [±21.8]
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWise1()	+3.9% [±0.3]	+2.1% [±0.0]	n/a
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWiseMoveCaret1()	+0.0% [±0.1]	+0.1% [±0.1]	n/a
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWiseSelect1()	+0.4% [±0.2]	- 0.6% [±0.0]	n/a
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorPageWise()	- 0.3% [±0.1]	- 2.8% [±0.1]	n/a
ScrollJavaEditorTest#testScrollJavaEditorLineWise2()	+4.9% [±0.2]	+1.4% [±0.1]	n/a
ScrollJavaEditorTest#testScrollJavaEditorLineWiseMoveCaret2()	+0.6% [±0.1]	+0.4% [±0.1]	n/a
ScrollJavaEditorTest#testScrollJavaEditorLineWiseSelect2()	+0.7% [±0.1]	- 0.6% [±0.0]	n/a
ScrollJavaEditorTest#testScrollJavaEditorPageWise()	+2.2% [±0.3]	- 1.2% [±0.1]	n/a
ScrollTextEditorTest#testScrollTextEditorLineWise2()	- 0.3% [±0.3]	- 1.6% [±0.2]	n/a
ScrollTextEditorTest#testScrollTextEditorLineWiseMoveCaret2()	- 3.8% [±0.1]	- 3.8% [±0.1]	n/a
ScrollTextEditorTest#testScrollTextEditorLineWiseSelect2()	- 2.8% [±0.2]	- 3.4% [±0.0]	n/a
ScrollTextEditorTest#testScrollTextEditorPageWise()	- 1.6% [±0.1]	+0.2% [±0.0]	n/a
ScrollVerticalRuler1000Test#testScrollTextEditorLineWiseMoveCaret2()	- 1.2% [±0.1]	- 4.2% [±0.1]	n/a
ScrollVerticalRuler1000Test#testScrollTextEditorPageWise()	+0.0% [±0.5]	- 0.3% [±0.5]	n/a
ScrollVerticalRuler100Test#testScrollTextEditorLineWiseMoveCaret2()	- 3.0% [±0.8]	- 3.0% [±0.3]	n/a
ScrollVerticalRuler100Test#testScrollTextEditorPageWise()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.13s is not within [0%, 110'%] of 0.1s - 21.9% [±13.4]	+29.8% [±8.2]	n/a
ScrollVerticalRuler5000Test#testScrollTextEditorPageWise()	+1.4% [±0.2]	+1.2% [±0.2]	n/a
SynchronizedLineDifferInitializationTest#testInitializationWithFewChanges()	+0.0% [±0.0]	+87.5% [±0.0]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.6s is not within [0%, 110'%] of 0.55s - 10.1% [±0.5]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChanges()	+14.3% [±0.0]	+0.2% [±0.1]	+9.8% [±1.5]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+14.1% [±0.2]	+0.0% [±0.6]	+9.7% [±0.7]
SynchronizedLineDifferInitializationTest#testInitializationWithNoChanges()	- 0.7% [±0.2]	+0.0% [±0.0]	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.6s is not within [0%, 110'%] of 0.55s - 10.4% [±0.8]
TextMoveLineTest#test()	+44.4% [±46.8]	+6.6% [±35.7]	n/a
TextNonInitialTypingTest#testTypeAMethod()	Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 53ms is not within [0%, 110'%] of 1ms - 5,200.0% [±2,617.0]	+0.0% [±47.1]	n/a
ToggleCommentTest#testToggleComment2()-comment	+18.6% [±2.4]	+3.3% [±0.5]	+0.0% [±0.4]
ToggleCommentTest#testToggleComment2()-uncomment	+20.7% [±3.3]	+6.3% [±2.0]	+4.4% [±4.5]
UndoJavaEditorTest#testUndoJavaEditor2()	+8.7% [±1.8]	- 0.5% [±0.9]	+5.1% [±0.3]
UndoTextEditorTest#testUndoTextEditor2()	+15.5% [±18.2]	+60.5% [±63.0]	+13.8% [±14.2]

Performance of org.eclipse.jdt.text: I20090611-1540 relative to R-3.4-200806172000 (200906121800)

Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)

SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)

RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)

Scenario Status

All 113 scenarios

SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)

RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)

Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)