Performance of org.eclipse.jdt.text: M20150204-1700 relative to R-4.4-201406061215

SUSE Linux Enterprise Server 11 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 94 scenarios	SUSE Linux Enterprise Server 11 (x86_64)
ActivateJavaEditorTest#testActivateEditor()	+5.7% [±2.6]
ActivateTextEditorTest#testActivateEditor()	- 0.6% [±6.9]
CodeCompletionPerformanceTest#testApplicationNoParamters()	+14.3% [±2.5]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 10.4s is not within [0%, 110'%] of 8.3s - 25.5% [±6.4]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses2()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 10.5s is not within [0%, 110'%] of 8.2s - 27.6% [±6.9]
CodeCompletionPerformanceTest#testApplicationWithParamterNames()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 11.1s is not within [0%, 110'%] of 8.3s - 33.9% [±9.7]
CodeCompletionPerformanceTest#testCompletionNoParamters()	+7.6% [±9.0]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses()	- 3.7% [±1.4]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses2()	+3.8% [±9.4]
CodeCompletionPerformanceTest#testCompletionWithParamterNames()	- 2.9% [±2.8]
ContentTypeTest#testJavaDirty()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 23ms is not within [0%, 110'%] of 19ms - 19.3% [±13.3]
ContentTypeTest#testPluginPropertiesDirty()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 93ms is not within [0%, 110'%] of 59ms - 55.4% [±2.0]
ContentTypeTest#testPluginXMLDirty()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1.9s is not within [0%, 110'%] of 1.7s - 16.0% [±2.0]
ContentTypeTest#testPropertiesDirty()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.2s is not within [0%, 110'%] of 72ms - 172.5% [±220.6]
ContentTypeTest#testTextDirty()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 31ms is not within [0%, 110'%] of 18ms - 63.7% [±10.2]
ConvertLineDelimitersProjectTest#test()	+24.3% [±1.7]
DocumentLineDifferInitializationTest#testInitializationWithFewChanges()	- 0.1% [±0.0]
DocumentLineDifferInitializationTest#testInitializationWithManyChanges()	+0.5% [±0.2]
DocumentLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+0.0% [±0.1]
DocumentLineDifferInitializationTest#testInitializationWithNoChanges()	+0.1% [±0.0]
DocumentLineDifferModificationTest#testEditingChanged()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 5ms is not within [0%, 110'%] of 3ms - 48.1% [±31.6]
DocumentLineDifferModificationTest#testEditingUnchanged()	+100.0% [±266.9]
DocumentPerformanceTest#measureDeleteInsert()	- 2.2% [±6.1]
DocumentPerformanceTest#measureGetChar()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 10ms is not within [0%, 110'%] of 6ms - 51.2% [±57.9]
DocumentPerformanceTest#measureGetLength()	+100.0% [±15.2]
DocumentPerformanceTest#measureGetLine()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 27ms is not within [0%, 110'%] of 24ms - 11.9% [±18.5]
DocumentPerformanceTest#measureGetNumberOfLines()	+100.0% [±72.9]
DocumentPerformanceTest#measureInsertAtEnd()	+12.4% [±5.3]
DocumentPerformanceTest#measureInsertAtStart()	+12.3% [±2.2]
DocumentPerformanceTest#measureLineByIndex()	+46.7% [±6.1]
DocumentPerformanceTest#measureLineByOffset()	+2.3% [±2.6]
DocumentPerformanceTest#measureLineDelimiterByIndex()	+17.3% [±13.6]
DocumentPerformanceTest#measureLineInfoByIndex()	+4.9% [±3.9]
DocumentPerformanceTest#measureLineInfoByOffset()	+3.8% [±2.4]
DocumentPerformanceTest#measureLineLengthByIndex()	+9.9% [±8.7]
DocumentPerformanceTest#measureRandomReplace()	- 1.4% [±2.7]
DocumentPerformanceTest#measureRepeatedReplace()	+4.3% [±1.5]
DocumentPerformanceTest#measureSet()	+0.0% [±2.6]
DocumentPerformanceTest#measureTypingReplaceInLargeFile()	+6.1% [±25.8]
DocumentPerformanceTest#measureTypingReplaceInSmallFile()	+100.0% [±118.0]
GapTextStorePerformanceTest#measureDeleteInsert()	+5.0% [±7.0]
GapTextStorePerformanceTest#measureGetAll()	+7.0% [±0.7]
GapTextStorePerformanceTest#measureGetChar()	+0.1% [±1.2]
GapTextStorePerformanceTest#measureGetLine()	+4.4% [±3.9]
GapTextStorePerformanceTest#measureInsertAtEnd()	+9.4% [±5.3]
GapTextStorePerformanceTest#measureInsertAtStart()	+0.9% [±8.1]
GapTextStorePerformanceTest#measureRandomReplace()	+2.5% [±1.3]
GapTextStorePerformanceTest#measureRepeatedReplace()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 9ms is not within [0%, 110'%] of 7ms - 20.3% [±3.2]
GapTextStorePerformanceTest#measureSet()	+0.7% [±0.4]
GapTextStorePerformanceTest#measureTypingReplaceInLargeFile()	+100.0% [±234.5]
GapTextStorePerformanceTest#measureTypingReplaceInSmallFile()	[n/a]
JavaExpandSelectionTest#test()	- 1.8% [±9.7]
JavaFormatterProjectTest#test()	+5.1% [±4.4]
JavaFormatterTest#test()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 2.8s is not within [0%, 110'%] of 2.5s - 12.8% [±11.8]
LineTrackerPerformanceTest#testLineByIndex()	+5.6% [±0.3]
LineTrackerPerformanceTest#testLineByIndex2()	+29.7% [±3.3]
LineTrackerPerformanceTest#testLineByOffset()	- 0.8% [±0.7]
LineTrackerPerformanceTest#testLineByOffset2()	- 3.9% [±0.1]
LineTrackerPerformanceTest#testRandomReplace()	+4.1% [±1.3]
LineTrackerPerformanceTest#testReplace()	+0.0% [±1.9]
LineTrackerPerformanceTest#testSet()	+1.2% [±1.0]
OpenJavaContentAssistTest#test1()-cold	- 3.1% [±7.0]
OpenJavaContentAssistTest#test1()-warm	- 6.3% [±6.3]
OpenJavaEditorStartupTest#testJavaEditorStartup()	+5.6% [n/a]
OpenJavaEditorTest#testOpenEditor3()	- 6.5% [±3.8]
OpenJavaEditorTest#testOpenEditor4()	+0.0% [±4.9]
OpenJavaEditorTest#testOpenEditor5()	- 8.8% [±3.9]
OpenJavaEditorTest#testOpenEditor6()	- 2.8% [±5.8]
OpenJavaEditorTest#testOpenJavaEditor1()	+2.3% [±3.3]
OpenJavaEditorTest#testOpenJavaEditor2()	+0.0% [±4.9]
OpenQuickOutlineTest#testOpenQuickOutline1()-cold	+3.4% [±1.7]
OpenQuickOutlineTest#testOpenQuickOutline1()-warm	+3.0% [±2.1]
OpenTextEditorStartupTest#testTextEditorStartup()	+6.1% [n/a]
OpenTextEditorTest#testOpenEditor3()	+4.4% [±1.7]
OpenTextEditorTest#testOpenTextEditor1()	+1.0% [±3.1]
OpenTextEditorTest#testOpenTextEditor2()	+3.8% [±4.0]
RevertJavaEditorTest#test2()	+15.4% [±6.4]
RevertJavaEditorTest#testRevertJavaEditor()	- 0.8% [±9.1]
RevertTextEditorTest#testRevertTextEditor()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 62ms is not within [0%, 110'%] of 54ms - 14.5% [±13.2]
SaveJavaEditorTest#test1()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 10s is not within [0%, 110'%] of 6.6s - 52.8% [±6.3]
SaveTextEditorTest#test1()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1.3s is not within [0%, 110'%] of 0.93s - 34.7% [±12.7]
ScrollJavaEditorTest#testScrollJavaEditorLineWise2()	+0.8% [±4.3]
ScrollTextEditorTest#testScrollTextEditorLineWise2()	+1.5% [±0.3]
SpellCheckingTest#test()Java Editor: Spell checking	+13.8% [±2.8]
SynchronizedLineDifferInitializationTest#testInitializationWithFewChanges()	+0.0% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChanges()	+0.0% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+0.0% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithNoChanges()	+0.0% [±0.0]
ToggleCommentTest#testToggleComment2()-comment	+35.3% [±0.9]
ToggleCommentTest#testToggleComment2()-uncomment	+25.4% [±3.4]
UndoJavaEditorTest#testUndoJavaEditor2()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 3.4s is not within [0%, 110'%] of 1.9s - 82.6% [±27.2]
UndoTextEditorTest#testUndoTextEditor2()	+17.6% [±11.5]
WhitespaceCharacterPainterTest#testScrollTextEditorLineWise2()	+1.4% [±0.3]
WordRulePerformanceTest#measureIgnoreCase()	+7.7% [±4.2]