Performance of org.eclipse.jdt.text: I20150603-2000 relative to R-4.4-201406061215

SUSE Linux Enterprise Server 11 (x86_64)

Scenario Status

• an icon showing whether the test fails or passes and whether it's reliable or not.
  The legend for this icon is:
  • Green (): mark a successful result, which means this test has neither significant performance regression nor significant standard error
  • Red (): mark a failing result, which means this test shows a significant performance regression (more than 10%)
  • Gray (): mark a failing result (see above) with a comment explaining this degradation.
  • Yellow ( or ): mark a failing or successful result with a significant standard error (more than 3%)
  • Black (): mark an undefined result, which means that deviation on this test is not a number (NaN) or is infinite (happens when the reference value is equals to 0!)
  • "n/a": mark a test for with no performance results
• the value of the deviation from the baseline as a percentage (ie. formula is: (build_test_time - baseline_test_time) / baseline_test_time)
• the value of the standard error of this deviation as a percentage (ie. formula is: sqrt(build_test_stddev^2 / N + baseline_test_stddev^2 / N) / baseline_test_time)
  When test only has one measure, the standard error cannot be computed and is replaced with a '[n/a]'.

• fly over image of failing tests to see the complete error message
• to look at the complete and detailed test results, click on its image

All 94 scenarios	SUSE Linux Enterprise Server 11 (x86_64)
ActivateJavaEditorTest#testActivateEditor()	+1.6% [±7.1]
ActivateTextEditorTest#testActivateEditor()	+7.0% [±1.4]
CodeCompletionPerformanceTest#testApplicationNoParamters()	+9.3% [±3.9]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1mn 21s is not within [0%, 110'%] of 9.9s - 725.4% [±72.8]
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses2()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1mn 19s is not within [0%, 110'%] of 9.9s - 699.6% [±59.1]
CodeCompletionPerformanceTest#testApplicationWithParamterNames()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1mn 23s is not within [0%, 110'%] of 10.4s - 698.8% [±68.4]
CodeCompletionPerformanceTest#testCompletionNoParamters()	+1.0% [±7.7]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.31s is not within [0%, 110'%] of 0.26s - 19.7% [±2.1]
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses2()	+12.3% [±8.5]
CodeCompletionPerformanceTest#testCompletionWithParamterNames()	+3.3% [±2.2]
ContentTypeTest#testJavaDirty()	+23.4% [±8.5]
ContentTypeTest#testPluginPropertiesDirty()	+42.1% [±9.7]
ContentTypeTest#testPluginXMLDirty()	+3.5% [±13.3]
ContentTypeTest#testPropertiesDirty()	- 4.1% [±12.0]
ContentTypeTest#testTextDirty()	+39.3% [±0.7]
ConvertLineDelimitersProjectTest#test()	+16.2% [±2.4]
DocumentLineDifferInitializationTest#testInitializationWithFewChanges()	- 0.2% [±0.1]
DocumentLineDifferInitializationTest#testInitializationWithManyChanges()	+0.5% [±0.1]
DocumentLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+0.6% [±0.2]
DocumentLineDifferInitializationTest#testInitializationWithNoChanges()	+0.0% [±0.1]
DocumentLineDifferModificationTest#testEditingChanged()	+21.6% [±24.5]
DocumentLineDifferModificationTest#testEditingUnchanged()	+100.0% [±658.3]
DocumentPerformanceTest#measureDeleteInsert()	+51.2% [±2.7]
DocumentPerformanceTest#measureGetChar()	+68.0% [±9.9]
DocumentPerformanceTest#measureGetLength()	+100.0% [±8.5]
DocumentPerformanceTest#measureGetLine()	- 9.5% [±7.9]
DocumentPerformanceTest#measureGetNumberOfLines()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1ms is not within [0%, 110'%] of 0ms - 40.3% [±60.6]
DocumentPerformanceTest#measureInsertAtEnd()	- 3.0% [±8.2]
DocumentPerformanceTest#measureInsertAtStart()	+2.4% [±5.9]
DocumentPerformanceTest#measureLineByIndex()	+22.6% [±9.7]
DocumentPerformanceTest#measureLineByOffset()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 38ms is not within [0%, 110'%] of 31ms - 22.5% [±10.0]
DocumentPerformanceTest#measureLineDelimiterByIndex()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 9ms is not within [0%, 110'%] of 5ms - 53.2% [±43.9]
DocumentPerformanceTest#measureLineInfoByIndex()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 54ms is not within [0%, 110'%] of 21ms - 150.3% [±9.7]
DocumentPerformanceTest#measureLineInfoByOffset()	- 0.2% [±4.1]
DocumentPerformanceTest#measureLineLengthByIndex()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 7ms is not within [0%, 110'%] of 5ms - 17.3% [±36.6]
DocumentPerformanceTest#measureRandomReplace()	+1.5% [±1.5]
DocumentPerformanceTest#measureRepeatedReplace()	+4.8% [±1.0]
DocumentPerformanceTest#measureSet()	+10.9% [±0.5]
DocumentPerformanceTest#measureTypingReplaceInLargeFile()	- 9.4% [±66.4]
DocumentPerformanceTest#measureTypingReplaceInSmallFile()	+100.0% [±23.5]
GapTextStorePerformanceTest#measureDeleteInsert()	+14.8% [±6.3]
GapTextStorePerformanceTest#measureGetAll()	+9.0% [±0.6]
GapTextStorePerformanceTest#measureGetChar()	+100.0% [±0.8]
GapTextStorePerformanceTest#measureGetLine()	+0.1% [±7.7]
GapTextStorePerformanceTest#measureInsertAtEnd()	+7.8% [±6.0]
GapTextStorePerformanceTest#measureInsertAtStart()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 12ms is not within [0%, 110'%] of 6ms - 76.6% [±14.9]
GapTextStorePerformanceTest#measureRandomReplace()	- 5.9% [±4.5]
GapTextStorePerformanceTest#measureRepeatedReplace()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 19ms is not within [0%, 110'%] of 7ms - 142.7% [±11.8]
GapTextStorePerformanceTest#measureSet()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 27ms is not within [0%, 110'%] of 24ms - 11.5% [±0.4]
GapTextStorePerformanceTest#measureTypingReplaceInLargeFile()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1ms is not within [0%, 110'%] of 0ms - 47.5% [±161.7]
GapTextStorePerformanceTest#measureTypingReplaceInSmallFile()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1ms is not within [0%, 110'%] of 0ms - 117.5% [±369.5]
JavaExpandSelectionTest#test()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.26s is not within [0%, 110'%] of 0.21s - 21.2% [±12.3]
JavaFormatterProjectTest#test()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 2.9s is not within [0%, 110'%] of 2.4s - 20.0% [±7.3]
JavaFormatterTest#test()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 2.7s is not within [0%, 110'%] of 2.3s - 17.3% [±1.1]
LineTrackerPerformanceTest#testLineByIndex()	+6.6% [±0.6]
LineTrackerPerformanceTest#testLineByIndex2()	+20.0% [±5.7]
LineTrackerPerformanceTest#testLineByOffset()	+7.8% [±1.3]
LineTrackerPerformanceTest#testLineByOffset2()	+10.3% [±0.7]
LineTrackerPerformanceTest#testRandomReplace()	+0.6% [±0.6]
LineTrackerPerformanceTest#testReplace()	+3.1% [±1.4]
LineTrackerPerformanceTest#testSet()	+3.9% [±2.1]
OpenJavaContentAssistTest#test1()-cold	+10.8% [±4.5]
OpenJavaContentAssistTest#test1()-warm	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.17s is not within [0%, 110'%] of 0.13s - 29.9% [±4.1]
OpenJavaEditorStartupTest#testJavaEditorStartup()	- 8.5% [n/a]
OpenJavaEditorTest#testOpenEditor3()	- 9.4% [±3.7]
OpenJavaEditorTest#testOpenEditor4()	- 1.7% [±4.4]
OpenJavaEditorTest#testOpenEditor5()	+1.2% [±2.8]
OpenJavaEditorTest#testOpenEditor6()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 0.35s is not within [0%, 110'%] of 0.31s - 14.5% [±4.4]
OpenJavaEditorTest#testOpenJavaEditor1()	- 1.0% [±2.6]
OpenJavaEditorTest#testOpenJavaEditor2()	+1.7% [±3.4]
OpenQuickOutlineTest#testOpenQuickOutline1()-cold	+0.1% [±1.9]
OpenQuickOutlineTest#testOpenQuickOutline1()-warm	- 8.0% [±3.3]
OpenTextEditorStartupTest#testTextEditorStartup()	- 4.4% [n/a]
OpenTextEditorTest#testOpenEditor3()	+1.2% [±2.1]
OpenTextEditorTest#testOpenTextEditor1()	+5.3% [±1.9]
OpenTextEditorTest#testOpenTextEditor2()	+0.0% [±2.2]
RevertJavaEditorTest#test2()	+0.9% [±8.1]
RevertJavaEditorTest#testRevertJavaEditor()	+2.2% [±1.5]
RevertTextEditorTest#testRevertTextEditor()	+17.7% [±12.6]
SaveJavaEditorTest#test1()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 4.9s is not within [0%, 110'%] of 3.8s - 30.6% [±7.0]
SaveTextEditorTest#test1()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 1s is not within [0%, 110'%] of 0.66s - 57.7% [±15.0]
ScrollJavaEditorTest#testScrollJavaEditorLineWise2()	+2.8% [±5.3]
ScrollTextEditorTest#testScrollTextEditorLineWise2()	+2.1% [±1.2]
SpellCheckingTest#test()Java Editor: Spell checking	+15.4% [±5.7]
SynchronizedLineDifferInitializationTest#testInitializationWithFewChanges()	- 0.1% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChanges()	+0.0% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()	+0.0% [±0.1]
SynchronizedLineDifferInitializationTest#testInitializationWithNoChanges()	- 0.2% [±0.0]
ToggleCommentTest#testToggleComment2()-comment	- 4.8% [±19.6]
ToggleCommentTest#testToggleComment2()-uncomment	- 1.7% [±18.1]
UndoJavaEditorTest#testUndoJavaEditor2()	Performance criteria not met when compared to 'R-4.4-201406061215': Elapsed Process= 2.6s is not within [0%, 110'%] of 1.7s - 53.3% [±3.0]
UndoTextEditorTest#testUndoTextEditor2()	- 6.3% [±15.4]
WhitespaceCharacterPainterTest#testScrollTextEditorLineWise2()	+4.7% [±0.6]
WordRulePerformanceTest#measureIgnoreCase()	+9.6% [±6.7]