All 113 scenarios |
SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)
| RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)
| Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)
|
ActivateJavaEditorTest#testActivateEditor()
|
+73.5% [±0.8]
|
+68.2% [±0.7]
|
+80.2% [±0.6]
|
ActivateTextEditorTest#testActivateEditor()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 2s is not within [0%, 110'%] of 1.7s
- 19.7% [±1.4]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 3.9s is not within [0%, 110'%] of 3s
- 30.8% [±1.5]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.44s is not within [0%, 110'%] of 0.29s
- 52.3% [±1.3]
|
CodeCompletionPerformanceTest#testApplicationNoParamters()
|
+1.5% [±1.2]
|
+2.7% [±1.4]
|
+3.7% [±1.2]
|
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses()
|
- 3.8% [±0.8]
|
- 3.6% [±0.9]
|
- 2.8% [±1.3]
|
CodeCompletionPerformanceTest#testApplicationWithParamterGuesses2()
|
+4.8% [±1.1]
|
+0.6% [±0.8]
|
+6.8% [±1.5]
|
CodeCompletionPerformanceTest#testApplicationWithParamterNames()
|
+3.9% [±1.1]
|
+2.8% [±0.7]
|
+5.6% [±1.5]
|
CodeCompletionPerformanceTest#testCompletionNoParamters()
|
- 2.9% [±0.7]
|
- 4.7% [±1.6]
|
+1.4% [±3.6]
|
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses()
|
- 5.8% [±0.6]
|
- 3.1% [±0.6]
|
- 3.3% [±1.5]
|
CodeCompletionPerformanceTest#testCompletionWithParamterGuesses2()
|
- 1.5% [±0.4]
|
- 0.3% [±0.2]
|
- 2.0% [±0.8]
|
CodeCompletionPerformanceTest#testCompletionWithParamterNames()
|
- 4.3% [±1.0]
|
- 4.1% [±1.0]
|
- 3.8% [±1.4]
|
ContentTypeTest#testJavaDirty()
|
- 3.6% [±4.3]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 60ms is not within [0%, 110'%] of 45ms
- 33.3% [±2.8]
|
+17.9% [±6.9]
|
ContentTypeTest#testPluginPropertiesDirty()
|
- 4.8% [±2.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 92ms is not within [0%, 110'%] of 70ms
- 31.4% [±1.4]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.11s is not within [0%, 110'%] of 87ms
- 25.3% [±4.4]
|
ContentTypeTest#testPluginXMLDirty()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 5.7s is not within [0%, 110'%] of 4.1s
- 41.5% [±0.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 4.6s is not within [0%, 110'%] of 3.2s
- 41.0% [±0.2]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 5.7s is not within [0%, 110'%] of 4.1s
- 39.6% [±0.1]
|
ContentTypeTest#testPropertiesDirty()
|
- 9.8% [±3.5]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 66ms is not within [0%, 110'%] of 50ms
- 32.0% [±2.6]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 81ms is not within [0%, 110'%] of 59ms
- 37.3% [±11.2]
|
ContentTypeTest#testTextDirty()
|
- 6.0% [±2.3]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 66ms is not within [0%, 110'%] of 47ms
- 40.4% [±4.2]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 72ms is not within [0%, 110'%] of 53ms
- 35.8% [±10.0]
|
ConvertLineDelimitersProjectTest#test()
|
+41.4% [±40.9]
|
+6.5% [±13.0]
|
- 4.4% [±35.7]
|
DocumentLineDifferInitializationTest#testInitializationWithFewChanges()
|
- 0.5% [±1.5]
|
+87.5% [±0.0]
|
+88.6% [±89.6]
|
DocumentLineDifferInitializationTest#testInitializationWithManyChanges()
|
+13.6% [±0.0]
|
- 0.2% [±0.1]
|
+64.3% [±60.5]
|
DocumentLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()
|
+13.7% [±0.3]
|
+83.9% [±83.9]
|
+8.4% [±0.8]
|
DocumentLineDifferInitializationTest#testInitializationWithNoChanges()
|
+1.2% [±0.2]
|
+0.0% [±0.0]
|
+83.7% [±85.2]
|
DocumentLineDifferModificationTest#testEditingChanged()
|
+22.2% [±1.4]
|
+5.6% [±5.3]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 14ms is not within [0%, 110'%] of 2ms
- 600.0% [±133.1]
|
DocumentLineDifferModificationTest#testEditingUnchanged()
|
+50.0% [±16.7]
|
+40.0% [±19.9]
|
[n/a]
|
DocumentPerformanceTest#measureDeleteInsert()
|
+5.3% [±1.8]
|
+0.0% [±2.0]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 21ms is not within [0%, 110'%] of 19ms
- 10.5% [±12.0]
|
DocumentPerformanceTest#measureGetChar()
|
+1.4% [±0.7]
|
- 1.7% [±0.2]
|
+6.8% [±3.7]
|
DocumentPerformanceTest#measureGetLength()
|
+0.0% [±0.3]
|
+1.4% [±0.2]
|
+1.2% [±2.4]
|
DocumentPerformanceTest#measureGetLine()
|
+0.0% [±1.5]
|
+0.0% [±1.8]
|
+13.8% [±7.1]
|
DocumentPerformanceTest#measureGetNumberOfLines()
|
+0.0% [±0.2]
|
+0.0% [±0.1]
|
+0.0% [±2.9]
|
DocumentPerformanceTest#measureInsertAtEnd()
|
+0.0% [±1.9]
|
+0.0% [±1.6]
|
- 3.0% [±5.8]
|
DocumentPerformanceTest#measureInsertAtStart()
|
+2.7% [±2.4]
|
+0.0% [±1.1]
|
+8.0% [±3.1]
|
DocumentPerformanceTest#measureLineByIndex()
|
+0.0% [±0.2]
|
+0.0% [±0.1]
|
+6.2% [±2.9]
|
DocumentPerformanceTest#measureLineByOffset()
|
+1.6% [±0.8]
|
+0.0% [±0.2]
|
+2.3% [±2.0]
|
DocumentPerformanceTest#measureLineDelimiterByIndex()
|
+0.0% [±0.4]
|
+0.0% [±0.1]
|
- 4.0% [±2.6]
|
DocumentPerformanceTest#measureLineInfoByIndex()
|
+9.0% [±1.1]
|
+0.7% [±0.3]
|
+1.9% [±2.2]
|
DocumentPerformanceTest#measureLineInfoByOffset()
|
- 1.4% [±0.9]
|
- 1.6% [±0.2]
|
- 1.4% [±1.5]
|
DocumentPerformanceTest#measureLineLengthByIndex()
|
+0.0% [±0.2]
|
+0.0% [±0.2]
|
+0.0% [±1.8]
|
DocumentPerformanceTest#measureRandomReplace()
|
+0.0% [±0.6]
|
+0.0% [±0.2]
|
+2.4% [±1.4]
|
DocumentPerformanceTest#measureRepeatedReplace()
|
+0.7% [±0.6]
|
- 0.9% [±0.5]
|
- 0.7% [±1.8]
|
DocumentPerformanceTest#measureSet()
|
- 2.1% [±1.8]
|
+2.6% [±1.3]
|
- 2.3% [±3.8]
|
DocumentPerformanceTest#measureTypingReplaceInLargeFile()
|
+0.0% [±0.6]
|
+2.0% [±0.3]
|
- 2.1% [±1.9]
|
DocumentPerformanceTest#measureTypingReplaceInSmallFile()
|
- 0.7% [±0.5]
|
+0.0% [±0.3]
|
+0.7% [±1.6]
|
GapTextStorePerformanceTest#measureDeleteInsert()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 13ms is not within [0%, 110'%] of 11ms
- 18.2% [±6.2]
|
+0.0% [±1.9]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 11ms is not within [0%, 110'%] of 10ms
- 10.0% [±22.7]
|
GapTextStorePerformanceTest#measureGetAll()
|
+0.0% [±0.1]
|
+0.0% [±0.1]
|
- 0.7% [±1.6]
|
GapTextStorePerformanceTest#measureGetChar()
|
+0.0% [±0.3]
|
+0.5% [±0.1]
|
+0.4% [±0.8]
|
GapTextStorePerformanceTest#measureGetLine()
|
+0.7% [±0.6]
|
- 0.7% [±0.3]
|
+1.5% [±1.8]
|
GapTextStorePerformanceTest#measureInsertAtEnd()
|
+0.0% [±1.5]
|
- 5.0% [±1.6]
|
+0.0% [±11.5]
|
GapTextStorePerformanceTest#measureInsertAtStart()
|
+0.0% [±3.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 11ms is not within [0%, 110'%] of 10ms
- 10.0% [±3.3]
|
+0.0% [±23.3]
|
GapTextStorePerformanceTest#measureRandomReplace()
|
+0.0% [±0.3]
|
- 1.0% [±0.2]
|
+0.0% [±2.0]
|
GapTextStorePerformanceTest#measureRepeatedReplace()
|
+0.0% [±0.7]
|
+0.0% [±0.5]
|
+0.0% [±13.0]
|
GapTextStorePerformanceTest#measureSet()
|
+0.0% [±0.4]
|
+2.1% [±0.4]
|
+2.7% [±6.6]
|
GapTextStorePerformanceTest#measureTypingReplaceInLargeFile()
|
- 4.2% [±2.7]
|
+4.0% [±1.5]
|
- 4.2% [±10.5]
|
GapTextStorePerformanceTest#measureTypingReplaceInSmallFile()
|
+0.0% [±0.7]
|
- 8.3% [±1.3]
|
+0.0% [±10.5]
|
JavaExpandSelectionTest#test()
|
+5.5% [±0.9]
|
+0.0% [±0.3]
|
- 2.5% [±4.8]
|
JavaFormatterProjectTest#test()
|
- 8.2% [±5.5]
|
- 1.6% [±6.9]
|
+10.8% [±17.2]
|
JavaFormatterTest#test()
|
+5.0% [±0.5]
|
+0.0% [±0.3]
|
- 2.5% [±5.2]
|
JavaIndenterTest#testJavaIndenter2()
|
- 7.7% [±3.8]
|
- 4.5% [±18.7]
| n/a |
*JavaMoveLineTest#test() (vs. R-3.4-200806172000_200905291800)
| n/a |
+79.2% [±21.3]
| n/a |
JavaReplaceAllWithQuickDiffTest#test()
|
- 0.5% [±9.1]
|
+13.7% [±1.1]
| n/a |
LineTrackerPerformanceTest#testLineByIndex()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.76s is not within [0%, 110'%] of 0.65s
- 16.3% [±0.8]
|
+23.6% [±0.3]
|
- 0.3% [±0.6]
|
LineTrackerPerformanceTest#testLineByIndex2()
|
+0.0% [±0.2]
|
+0.0% [±0.1]
|
+4.6% [±1.0]
|
LineTrackerPerformanceTest#testLineByOffset()
|
+0.3% [±0.2]
|
- 0.7% [±0.4]
|
- 0.3% [±0.6]
|
LineTrackerPerformanceTest#testLineByOffset2()
|
- 0.1% [±0.0]
|
+0.3% [±0.0]
|
+0.7% [±0.1]
|
LineTrackerPerformanceTest#testRandomReplace()
|
+0.3% [±0.1]
|
+0.8% [±0.1]
|
+0.3% [±1.0]
|
LineTrackerPerformanceTest#testReplace()
|
+0.0% [±0.4]
|
+0.7% [±0.4]
|
+0.0% [±1.3]
|
LineTrackerPerformanceTest#testSet()
|
- 1.3% [±0.4]
|
+0.0% [±0.2]
|
- 2.1% [±1.6]
|
NonInitialTypingTest#testTypeAMethod()
|
+58.8% [±25.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.14s is not within [0%, 110'%] of 1ms
- 14,000.0% [±13,983.4]
| n/a |
OpenJavaContentAssistTest#test1()-cold
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.26s is not within [0%, 110'%] of 0.23s
- 10.7% [±4.6]
|
+6.7% [±4.3]
|
+61.0% [±4.0]
|
OpenJavaContentAssistTest#test1()-warm
|
+43.3% [±2.8]
|
+28.6% [±2.2]
|
+52.0% [±3.7]
|
OpenJavaEditorStartupTest#testJavaEditorStartup()
|
+1.8% [n/a]
|
- 2.8% [n/a]
|
- 1.4% [n/a]
|
OpenJavaEditorTest#testOpenEditor3()
|
+6.2% [±1.8]
|
+5.1% [±2.2]
|
+0.4% [±3.7]
|
OpenJavaEditorTest#testOpenEditor4()
|
+2.3% [±2.5]
|
+8.6% [±2.2]
|
- 0.9% [±3.5]
|
OpenJavaEditorTest#testOpenEditor5()
|
+17.3% [±3.2]
|
+18.9% [±2.6]
|
+7.5% [±4.1]
|
OpenJavaEditorTest#testOpenEditor6()
|
+9.5% [±3.7]
|
+15.1% [±3.7]
|
- 1.9% [±4.9]
|
OpenJavaEditorTest#testOpenJavaEditor1()
|
+6.1% [±6.2]
|
+18.0% [±4.2]
|
+4.8% [±8.6]
|
OpenJavaEditorTest#testOpenJavaEditor2()
|
+5.6% [±5.1]
|
+10.3% [±4.2]
|
- 2.4% [±7.7]
|
OpenPreferencePageTest#testOpenPreferencePage()-cold
|
+21.5% [n/a]
|
- 5.8% [n/a]
| n/a |
OpenQuickOutlineTest#testOpenQuickOutline1()-cold
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.29s is not within [0%, 110'%] of 0.24s
- 18.1% [±0.7]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.37s is not within [0%, 110'%] of 0.31s
- 21.2% [±2.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.14s is not within [0%, 110'%] of 0.12s
- 10.4% [±2.5]
|
OpenQuickOutlineTest#testOpenQuickOutline1()-warm
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.29s is not within [0%, 110'%] of 0.25s
- 16.3% [±1.7]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.35s is not within [0%, 110'%] of 0.31s
- 13.1% [±2.4]
|
+1.6% [±2.2]
|
OpenTextEditorStartupTest#testTextEditorStartup()
|
- 3.9% [n/a]
|
- 0.9% [n/a]
|
+0.6% [n/a]
|
OpenTextEditorTest#testOpenEditor3()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.15s is not within [0%, 110'%] of 0.12s
- 19.2% [±0.4]
|
+7.4% [±1.4]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 94ms is not within [0%, 110'%] of 62ms
- 51.6% [±0.4]
|
OpenTextEditorTest#testOpenTextEditor1()
|
- 8.5% [±2.5]
|
+1.5% [±1.4]
|
- 1.5% [±1.8]
|
OpenTextEditorTest#testOpenTextEditor2()
|
- 8.9% [±3.8]
|
+0.0% [±1.3]
|
- 1.6% [±7.6]
|
RevertJavaEditorTest#test2()
|
+11.7% [±3.8]
|
+3.4% [±3.6]
|
+31.1% [±6.3]
|
RevertJavaEditorTest#testRevertJavaEditor()
|
+16.9% [±5.1]
|
+12.8% [±3.3]
|
+58.0% [±45.4]
|
RevertTextEditorTest#testRevertTextEditor()
|
+0.0% [±16.0]
|
+7.4% [±12.5]
|
+0.0% [±25.2]
|
SaveJavaEditorTest#test1()
|
- 0.6% [±3.5]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.41s is not within [0%, 110'%] of 0.26s
- 56.7% [±50.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.88s is not within [0%, 110'%] of 0.39s
- 125.9% [±84.7]
|
SaveTextEditorTest#test1()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.49s is not within [0%, 110'%] of 0.43s
- 14.4% [±37.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.39s is not within [0%, 110'%] of 0.3s
- 31.1% [±42.4]
|
+25.5% [±21.8]
|
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWise1()
|
+3.9% [±0.3]
|
+2.1% [±0.0]
| n/a |
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWiseMoveCaret1()
|
+0.0% [±0.1]
|
+0.1% [±0.1]
| n/a |
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorLineWiseSelect1()
|
+0.4% [±0.2]
|
- 0.6% [±0.0]
| n/a |
ScrollAnnotatedJavaEditorTest#testScrollJavaEditorPageWise()
|
- 0.3% [±0.1]
|
- 2.8% [±0.1]
| n/a |
ScrollJavaEditorTest#testScrollJavaEditorLineWise2()
|
+4.9% [±0.2]
|
+1.4% [±0.1]
| n/a |
ScrollJavaEditorTest#testScrollJavaEditorLineWiseMoveCaret2()
|
+0.6% [±0.1]
|
+0.4% [±0.1]
| n/a |
ScrollJavaEditorTest#testScrollJavaEditorLineWiseSelect2()
|
+0.7% [±0.1]
|
- 0.6% [±0.0]
| n/a |
ScrollJavaEditorTest#testScrollJavaEditorPageWise()
|
+2.2% [±0.3]
|
- 1.2% [±0.1]
| n/a |
ScrollTextEditorTest#testScrollTextEditorLineWise2()
|
- 0.3% [±0.3]
|
- 1.6% [±0.2]
| n/a |
ScrollTextEditorTest#testScrollTextEditorLineWiseMoveCaret2()
|
- 3.8% [±0.1]
|
- 3.8% [±0.1]
| n/a |
ScrollTextEditorTest#testScrollTextEditorLineWiseSelect2()
|
- 2.8% [±0.2]
|
- 3.4% [±0.0]
| n/a |
ScrollTextEditorTest#testScrollTextEditorPageWise()
|
- 1.6% [±0.1]
|
+0.2% [±0.0]
| n/a |
ScrollVerticalRuler1000Test#testScrollTextEditorLineWiseMoveCaret2()
|
- 1.2% [±0.1]
|
- 4.2% [±0.1]
| n/a |
ScrollVerticalRuler1000Test#testScrollTextEditorPageWise()
|
+0.0% [±0.5]
|
- 0.3% [±0.5]
| n/a |
ScrollVerticalRuler100Test#testScrollTextEditorLineWiseMoveCaret2()
|
- 3.0% [±0.8]
|
- 3.0% [±0.3]
| n/a |
ScrollVerticalRuler100Test#testScrollTextEditorPageWise()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.13s is not within [0%, 110'%] of 0.1s
- 21.9% [±13.4]
|
+29.8% [±8.2]
| n/a |
ScrollVerticalRuler5000Test#testScrollTextEditorPageWise()
|
+1.4% [±0.2]
|
+1.2% [±0.2]
| n/a |
SynchronizedLineDifferInitializationTest#testInitializationWithFewChanges()
|
+0.0% [±0.0]
|
+87.5% [±0.0]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.6s is not within [0%, 110'%] of 0.55s
- 10.1% [±0.5]
|
SynchronizedLineDifferInitializationTest#testInitializationWithManyChanges()
|
+14.3% [±0.0]
|
+0.2% [±0.1]
|
+9.8% [±1.5]
|
SynchronizedLineDifferInitializationTest#testInitializationWithManyChangesButEqualSize()
|
+14.1% [±0.2]
|
+0.0% [±0.6]
|
+9.7% [±0.7]
|
SynchronizedLineDifferInitializationTest#testInitializationWithNoChanges()
|
- 0.7% [±0.2]
|
+0.0% [±0.0]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 0.6s is not within [0%, 110'%] of 0.55s
- 10.4% [±0.8]
|
TextMoveLineTest#test()
|
+44.4% [±46.8]
|
+6.6% [±35.7]
| n/a |
TextNonInitialTypingTest#testTypeAMethod()
|
Performance criteria not met when compared to 'R-3.4-200806172000_200906121800': Elapsed Process= 53ms is not within [0%, 110'%] of 1ms
- 5,200.0% [±2,617.0]
|
+0.0% [±47.1]
| n/a |
ToggleCommentTest#testToggleComment2()-comment
|
+18.6% [±2.4]
|
+3.3% [±0.5]
|
+0.0% [±0.4]
|
ToggleCommentTest#testToggleComment2()-uncomment
|
+20.7% [±3.3]
|
+6.3% [±2.0]
|
+4.4% [±4.5]
|
UndoJavaEditorTest#testUndoJavaEditor2()
|
+8.7% [±1.8]
|
- 0.5% [±0.9]
|
+5.1% [±0.3]
|
UndoTextEditorTest#testUndoTextEditor2()
|
+15.5% [±18.2]
|
+60.5% [±63.0]
|
+13.8% [±14.2]
|