All 52 scenarios |
SLED 10 Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)
| RHEL 5.0 Sun 6.0_04 (2 x 3.00GHz - 3GB RAM)
| Win XP Sun 1.5.0_10 (2 x 3.00GHz - 3GB RAM)
|
BenchFileStore#testStoreExists()
|
- 6.0% [±1.0]
|
- 1.4% [±2.0]
|
+8.4% [±1.0]
|
BenchFileStore#testStoreIsReadOnly()
|
- 7.1% [±0.4]
|
- 2.9% [±0.9]
|
+8.4% [±0.6]
|
BenchFileStore#testStoreLastModified()
|
- 7.1% [±0.5]
|
- 2.9% [±0.7]
|
+8.3% [±0.5]
|
BenchMiscWorkspace#testGetProject()
|
- 0.8% [±1.0]
|
+0.0% [±1.1]
|
+1.8% [±0.8]
|
BenchMiscWorkspace#testNoOp()
|
- 1.1% [±2.3]
|
+2.0% [±1.8]
|
+1.8% [±1.6]
|
BenchWorkspace#testCountResources()
|
- 7.2% [±0.5]
|
+0.0% [±0.4]
|
+1.1% [±3.0]
|
BenchWorkspace#testCountResourcesDuringOperation()
|
+6.7% [±0.5]
|
+1.2% [±0.4]
|
+12.8% [±0.5]
|
BenchWorkspace#testFindMaxProblemSeverity()
|
- 3.1% [±0.2]
|
- 3.2% [±0.1]
|
- 2.4% [±0.1]
|
BuilderPerformanceTest#testManualBuildWithAutobuildOn()
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.47s is not within [0%, 110'%] of 0.32s
- 46.4% [±17.6]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.38s is not within [0%, 110'%] of 0.29s
- 29.4% [±0.9]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.2s is not within [0%, 110'%] of 0.3s
- 280.9% [±36.4]
|
LocalHistoryPerformanceTest#testAddState()
|
+5.0% [±4.6]
|
+0.0% [±6.4]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.25s is not within [0%, 110'%] of 0.18s
- 43.2% [±3.9]
|
LocalHistoryPerformanceTest#testBug28603()
|
+0.0% [±7.6]
|
+0.0% [±3.7]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.18s is not within [0%, 110'%] of 90ms
- 95.6% [±7.4]
|
LocalHistoryPerformanceTest#testClearHistory100x4()
|
- 7.7% [±21.2]
|
+0.0% [±18.9]
|
+88.8% [±85.6]
|
LocalHistoryPerformanceTest#testClearHistory20x20()
|
+0.0% [±5.8]
|
+0.0% [±0.0]
|
+15.4% [±1.0]
|
LocalHistoryPerformanceTest#testClearHistory4x100()
|
+14.3% [±3.6]
|
+0.0% [±7.6]
|
+34.0% [±11.6]
|
LocalHistoryPerformanceTest#testCopyHistory100x4()
|
+0.4% [±1.9]
|
+20.0% [±7.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 3.3s is not within [0%, 110'%] of 2s
- 63.7% [±7.1]
|
LocalHistoryPerformanceTest#testCopyHistory20x20()
|
+0.0% [±1.3]
|
+7.1% [±2.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.4s is not within [0%, 110'%] of 0.8s
- 78.9% [±30.0]
|
LocalHistoryPerformanceTest#testCopyHistory4x100()
|
- 2.6% [±4.3]
|
+62.0% [±30.1]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.66s is not within [0%, 110'%] of 0.39s
- 70.7% [±38.2]
|
LocalHistoryPerformanceTest#testGetDeletedMembers100x4()
|
+4.0% [±19.7]
|
- 5.0% [±23.7]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.6s is not within [0%, 110'%] of 31ms
- 4,912.9% [±1,996.5]
|
LocalHistoryPerformanceTest#testGetDeletedMembers20x20()
|
+9.1% [±6.4]
|
+0.0% [±0.0]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.84s is not within [0%, 110'%] of 23ms
- 3,530.4% [±1,456.7]
|
LocalHistoryPerformanceTest#testGetDeletedMembers4x100()
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 11ms is not within [0%, 110'%] of 9ms
- 22.2% [±22.2]
|
+0.0% [±8.3]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 27ms is not within [0%, 110'%] of 23ms
- 17.4% [±35.5]
|
LocalHistoryPerformanceTest#testGetHistory()
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 10ms is not within [0%, 110'%] of 9ms
- 11.1% [n/a]
|
+0.0% [n/a]
|
+6.2% [n/a]
|
LocalHistoryPerformanceTest#testHistoryCleanUp100x4()
|
- 2.0% [±7.4]
|
+1.0% [±6.9]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 11s is not within [0%, 110'%] of 5.5s
- 99.9% [±16.2]
|
LocalHistoryPerformanceTest#testHistoryCleanUp20x20()
|
+1.4% [±1.1]
|
+2.8% [±1.6]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 10.6s is not within [0%, 110'%] of 6.5s
- 62.6% [±26.3]
|
MarkerPerformanceTest#testSetAttributes1()
|
+14.2% [n/a]
|
- 5.1% [n/a]
|
- 0.5% [n/a]
|
MarkerPerformanceTest#testSetAttributes2()
|
+6.5% [n/a]
|
- 5.5% [n/a]
|
+0.7% [n/a]
|
PropertyManagerPerformanceTest#testGetProperty100x4()
|
- 0.9% [±1.7]
|
+0.0% [±1.3]
|
- 3.2% [±10.3]
|
PropertyManagerPerformanceTest#testGetProperty20x20()
|
- 1.2% [±0.2]
|
- 0.4% [±0.2]
|
+0.5% [±0.6]
|
PropertyManagerPerformanceTest#testGetProperty4x100()
|
+0.0% [±0.1]
|
+0.0% [±0.0]
|
+0.5% [±0.2]
|
PropertyManagerPerformanceTest#testSetProperty100x4()
|
- 2.8% [±0.9]
|
+0.0% [±0.5]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.1s is not within [0%, 110'%] of 0.92s
- 18.2% [±11.7]
|
PropertyManagerPerformanceTest#testSetProperty20x20()
|
+1.2% [±1.2]
|
- 0.9% [±1.2]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.66s is not within [0%, 110'%] of 0.53s
- 25.7% [±23.3]
|
PropertyManagerPerformanceTest#testSetProperty4x100()
|
+2.6% [±1.0]
|
- 3.3% [±1.6]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 0.49s is not within [0%, 110'%] of 0.43s
- 14.8% [±19.0]
|
WorkspacePerformanceTest#testCreateResources()
|
+0.8% [±1.0]
|
+0.8% [±1.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1mn 27s is not within [0%, 110'%] of 59.1s
- 47.9% [±2.6]
|
WorkspacePerformanceTest#testDeleteProject()
|
- 1.6% [±12.6]
|
+9.5% [±11.8]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 37.6s is not within [0%, 110'%] of 28.9s
- 29.8% [±2.3]
|
WorkspacePerformanceTest#testFolderCopy()
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 10.4s is not within [0%, 110'%] of 8.5s
- 23.3% [±18.2]
|
+0.0% [±8.9]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1mn 54s is not within [0%, 110'%] of 1mn 25s
- 34.2% [±0.6]
|
WorkspacePerformanceTest#testFolderMove()
|
- 2.9% [±1.2]
|
+5.4% [±1.0]
|
+8.0% [±6.3]
|
WorkspacePerformanceTest#testRefreshProject()
|
+0.2% [±1.2]
|
+7.2% [±0.6]
|
+12.5% [±10.4]
|
BenchPath#testHash()
|
- 0.7% [±0.1]
|
+0.0% [±0.3]
|
- 1.4% [±0.2]
|
BenchPath#testPathCreation()
|
+0.2% [±0.2]
|
+2.1% [±0.4]
|
+2.5% [±0.2]
|
BenchPath#testToOSString()
|
- 1.0% [±0.5]
|
- 3.6% [±0.5]
|
+4.4% [±0.6]
|
BenchPath#testToString()
|
+2.9% [±0.4]
|
- 3.6% [±0.7]
|
+3.6% [±0.8]
|
*ContentTypePerformanceTest#testContentMatching() (vs. I20090407-1430)
|
+0.2% [±0.9]
|
+0.3% [±1.0]
|
+0.2% [±1.1]
|
*ContentTypePerformanceTest#testIsKindOf() (vs. I20090407-1430)
|
- 6.7% [±1.6]
|
- 5.1% [±1.4]
|
- 4.4% [±1.0]
|
*ContentTypePerformanceTest#testLoadCatalog() (vs. R-3.4-200806172000_200911301800)
|
Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 48ms is not within [0%, 110'%] of 35ms
- 37.1% [n/a]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200912141800': Elapsed Process= 50ms is not within [0%, 110'%] of 34ms
- 47.1% [n/a]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 49ms is not within [0%, 110'%] of 31ms
- 58.1% [n/a]
|
*ContentTypePerformanceTest#testNameMatching() (vs. R-3.4-200806172000_200911301800)
|
+34.0% [±0.6]
|
+27.3% [±0.8]
|
+35.1% [±0.8]
|
PreferencePerformanceTest#testGetStringCommonPrefixKeys()
|
+13.3% [±0.8]
|
- 1.5% [±1.5]
|
+9.4% [±0.6]
|
PreferencePerformanceTest#testGetStringMisses()
|
+7.9% [±0.2]
|
- 1.3% [±0.2]
|
+9.6% [±2.3]
|
PreferencePerformanceTest#testGetStringSequentialKeys()
|
+14.1% [±0.3]
|
+0.0% [±0.5]
|
+7.4% [±0.4]
|
PreferencePerformanceTest#testGetStringUniqueKeys()
|
+12.0% [±0.7]
|
- 0.9% [±1.8]
|
+9.8% [±0.8]
|
PreferencePerformanceTest#testPutStringKeys()
|
+3.0% [±0.6]
|
- 0.1% [±0.9]
|
- 1.1% [±0.4]
|
PreferencePerformanceTest#testRemoveStringKeys()
|
+3.6% [±0.3]
|
+0.0% [±0.4]
|
+5.7% [±2.1]
|
StartupTest.testApplicationStartup
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.5s is not within [0%, 110'%] of 1.3s
- 13.1% [±17.6]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.2s is not within [0%, 110'%] of 1s
- 12.6% [±18.7]
|
Performance criteria not met when compared to 'R-3.4-200806172000_201002011800': Elapsed Process= 1.8s is not within [0%, 110'%] of 1.5s
- 16.0% [±16.1]
|
*UIStartupTest.testUIApplicationStartup (vs. R-3.4-200806172000_200911301800)
|
Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 11.4s is not within [0%, 110'%] of 8.8s
- 29.2% [±5.9]
|
Performance criteria not met when compared to 'R-3.4-200806172000_200911301800': Elapsed Process= 8.1s is not within [0%, 110'%] of 6.9s
- 16.6% [±8.0]
|
- 2.4% [±6.0]
|