Android Benchmark Results for the 2011-10 Engineering Cycle
About
The results in this page were conducted to compare various Linaro and Android toolchains using the Android Benchmarking Tools
The toolchains used were:
The target device tested was a PandaBoard running
Contents
Summary of Results
The chart to the right shows percentage improvements versus the Android 4.4 toolchain.
skia - 4.6 showed a .5% month to month improvement.
cximage - no measurable difference
gcstone - the variance of results were too high make any conclusions.
gnugo - Linaro toolchains still faster than standard Android. There were no measurable differences between the various Linaro toolchains used.
python - 4.5 remains a bit faster 4.6
The chart on the right shows percent improvements against Android 4.4. This was done by comparing the cumulative totals for each benchmark.
Methodology
The raw data this page is based on can be found at:
The scripts used to generate the data can be found at
http://git.linaro.org/gitweb?p=people/doanac/linaro-toolchain-benchmarking.git
run-benchmarks - downloads, builds, and runs each suite of benchmarks for a toolchain
generate-wiki-page - takes the outputs from run-benchmarks and builds this page
skia results
NOTE: smaller numbers are better
TEST |
skia-android-4.4 |
skia-linaro-4.5 |
skia-linaro-4.6 |
skia-linaro-4.6-09 |
skia-linaro-4.6-O3graphite |
decode_4444_(null) |
0.00 |
0.27 |
0.00 |
0.00 |
0.00 |
decode_565_(null) |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
decode_8888_(null) |
0.00 |
0.00 |
0.00 |
0.33 |
0.00 |
repeatTile_index8 |
185.13 |
184.47 |
184.60 |
190.07 |
189.27 |
repeatTile_4444 |
243.87 |
239.07 |
219.33 |
224.60 |
222.07 |
repeatTile_565 |
178.07 |
159.40 |
157.40 |
156.53 |
156.80 |
repeatTile_8888 |
186.00 |
184.53 |
183.67 |
184.67 |
187.00 |
bitmap_index8 |
87.53 |
84.87 |
85.53 |
86.07 |
86.00 |
bitmap_index8_A |
180.27 |
176.07 |
177.47 |
178.60 |
179.00 |
bitmap_4444 |
64.60 |
75.00 |
67.00 |
67.00 |
66.67 |
bitmap_4444_A |
163.47 |
163.00 |
156.27 |
156.27 |
155.53 |
bitmap_565 |
178.93 |
167.60 |
166.13 |
166.20 |
166.20 |
bitmap_8888 |
29.40 |
30.40 |
31.13 |
30.73 |
30.53 |
bitmap_8888_A |
99.33 |
98.40 |
100.73 |
99.60 |
101.00 |
polygon |
163.73 |
157.73 |
158.00 |
157.60 |
157.47 |
lines |
46.93 |
45.13 |
45.00 |
45.47 |
45.47 |
points |
10.40 |
10.20 |
10.73 |
10.07 |
10.80 |
rrects3 |
18.00 |
17.73 |
17.73 |
17.67 |
17.73 |
rrects1 |
55.20 |
54.47 |
54.13 |
54.73 |
55.67 |
ovals3 |
19.27 |
18.13 |
19.33 |
19.13 |
20.00 |
ovals1 |
63.07 |
63.40 |
63.73 |
63.00 |
65.27 |
rects3 |
1.80 |
1.80 |
1.73 |
1.73 |
1.60 |
rects1 |
10.93 |
11.13 |
11.33 |
10.93 |
11.07 |
Totals |
1985.93 |
1942.80 |
1910.97 |
1921.00 |
1925.15 |
python results
NOTE: smaller numbers are better
TEST |
python-android-4.4 |
python-linaro-4.5 |
python-linaro-4.6 |
python-linaro-4.6-09 |
python-linaro-4.6-O3graphite |
BuiltinFunctionCalls |
711.00 |
709.00 |
693.00 |
672.00 |
671.00 |
BuiltinMethodLookup |
480.00 |
455.00 |
501.00 |
506.00 |
501.00 |
CompareFloats |
519.00 |
470.00 |
470.00 |
466.00 |
464.00 |
CompareFloatsIntegers |
465.00 |
429.00 |
433.00 |
435.00 |
437.00 |
CompareIntegers |
532.00 |
508.00 |
510.00 |
505.00 |
509.00 |
CompareInternedStrings |
571.00 |
526.00 |
553.00 |
553.00 |
556.00 |
CompareLongs |
474.00 |
451.00 |
480.00 |
485.00 |
480.00 |
CompareStrings |
491.00 |
429.00 |
479.00 |
495.00 |
499.00 |
CompareUnicode |
555.00 |
451.00 |
459.00 |
433.00 |
433.00 |
ComplexPythonFunctionCalls |
679.00 |
673.00 |
682.00 |
631.00 |
629.00 |
ConcatStrings |
820.00 |
800.00 |
795.00 |
797.00 |
781.00 |
ConcatUnicode |
654.00 |
650.00 |
679.00 |
691.00 |
741.00 |
CreateInstances |
756.00 |
702.00 |
696.00 |
741.00 |
745.00 |
CreateNewInstances |
596.00 |
529.00 |
555.00 |
550.00 |
558.00 |
CreateStringsWithConcat |
520.00 |
563.00 |
511.00 |
524.00 |
516.00 |
CreateUnicodeWithConcat |
427.00 |
416.00 |
454.00 |
447.00 |
453.00 |
DictCreation |
383.00 |
327.00 |
339.00 |
345.00 |
344.00 |
DictWithFloatKeys |
529.00 |
449.00 |
466.00 |
457.00 |
458.00 |
DictWithIntegerKeys |
516.00 |
444.00 |
446.00 |
450.00 |
449.00 |
DictWithStringKeys |
496.00 |
423.00 |
431.00 |
426.00 |
429.00 |
ForLoops |
426.00 |
414.00 |
409.00 |
429.00 |
424.00 |
IfThenElse |
407.00 |
394.00 |
379.00 |
393.00 |
388.00 |
ListSlicing |
565.00 |
568.00 |
568.00 |
572.00 |
578.00 |
NestedForLoops |
526.00 |
501.00 |
507.00 |
524.00 |
522.00 |
NestedListComprehensions |
707.00 |
719.00 |
718.00 |
724.00 |
721.00 |
NormalClassAttribute |
551.00 |
505.00 |
532.00 |
555.00 |
550.00 |
NormalInstanceAttribute |
467.00 |
433.00 |
509.00 |
529.00 |
530.00 |
PythonFunctionCalls |
548.00 |
528.00 |
509.00 |
515.00 |
515.00 |
PythonMethodCalls |
738.00 |
725.00 |
766.00 |
777.00 |
769.00 |
Recursion |
750.00 |
689.00 |
705.00 |
703.00 |
696.00 |
SecondImport |
773.00 |
775.00 |
777.00 |
787.00 |
795.00 |
SecondPackageImport |
809.00 |
784.00 |
816.00 |
824.00 |
815.00 |
SecondSubmoduleImport |
951.00 |
949.00 |
969.00 |
967.00 |
968.00 |
SimpleComplexArithmetic |
441.00 |
443.00 |
418.00 |
445.00 |
446.00 |
SimpleDictManipulation |
544.00 |
488.00 |
487.00 |
488.00 |
489.00 |
SimpleFloatArithmetic |
591.00 |
510.00 |
557.00 |
540.00 |
545.00 |
SimpleIntFloatArithmetic |
407.00 |
372.00 |
394.00 |
398.00 |
391.00 |
SimpleIntegerArithmetic |
411.00 |
369.00 |
399.00 |
394.00 |
401.00 |
SimpleListComprehensions |
626.00 |
630.00 |
615.00 |
628.00 |
615.00 |
SimpleListManipulation |
447.00 |
407.00 |
423.00 |
417.00 |
427.00 |
SimpleLongArithmetic |
523.00 |
459.00 |
528.00 |
528.00 |
525.00 |
SmallLists |
595.00 |
627.00 |
595.00 |
603.00 |
606.00 |
SmallTuples |
583.00 |
539.00 |
587.00 |
583.00 |
595.00 |
SpecialClassAttribute |
533.00 |
494.00 |
528.00 |
548.00 |
541.00 |
SpecialInstanceAttribute |
631.00 |
595.00 |
619.00 |
679.00 |
676.00 |
StringMappings |
610.00 |
487.00 |
495.00 |
501.00 |
490.00 |
StringPredicates |
551.00 |
493.00 |
542.00 |
548.00 |
546.00 |
StringSlicing |
535.00 |
507.00 |
561.00 |
552.00 |
557.00 |
TryExcept |
552.00 |
485.00 |
477.00 |
492.00 |
494.00 |
TryFinally |
642.00 |
566.00 |
593.00 |
580.00 |
578.00 |
TryRaiseExcept |
554.00 |
562.00 |
583.00 |
578.00 |
579.00 |
TupleSlicing |
499.00 |
495.00 |
532.00 |
510.00 |
488.00 |
UnicodeMappings |
647.00 |
683.00 |
658.00 |
651.00 |
651.00 |
UnicodePredicates |
603.00 |
562.00 |
588.00 |
595.00 |
600.00 |
UnicodeSlicing |
509.00 |
527.00 |
579.00 |
570.00 |
563.00 |
WithFinally |
837.00 |
735.00 |
768.00 |
749.00 |
767.00 |
WithRaiseExcept |
716.00 |
712.00 |
739.00 |
726.00 |
738.00 |
Totals |
32978.00 |
31077.00 |
32058.00 |
31994.00 |
32072.00 |
gnugo
NOTE: smaller numbers are better
Toolchain |
Time (in seconds) |
gnugo-android-4.4 |
133.28 |
gnugo-linaro-4.5 |
122.92 |
gnugo-linaro-4.6 |
123.86 |
gnugo-linaro-4.6-09 |
123.70 |
gnugo-linaro-4.6-O3graphite |
123.73 |
gcstone results
NOTE: smaller numbers are better
TEST |
gcstone-android-4.4 |
gcstone-linaro-4.5 |
gcstone-linaro-4.6 |
gcstone-linaro-4.6-09 |
gcstone-linaro-4.6-O3graphite |
Total Array Alloc Time |
178.00 |
156.00 |
126.00 |
166.00 |
141.00 |
Total Array GC Time |
22.00 |
22.00 |
23.00 |
21.00 |
22.00 |
Total List Alloc Time |
3906.00 |
3769.00 |
3938.00 |
3862.00 |
3816.00 |
Total List GC Time |
703.00 |
701.00 |
705.00 |
706.00 |
704.00 |
Totals |
4809.00 |
4648.00 |
4792.00 |
4755.00 |
4683.00 |
cximage
NOTE: smaller numbers are better
Toolchain |
Time (in seconds) |
cximage-android-4.4 |
39.69 |
cximage-linaro-4.5 |
40.24 |
cximage-linaro-4.6 |
40.29 |
cximage-linaro-4.6-09 |
40.25 |
cximage-linaro-4.6-O3graphite |
40.19 |
Platform/Android/AndroidToolchainBenchmarking/2011-10 (last modified 2011-10-27 21:54:23)