Resource: LCA14 Name: LCA14-205: Optimizing SQLite for Android mobile Date: 04-03-2014 Speaker: Khasim Syed Mohammed Video: https://www.youtube.com/watch?v=iobAwaEu1oM

1. Tue-4-Mar, 11:15am, Khasim Syed Mohammed
LCA14-205: Optimizing SQLite for
Android mobile

2. www.linaro.org
Boot Time Reduction investigation
Back ground of this work
FutureQ1 Q22013
2014
12-DEC-2013
Android Optimizations
Cortex-A7/A15 Investigation
Q3 Q4
Power Optimization investigation
fdlibm
sqlite
Migrate AOSP to latest
zlib
guava
LLVM / Clang Android
explore libart
system start-up
A7/A15 Optimized
bionic
LLVM wrapper
LLVM patches
profile and benchmark
LLVM and GCC
NEON MM libraries
Open Crypto
libraries
Optimize
Speech comp
compiler-rt
Q4
Closed
Development
Drafting
Community/
External
Upstream
Approved
Scheduled
Ongoing

3. www.linaro.org
Agenda
Progress update
■ What’s the approach
■ Our findings so far
■ Remaining work
NOTE : This is still a work in progress - estimated date of
closure end of April 2014

4. www.linaro.org
What is SQLite ?
■ SQLite is a software library that implements a self-
contained, serverless, zero-configuration,transactional
SQL database engine.
■ SQLite is the most widely deployed SQL database engine
in the world.
■ The source code for SQLite and more information
available on sqlite.org

5. www.linaro.org
Do we really need database
in
Mobile devices ?

6. Do we need database in Mobile device ?
Source : http://gigaom.com/2013/05/11/welcome-to-the-new-and-fast-growing-ecosystem-of-mobile-business-apps/
- Mobile apps are
transforming the
business software
market
- Business / Enterprise
apps are getting built
for “mobile first”
- Most of these
applications need data
base management
system on mobile
-basic requirement:
Large/Small amount of
data should be stored
and important data
should be retrieved
from a large database
in the least possible
time.

7. • Explore the folder external/sqlite in Android sources
• Finding possible ways of improving the overall
performance of SQlite
• Ensure that we don’t break any compatibility
Expectation

8. • Understand how SQLite operates and find it’s
dependencies.
Our approach
Commands
Use CPU to
run Queries
eMMC access to
store or retrieve
information
• Possible optimizations areas :
• Data : Improve data throughput
• CPU : Utilize CPU efficiently to run queries faster
and reduce CPU utilization by offloading the SQL
queries to other cores if possible.

9. • RL Benchmark is an SQLite benchmark that
runs thousands of insert, select, update, and
delete functions - the same functions Android
uses to store data.
• The resulting number is the total amount of
time (in seconds) that it took for your device
to run through all the tests.
• The faster it completed it, the faster your
device stores and retrieves data; thus, lower
is better.
Tools / Measurement
Also looked at SQL Bench, Andro Bench,
but this one was most widely used by app
developers.

10. • The following functions from cortex-strings optimizations from Linaro perform better than
AOSP
• strlen
• memchr
• memcpy
Learnings
Test with out optimization with optimization
500000000 * strlen(15*a) 8.563289 7.3658144
1000000 * strlen(15000*a) 4.795874 4.4586328
10000000 * memcpy of 15000 bytes 14.0750624 12.276912
900000000 * memcpy of 15 bytes 3.1883678 3.1854456
https://wiki.linaro.
org/Platform/Android/CortexStringsInBionic
Thanks to Bero @ Linaro.org
NOTE: SQLite source has around
a. 240 “memcpy”
b. 60 “strlen”

11. • Accelerating SQL Database Operations on a GPU is possible (Example with
CUDA) - Thanks to Tom Gall @ Linaro.org for sharing the information
• http://pbbakkum.com/db/
• Nexus 10 device can be prevented from suspending or entering lower power
modes by
echo benchmark >/sys/power/wake_lock
cd /sys/devices/system/cpu/cpu0/cpufreq
echo userspace >scaling_governor
echo 1700000 >scaling_max_freq
echo 1700000 >scaling_min_freq
echo 1700000 >scaling_setspeed
• From 4.3 onwards Android supports FSTRIM, that basically calls DISCARD
command - This marks all the unused blocks on the underlying flash device as
unused, so the device performs better.
Learnings

12. • On Nexus 10 with AOSP master + Linaro patches for
BIONIC optimizations
• Preventing the device to enter lower power states
Performance improved by 15 %
Observation

13. BIONIC (memcpy, strlen) optimizations
Test AOSP
4.4.2
Master +
BIONIC
optimizations
Master + BIONIC
optimization + suspend
disabled
1000 INSERTs 8.3 7.328 7.027
25000 INSERTs in a transaction 2.1 2.026 1.997
25000 INSERTs into an indexed
table
2.5 2.012 1.957
100 SELECTs without an index 0.042 0.012 0.008
100 SELECTs on a string
comparison
0.012 0.008 0.015
Creating an index 0.428 0.305 0.312
5000 SELECTs with an index 0.718 0.556 0.515
1000 UPDATEs without an index 2.38 2.138 2.068

14. BIONIC (memcpy, strlen) optimizations
Test AOSP
4.4.2
Master +
BIONIC
optimizatio
ns
Master + BIONIC
optimization + suspend
disabled
25000 UPDATEs with an index 3.144 3.197 2.965
INSERTs from a SELECT 0.58 0.635 0.552
DELETE without an index 0.63 0.509 0.509
DELETE with an index 0.5 0.468 0.48
DROP TABLE 0.29 0.309 0.292
OVER ALL 21.747 19.503 18.697

15. • On Nexus 7 with AOSP master
• Enabling ART instead of Dalvik
Performance improved by 11 % (surprising ? despite
SQLite is a native library, the Java API imposes an
significant overhead)
HOWEVER, for some reason ART on Nexus 10 failed to
give any better performance - maybe ART on Nexus 10 is
still not optimized efficiently
Observation

16. Lib ART (Nexus 7)
Test Dalvik ART
1000 INSERTs 10.594 10.369
25000 INSERTs in a transaction 3.96 3.36
25000 INSERTs into an indexed table 3.93 3.336
100 SELECTs without an index 0.026 0.016
100 SELECTs on a string comparison 0.029 0.016
Creating an index 0.602 0.566
5000 SELECTs with an index 1.222 0.81
1000 UPDATEs without an index 4.357 3.648

17. Lib ART (Nexus 7)
Test Dalvik ART
25000 UPDATEs with an index 6.079 5.038
INSERTs from a SELECT 0.935 0.96
DELETE without an index 0.886 0.915
DELETE with an index 0.776 0.809
DROP TABLE 0.462 0.433
OVER ALL 33.867 30.279

18. ● Android uses 3.7.11 and SQLite.org is on 3.8.3.1
○ Lot of bug fixes and optimizations - trying since yesterday …
○ Will file a AOSP bug once we try it out to migrate SQLite in AOSP to latest.
● libsql depends on following libraries - need to find if we could optimize any of
these routines.
○ libdl.so
○ liblog.so
○ libicuuc.so
○ libicui18n.so
○ libutils.so
○ libc.so
○ libstdc++.so
○ libm.so
● Android provides many APIs and FLAGS for APP developers like WRITE_AHEAD
(basically buffers multiple I/O access into one and reads/writes in one chunk)
○ Many books, tutorials available to app developers to refer to.
Next steps

19. • Accelerating SQL Database Operations on a GPU - Should we explore this ?
• The research paper uses CUDA
• Linaro GWG team has a card to implement the same with OpenCL
• Android supports only renderscript (not compatible with OpenCL or CUDA)
• Renderscript is mainly used in JAVA layers to leverage hardware
optimizations, but SQL code is already a “C” code, introducing renderscript
calls is something we haven’t done before.
• Is this worth of an effort ? DEBATE …
• May be wait till we get some numbers to compare with OpenCL.
Next steps - “Need a new slide for this one”

20. Please feel free to mail us on
linaro-android@lists.linaro.org
or IRC (freenode)
#linaro-android
Any questions ?

21. More about Linaro Connect: http://connect.linaro.org
More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
Linaro members: www.linaro.org/members