Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Applied Computer Science Concepts in Android

10.654 visualizaciones

Publicado el

In this presentation, the unaware or indirect applications of essential computer science concepts are dicussed as showcase. Jim Huang presented in Department of Computer Science and Engineering, National Taiwan University.

Publicado en: Tecnología
  • Sé el primero en comentar

Applied Computer Science Concepts in Android

  1. 1. Applied Computer Science Concepts in AndroidJim Huang ( 黃敬群 )Developer & Co-Founder, 0xlab Dec 31, 2010 / CSIE, NTU
  2. 2. Applied Computer Science Concepts in AndroidAndroid = A complete operating systemfor mobile computing, open sourced
  3. 3. Rights to copy © Copyright 2010 0xlab contact@0xlab.orgAttribution – ShareAlike 3.0You are free Corrections, suggestions, contributions and translations are welcome! to copy, distribute, display, and perform the work to make derivative works Latest update:Dec 31, 2010 to make commercial use of the workUnder the following conditions Attribution. You must give the original author credit. Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under a license identical to this one. For any reuse or distribution, you must make clear to others the license terms of this work. Any of these conditions can be waived if you get permission from the copyright holder.Your fair use and other rights are in no way affected by the above.License text:
  4. 4. Agenda (1) Android Internals (2) Compiler: 2D/3D Graphics (3) OS: Telephony (4) Virtual Machine: DatabaseIn this presentation, the unaware or indirect applications ofIn this presentation, the unaware or indirect applications ofessential computer science concepts are dicussed as showcase.essential computer science concepts are dicussed as showcase.
  5. 5. Android Internals… or, the low-level parts ...
  6. 6. Android Internals• Android Low-Level system – Architecture View – Everything from “Zygote” – Key design concepts in Android – System components• Hardware Abstraction Layer (HAL) – GPS, RIL, Graphics (2D/3D), ...
  7. 7. Android Low-level system• Architecture• Everything from “Zygote”• Key design concepts in Android
  8. 8. Android Low-Level System [ Architecture ]
  9. 9. GNU/Linux vs. Android Linux Android
  10. 10. Android = (patched) Linux Kernel + special user-space
  11. 11. Functional View Applications Gallery Phone Web Browser Google Maps ・・・・・ Android Framework Activity Window Content Notification View System Manager Manager Manager Manager Package Telephony Resource Location Manager Manager Manager Manager System Library Android RuntimeSurfaceFlinger OpenCORE SQLite Class Library Dalvik Virtual MachineOpenGL|ES AudioFlinger WebKit Skia OpenSSL bionic libc Freetype Linux Kernel
  12. 12. Android Low-Level System [ Everything from Zygote ]
  13. 13. Android Boot SequenceLow level initialization High level perspective (Android specific) ARM based Initialization process Target Board /etc/init.rc Android bootloader Start Android Service USB loader console adbd service manager Load Linux mountd debuggerd zygote mediaserver installd Start kernel flash_recovery - /init/main.c
  14. 14. Process View User-space:: process Process PID : 1 PPID : 2 1 PPID : 0 spawn spawn (fork+exec)(fork+exec) Process Process Process spawn (fork+exec) PPID : 3 PID : 2 PID : 3 PPID : 1 PPID : 1 2 Process Process PID : 4 PID : 5 PPID : 2 PPID : 3 3 4 fork() 5 Copy on Write
  15. 15. Role of Zygote
  16. 16. Everything spawn from Zygote
  17. 17. Android Low-Level System [ Key Design Concepts ]
  18. 18. System InteractionIPC: Inter-Process Communication
  19. 19. Activitieswindow widget Activities viewview group
  20. 20. Activity state transition Task 1Home Activity1 Activity2 Activity3 1 2 3 Home Back Key KeyHome Activity4 Activity5 Activity6 4 5 6 Task 2
  21. 21. Activity life cycle• Entire lifetime• Visible lifetime• Foreground lifetime
  22. 22. Services Activity Activity Service Invoking specific service (provided by Media server) Home orActivity1 Back Activity2
  23. 23. Broadcast Receivers ComponentsComponents ReceiverBroadcast SMS_RECEIVED Intent SMS_RECEIVED
  24. 24. Content Providers Components Content Providers Interface Content Provider Class Components   A Preferences Content Resolver Files Components   B SQLite(DB) Content Resolver NetworkContent Providers, Intents, Resolver
  25. 25. Components Intent Components Activity ActivityContext.startActivity() (Intents) orActivity.startActivityForResult() ServiceContext.startService()Context.bindService() or Broadcast BroadcastContext.sendBroadcast()Context.sendOrderedBroadcast() SystemContext.sendStickyBroadcast() 3 manifest Components 2 ActivityManager Class A Components Service Intent Intent 5 to class A 1 To Class A <class list> … class A 4 …
  26. 26. Android Services in ActionContent Telephony Bluetooth Connectivity LocationManager Service Service Service ManagerWindow Activity Package Power …Manager Manager Manager Manager Add Service Service Audio Flinger Add Service Managerdaemons Surface Service Flinger usbd Manager adbd System runtime Serverdebuggerd rild Init Zygote Dalvik VM Kernel Android Service Linux Kernel Dalvik Framework
  27. 27. IPC: Binder(1) Binder in Action  A pool of threads is associated to each service application to process incoming IPC (Inter-Process Communication).  Binder performs mapping of object between two processes.  Binder uses an object reference as an address in a process’s memory space.  Synchronous call, reference couting
  28. 28. IPC: Binder(2)
  29. 29. Understanding• Essential components – Compiler: Java – Virtual Machine: Dalvik – OS: Linux Kernel SSA form• Anything else?
  30. 30. CompilersApplied in 2D/3D Graphics
  31. 31. OpenGL Anywhere ARM® PowerVRCortex™- A8 SGXwith NEON VFP GraphicsC64x+ Other Display Peripherals Subsystem DSP
  32. 32. OpenGL Graphics PipelineState Variables Front Buffer Geometry Fragment Pipeline PipelineCamera Position Light Sources F Etc. C G E D Back Buffer A C GL_TRIANGLES A B B GL_TRIANLE_FAN A C E F B D Texture Maps GL_TRIANGLE_STRIP
  33. 33. Texturing ExampleBefore Texturing After Texturing
  34. 34. Graphics Pipeline LOD selection Frustum Culling Application Portal Culling … Clipping VERTEX SHADERProgrammable Geometry Division by w Processing Assembly Primitive Viewport transform Backface culling Scan Conversion Rasterization FRAGMENT SHADER Output Output to Device
  35. 35. Vertex and Fragment Shaders ( x, y, z, w ) ( x’, y’, z’, w’ ) ( nx, ny, nz ) ( nx’, ny’, nz’ ) ( s, t, r, q ) ( s’, t’, r’, q’ ) ( r, g, b, a ) ( r’, g’, b’, a’ ) VERTEX SHADER( x, y ) ( x, y )( r’, g’, b’, a’ ) ( r, g, b, a )( depth’ ) ( depth ) FRAGMENT SHADER
  36. 36. Embedded code with no lighting OpenGL|ES• Added – Fixed-point and byte data• Retained – Vertex Transforms and Lighting (mostly) Embedded code with lighting – Multi-texturing (2D only) – Full Scene Antialiasing via Multisampling (Smooth Shaded) – Alpha blending Surface Normal Vectors
  37. 37. OpenGL|ES 1.1 Example// Enable fixed-function shading (smooth or flat) glShadeModel(GL_SMOOTH);// Define the appearance of triangle surfaces glMaterialfv(GL_FRONT_AND_BACK, GL_AMBIENT, fMaterialAmbient); glMaterialf(GL_FRONT_AND_BACK, GL_SHININESS, Shininess);// Define the appearance and position of a light source glLightfv(GL_LIGHT0, GL_AMBIENT, fLightAmbient); glLightModelfv(GL_LIGHT_MODEL_AMBIENT, fAmbient); glLightfv(GL_LIGHT0, GL_POSITION, fLightPosition);// Set pointers to geometry and other attributes and draw it glVertexPointer(3, GL_FLOAT, 0, Vertices); glTexCoordPointer(2, GL_FLOAT, 0, TexCoords); glNormalPointer(GL_FLOAT, 0, NormalsPerVertex); glDrawArrays(GL_TRIANGLES, 0, Count); 38
  38. 38. OpenGL|ES 2.0 Example// Create a vertex shader object, load source code and compile it hVertexShader = glCreateShader(GL_VERTEX_SHADER); glShaderSource(hVertexShader, 1, pVertexShaderSourceCode, NULL); glCompileShader(hVertexShader);// Create a shader program and attach the fragment and vertex shaders to it hProgram = glCreateProgram(); glAttachShader(hProgram, hFragmentShader); glAttachShader(hProgram, hVertexShader);// Link and load the new shader programs into the PowerVR SGX glLinkProgram(hProgram); glUseProgram(hProgram);// Set pointers to geometry and other attributes and send to the vertex shader glVertexAttribPointer(Index, 3, GL_FLOAT, GL_TRUE, Stride, pAttributes); glDrawArrays(GL_TRIANGLES, 0, Count); 39
  39. 39. GPU = Graphics Processing Unit Pipelining 1 2 3  Number of stages Parallelism 1 2  Number of parallel processes 3 Parallelism + pipelining 1 2 3  Number of parallel pipelines 1 2 3 Operates on 4 tuples 1 2 3 – Position ( x, y, z, w ) – Color( red, green, blue, alpha ) – Texture Coordinates ( s, t, r, q )4 tuple ops, 1 clock cycle  SIMD [ Single Instruction Multiple Data ]
  40. 40. Reyes(scanline,polygon) Raytracing Optimized by SIMD Hard to be optimized by SIMD Computing dependency Dynamic fast-paths required
  41. 41. Specialize color space convertion takes lots of time. BGRA 444R → RGBA 8888Technique Speedup depends on src/dest format: – 5.4x speedup on average, 19.3x max speedup: (13.3MB/s to 257.7MB/s)
  42. 42. LLVM = Low Level Virtual Machine
  43. 43. Mandelbort is optimized through LLVM JIT up to 11x.
  44. 44. Android OpenGL|ES
  45. 45. Android SurfaceFlinger Properties  Can combine 2D/3D surfaces and surfaces from multiple applications  Surfaces passed as buffers via Binder IPC calls  Can use OpenGL ES and 2D hardware accelerator for its compositions  Double-buffering using page-flip
  46. 46. UI Element GLSurfaceView javax .microedition .khronos .openglesView and Canvas → SurfaceView and Canvas → Surface android. view. View android. view. Surface android graphics. Canvas . Java Graphics JAVA is implemented Framework via JNI Native Framework Surface JNI Graphic JNI OpenGL JNI SurfaceFlinger skia OpenGL|ES Libui + Libui + SurfaceFlinger + SurfaceFlinger + libpixelflinger libpixelflinger libui Overlay Camera Surface Key / Event format EglWindows libpixelflinger FrameBuffer Event Input Driver Driver
  47. 47. PixelFlinger : software renderer• Render functions: pointx, linex, recti, trianglex• Texture and color buffer: activeTexture, bindTexture, colorBuffer, readBuffer, depthBuffer, BindTextureLod• …• Device framebuffer functions: copyPixels, rasterPos2x, rasterPos2i• Optimizer: codeflinger (JIT assembler) I/SurfaceFlinger( 1931): OpenGL informations: I/SurfaceFlinger( 1931): vendor    : Android I/SurfaceFlinger( 1931): renderer : Android PixelFlinger 1.2 I/SurfaceFlinger( 1931): version   : OpenGL ES­CM 1.0
  48. 48. GGLAssembler –GGLAssembler –codeflingers JIT Assemblercodeflingers JIT Assembler
  49. 49. commit 77cadd2ffada95bb3279552e1a29f4bcf4012228 commit 77cadd2ffada95bb3279552e1a29f4bcf4012228 Author: Jim Huang <> Author: Jim Huang <> Date:   Wed Jan 13 01:01:18 2010 +0800 Date:   Wed Jan 13 01:01:18 2010 +0800     [libpixelflinger] Adds UXTB16 support to Pixelflinger     [libpixelflinger] Adds UXTB16 support to Pixelflinger     ...     ...     Uses UXTB16 to extract channels for SIMD operations, rather than creating and     Uses UXTB16 to extract channels for SIMD operations, rather than creating and     ANDing with masks. Saves a register and is faster on A8, as UXTB16 result can     ANDing with masks. Saves a register and is faster on A8, as UXTB16 result can     feed into first stage of multiply, unlike AND.     feed into first stage of multiply, unlike AND.               Also, used SMULWB rather than SMULBB, which allows removal of MOVs used to     Also, used SMULWB rather than SMULBB, which allows removal of MOVs used to     rescale results.     rescale results.               Code has been scheduled for A8 pipeline, specifically aiming to allow     Code has been scheduled for A8 pipeline, specifically aiming to allow     multiplies to issue in pipeline 0, for efficient dual issue operation.     multiplies to issue in pipeline 0, for efficient dual issue operation.               Testing on SpriteMethodTest (­for­android/) gives     Testing on SpriteMethodTest (­for­android/) gives     8% improvement (12.7 vs. 13.7 fps.)     8% improvement (12.7 vs. 13.7 fps.)libpixelflinger/codeflinger/ARMAssembler.cpplibpixelflinger/codeflinger/ARMAssembler.cpp@@ ­424,5 +424,15 @@ void ARMAssembler::SMLAW(int cc, int y,@@ ­424,5 +424,15 @@ void ARMAssembler::SMLAW(int cc, int y,     *mPC++ = (cc<<28) | 0x1200080 | (Rd<<16) | (Rn<<12) | (Rs<<8) | (y<<4) | Rm;     *mPC++ = (cc<<28) | 0x1200080 | (Rd<<16) | (Rn<<12) | (Rs<<8) | (y<<4) | Rm; } }  +#if 0+#if 0+#pragma mark ­+#pragma mark ­+#pragma mark Byte/half word extract and extend (ARMv6+ only)...+#pragma mark Byte/half word extract and extend (ARMv6+ only)...+#endif+#endif+++void ARMAssembler::UXTB16(int cc, int Rd, int Rm, int rotate)+void ARMAssembler::UXTB16(int cc, int Rd, int Rm, int rotate)+{+{+    *mPC++ = (cc<<28) | 0x6CF0070 | (Rd<<12) | ((rotate >> 3) << 10) | Rm;+    *mPC++ = (cc<<28) | 0x6CF0070 | (Rd<<12) | ((rotate >> 3) << 10) | Rm; opcode+}+}++ }; // namespace android }; // namespace android  
  50. 50. libpixelflinger/codeflinger/ARMAssemblerProxy.cpplibpixelflinger/codeflinger/ARMAssemblerProxy.cpp@@ ­195,6 +195,10 @@ void ARMAssemblerProxy::SMLAW(int cc, int y, int Rd, int Rm, int Rs, int Rn) {@@ ­195,6 +195,10 @@ void ARMAssemblerProxy::SMLAW(int cc, int y, int Rd, int Rm, int Rs, int Rn) {     mTarget­>SMLAW(cc, y, Rd, Rm, Rs, Rn);     mTarget­>SMLAW(cc, y, Rd, Rm, Rs, Rn); } }  +void ARMAssemblerProxy::UXTB16(int cc, int Rd, int Rm, int rotate) {+void ARMAssemblerProxy::UXTB16(int cc, int Rd, int Rm, int rotate) {+    mTarget­>UXTB16(cc, Rd, Rm, rotate);+    mTarget­>UXTB16(cc, Rd, Rm, rotate);+}+}++   }; // namespace android }; // namespace android   libpixelflinger/codeflinger/texturing.cpp libpixelflinger/codeflinger/texturing.cpp @@ ­868,6 +869,106 @@ void GGLAssembler::filter24( @@ ­868,6 +869,106 @@ void GGLAssembler::filter24(      load(txPtr, texel, 0);      load(txPtr, texel, 0);  }  }     +#if __ARM_ARCH__ >= 6 +#if __ARM_ARCH__ >= 6 +// ARMv6 version, using UXTB16, and scheduled for Cortex­A8 pipeline +// ARMv6 version, using UXTB16, and scheduled for Cortex­A8 pipeline +void GGLAssembler::filter32( +void GGLAssembler::filter32( +        const fragment_parts_t& parts, +        const fragment_parts_t& parts, +        pixel_t& texel, const texture_unit_t& tmu, +        pixel_t& texel, const texture_unit_t& tmu, +        int U, int V, pointer_t& txPtr, +        int U, int V, pointer_t& txPtr, +        int FRAC_BITS) +        int FRAC_BITS) +{ +{ ... ... +    UXTB16(AL, temp, pixel, 0); +    UXTB16(AL, temp, pixel, 0); Reimplement filter32() +    if (round) { +    if (round) { +        ADD(AL, 0, u, u, imm(1<<(adjust­1))); +        ADD(AL, 0, u, u, imm(1<<(adjust­1))); +        MOV(AL, 0, u, reg_imm(u, LSR, adjust)); +        MOV(AL, 0, u, reg_imm(u, LSR, adjust)); +    } In optimized ASM +    } +    LDR(AL, pixellb, txPtr.reg, reg_scale_pre(offsetlb)); +    LDR(AL, pixellb, txPtr.reg, reg_scale_pre(offsetlb)); +    MUL(AL, 0, dh, temp, u); +    MUL(AL, 0, dh, temp, u); +    UXTB16(AL, temp, pixel, 8); +    UXTB16(AL, temp, pixel, 8); +    MUL(AL, 0, dl, temp, u); +    MUL(AL, 0, dl, temp, u); +    RSB(AL, 0, k, u, imm(0x100)); +    RSB(AL, 0, k, u, imm(0x100)); ... ... +#else +#else  void GGLAssembler::filter32(  void GGLAssembler::filter32(          const fragment_parts_t& parts,          const fragment_parts_t& parts,          pixel_t& texel, const texture_unit_t& tmu,          pixel_t& texel, const texture_unit_t& tmu,
  51. 51. Operating SystemApplied in Telephony
  52. 52. RIL → Telephony Radio Interface Layer(RIL)  HAL  Android TelephonyManager  baseband modem  Voice, Data, SMS, SIM, SIMToolkit  Android RIL  AT command RIL API  Android follows GSM TS 27.007 standard RIL implementation  Qualcomm supports CDMA & CDMA-GSM Multi-mode
  53. 53. OS Kernels Monolithic Kernel Micro KernelTraditional UNIX, Linux, … Mach, L4, Symbian OS Exokernel
  54. 54. L4 Microkernel• Developed by Jochen Liedtke in 1995. – German National Research Center for IT• Developed from scratch• 2nd generation microkernel• Small and Fast Kernel – 7 system calls – 12KB• In November 2005, NICTA announced that Qualcomm was deploying NICTAs L4 version on their Mobile Station Modem chipsets.
  55. 55. Pipes and RPC
  56. 56. Communications Processor L4 Interrupt Handler -> Send Message L4 microkernel RPC System- Server call Handler RPC Interrupt RPC RT task Requests fromShared Memory Application Processor to Modem
  57. 57. Virtual MachineApplied in Database
  58. 58. sqlite3_prepare()/*** Compile the UTF­8 encoded SQL statement zSql into a statement handle.*/int sqlite3_prepare(  sqlite3 *db,            /* Database handle. */  const char *zSql,       /* UTF­8 encoded SQL statement. */  int nBytes,             /* Length of zSql in bytes. */  sqlite3_stmt **ppStmt,  /* OUT: A pointer to the prepared                               statement */  const char** pzTail     /* OUT: End of parsed string */){...      sqlite3VdbeSetNumCols(sParse.pVdbe, 5);      sqlite3VdbeSetColName(sParse.pVdbe, 0, COLNAME_NAME, "addr", P3_STATIC);      sqlite3VdbeSetColName(sParse.pVdbe, 1, COLNAME_NAME, "opcode", P3_STATIC);      sqlite3VdbeSetColName(sParse.pVdbe, 2, COLNAME_NAME, "p1", P3_STATIC);      sqlite3VdbeSetColName(sParse.pVdbe, 3, COLNAME_NAME, "p2", P3_STATIC);      sqlite3VdbeSetColName(sParse.pVdbe, 4, COLNAME_NAME, "p3", P3_STATIC);… Opcode! SQLite Virtual Machine (VDBE) 61
  59. 59. …And what about other platforms?The concept can be applied to all VM platformsApplication virtual machines (short list) .NET (CLR) Java Virtual Machine (JVM) Dalvik virtual machine (Google Android) PHP (Zend Engine) Flash Player / AIR - ActionScript Virtual Machine (AVM) SQLite virtual machine (VDBE; Virtual DataBase Engine) Perl virtual machine
  60. 60. Inspiration
  61. 61. Inspiration• Computer science concepts are applied everywhere in Android software stack.• Essential system software brings the further optimizations and feasibility of new technologies.• Advanced compiler techniques are already applied in every domain• Best Practice in Android
  62. 62.