SlideShare una empresa de Scribd logo
1 de 37
Descargar para leer sin conexión
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Taro L. Saito, Ph.D.
GitHub: @xerial
Arm Treasure Data
Airframe: Lightweight Building
Blocks for Scala
November 16th, 2018
Scale By The Bay 2018 - Unconference
1
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
About Me: Taro L. Saito (Leo)
● An Engineer with Research Background
● Ph.D., University of Tokyo
● DBMS & Genome Science
● Leading Query Engine Team
● Active OSS Developer
● airframe
● sqlite-jdbc
■ More than 1000 GitHub stars
● snappy-java
■ Compression library used in
Spark, Parquet
● sbt-sonatype
■ Used in 2000+ Scala projects
● ...
2
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe
● Lightweight Building Blocks for Scala
● Essential libraries for building any applications
● Used in production for 2+ years
● Based on my code collection since 2009
● Initially written in Java
● Gradually migrated to Scala
● Repackaged into wvlet.airframe in 2016
● For maintainability
● 18 Modules
● Simplifying your daily programming in Scala
3
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
18 Airframe Modules
● Bootstrap
● airframe-config Configuration loader
● airframe-launcher Command-line program launcher
● Object Serialization
● airframe-codec encoder/decoder SPI + standard codecs
● airframe-msgpack pure-Scala MessagePack implementation
● airframe-tablet CSV/TSV/JSON/JDBC ResultSet <-> Object
● Monitoring & Debugging
● airframe-log Logging
● airframe-metrics Human-readable metrics for time, date, data size, etc
● airframe-jmx Object metrics provider through JMX
● Building Service Objects
● airframe Dependency injection
● airframe-surface Object type inspector
● Misc:
● airframe-control, airframe-jdbc, airframe-json, airframe-http, etc.
4
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Utilities for Debugging Applications
5
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Debugging Applications: Airframe Log
● Airframe Log: A Modern Logging Library for Scala (Medium Blog)
● ANSI color, source code location support
6
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe Metrics
● Human Readable Data Format (Duration, DataSize, etc.)
● Handy Time Window String Support
7
“-1d”
“-1w”
“-7d”
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe JMX
● Checking the internal states of remote JVM processes
● JMX clients
● jconsole has JMX metric monitor
● Airframe JMX -> DataDog -> Dashboard
8
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe Launcher
● Using Class & Function Command Line Programs
9
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Dependency Injection with Airframe
10
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Today’s Goals
● Lean How to Use Airframe DI (Dependency Injection)
● Understand what can be simplified with DI
● Learn 5 Airframe DI Design Patterns
● That improve the thought processes in programming
11
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
What is Dependency Injection (DI)?
● Many Articles…
● Inversion of Control Containers and the Dependency Injection pattern. Martin
Fowler (2004)
● StackOverflow, Wikipedia, …
● Many Frameworks...
● Spring, Google Guice, Scaldi, Macwire, Grafter, Weld, etc.
● No framework approaches also exist (Pure-Scala DI)
● Recent Definition:
● Dependency Injection is the process of creating the static, stateless graph of
service objects, where each service is parameterised by its dependencies.
■ What is Dependency Injection? by Adam Warski (2018)
● However, it’s still difficult to understand what is DI
12
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Simplifying DI with Airframe
● Airframe Usage
● import wvlet.airframe._
● Simple 3 Step DI
● bind
● design
● build
13
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Design Patterns In Airframe
● Pattern 0 (basic): Building Service Objects with Auto-Wiring
● Pattern 1: Configuring Modules
● Pattern 2: Switching Bindings for Tests
● Pattern 3: Managing Lifecycle in FILO order
● Pattern 4: Bundling Service Traits
● Flower-bundle pattern
● Pattern 5: Binding Factory
14
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Constructor vs In-Trait Injections
● Constructor Injection
● In-Trait Injection
15
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Building Service Objects
● When coding A and B
● You can focus on only direct dependencies
● You can forget about indirect dependencies
● Airframe DI builds A, B, and direct/indirect dependencies on your behalf.
A
DB
Connection
Pool
DB
Client
DB Monitor
Fluentd
Logger
HttpClient
B
16
You can forget this part
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Code Example
● Focus on Logic
17
A
DB
Client
Fluentd
Logger
B
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Hand Wiring vs Auto Wiring
18
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Configuring Modules
● How to pass configuration objects to corresponding modules?
● new A(new DBClient(new ConnectionPool(new DB(new DBConfig(...)), new
ConnectionPoolConfig(...), new DBClientConfig(...)))
● Things You Need to Remember
● Argument orders (or argument names in Scala) of individual modules
● How to instantiate modules
A
DB
Connection
Pool
DB
Client
DB Monitor
Fluentd
Logger
HttpClient
B
19
DB
Config
Connection
Pool Config
HttpClient
Config
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 1: Adding Config
● Put config to the closest place in the code
20
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 2: Switching Bindings For Testing
● In Airframe Design
● You can replace DB and FluentdLogger to In-Memory Impl
● How to build A and B differs, but the same code can be used
A
Memory
DB
Connection
Pool
DB
Client
DB Monitor
Fluentd
Logger
In-memory
Logger
B
21
Overriding Design for Testing
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 2: Switching Bindings For Testing
22
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 3: Managing Lifecycle
● onStart, onShutdown hooks
● JSR-250 annotations
23
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Complex FILO Order Resource Management
● FILO := First-In Last-Out
● Airframe registers onStart and onShutdown lifecycle hooks when creating
instances
● When closing sessions, onShutdown will be called in the reverse order
● Dependencies forms DAG
● Dependencies will be generated when creating new service objects
A
DB
Connection
Pool
DB
Client
DB Monitor
Fluentd
Logger
HttpClient
B
134
56
7
2
8
Shared
Resource
24
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 4: Bundling Service Traits
● Flower-Bundle Pattern
● Create composable services
25
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Pattern 5: Bind Factory
● Create a factory that change partial dependencies
● e.g., creating differently configured instances of Databases
26
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
3 Things You Can Forget With Airframe DI
● 1. How to Build Service Objects
● config, auto-wiring, flower bundle pattern
● 2. How to Manage Resource Lifecycle
● FILO order
● 3. How to Use DI Itself (!!)
● Only need to understand bind, design, and build.
27
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Summary: Reducing Code Complexity with Airframe DI
● You can effectively forget about:
● How to build service objects
● How to manage resources in FILO order
● How to use DI itself
A
DB
Connection
Pool
DB
Client
DB Monitor
Fluentd
Logger
HttpClient
B
134
56
7
2
8
28
Implementation Details
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe Internals
29
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Details
● bind[X]
● X becomes singleton
● Usually there is no need to declare bind[X].toSingleton
■ Unless you want to initialize X early with production mode
● bindInstance[X]
● When you need to call new X(d1, d2, …) every time
● bindFactory[A1 => X]
● For customizing A1 in X
● Design
● Immutable & Serializable
● design1 + design2 + ….
■ Overriding the previous design (adding order is important)
● Session
● withLazyMode/withProductionMode
● noLifeCycleLogging
30
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Injecting Airframe Session with Scala Macros
31
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Airframe Surface: Object Shape Inspector
● Reading Type Signatures From ScalaSig
● Scala compiler embeds Scala Type Signatures (ScalaSig) to class files
● Surface supports
● type alias, tagged type
● higher-kinded types
class A (data:List[B])
class A
data: List[java.lang.Object]
class A
data: List[java.lang.Object]
ScalaSig: data:List[B]
javac
scalac
Surface.of[A]
data: List[B]
scala.reflect.runtime.
universe.TypeTag
Type Erasure
32
???
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Future Work & Summary
33
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Current State of Airframe
● Version 0.73 (As of November 2018)
● We already had 40+ releases in 2018
● Automated Release
● Cross building libraries for Scala 2.11, 2.12, 2.13, and Scala.js
● ‘sbt release’ command took 3 hours
■ Sequential steps:
○ compile -> test -> package -> upload x 18 modules x 4 Scala versions
● Now a new version can be released in 10 minutes on Travis CI
● Blog
● 3 Tips for Maintaining Scala Projects
● Future Work
● Adding child session support
● Support multiple constructor argument blocks
34
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Philosophy: Simplicity By Design
● “Simplicity” by Philippe Dufour
● A clock made by a legendary
watchmaker in Switzerland
● Every part of the clock is built
by himself
● Airframe
● Provides simplicity for
application developers
35
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Summary
● To understand DI, think about what you can
simplify
● How to build objects
● How to manage resources (FILO)
● Learning DI framework itself
● 5 Airframe Design Patterns
● Pattern 1: Configuring Modules
● Pattern 2: Overriding Bindings for Tests
● Pattern 3: Managing Lifecycle in FILO order
● Pattern 4: Bundling Service Traits
■ Flower-bundle pattern
● Pattern 5: Binding Factory
Don’t Forget Adding GitHub Star!
wvlet/airframe
36
Confidential © Arm 2017Confidential © Arm 2017Confidential © Arm 2017
Thank You!
Danke!
Merci!
谢谢!
ありがとう!
Gracias!
Kiitos!
37

Más contenido relacionado

La actualidad más candente

Madeo - a CAD Tool for reconfigurable Hardware
Madeo - a CAD Tool for reconfigurable HardwareMadeo - a CAD Tool for reconfigurable Hardware
Madeo - a CAD Tool for reconfigurable HardwareESUG
 
Functional APIs with Absinthe GraphQL
Functional APIs with Absinthe GraphQLFunctional APIs with Absinthe GraphQL
Functional APIs with Absinthe GraphQLZvi Avraham
 
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...Flink Forward
 
Introduction to Apache Hivemall v0.5.0
Introduction to Apache Hivemall v0.5.0Introduction to Apache Hivemall v0.5.0
Introduction to Apache Hivemall v0.5.0Makoto Yui
 
Grokking Techtalk #38: Escape Analysis in Go compiler
 Grokking Techtalk #38: Escape Analysis in Go compiler Grokking Techtalk #38: Escape Analysis in Go compiler
Grokking Techtalk #38: Escape Analysis in Go compilerGrokking VN
 
Applying Concurrency Cookbook Recipes to SPEC JBB
Applying Concurrency Cookbook Recipes to SPEC JBBApplying Concurrency Cookbook Recipes to SPEC JBB
Applying Concurrency Cookbook Recipes to SPEC JBBMonica Beckwith
 
OpenJDK Concurrent Collectors
OpenJDK Concurrent CollectorsOpenJDK Concurrent Collectors
OpenJDK Concurrent CollectorsMonica Beckwith
 
Supporting the "Rapi" C-laguage API in an R-compatible engine
Supporting the "Rapi" C-laguage API in an R-compatible engineSupporting the "Rapi" C-laguage API in an R-compatible engine
Supporting the "Rapi" C-laguage API in an R-compatible engineadamwelc
 
Understanding Implicits in Scala
Understanding Implicits in ScalaUnderstanding Implicits in Scala
Understanding Implicits in Scaladatamantra
 
Anaconda Python KNIME & Orange Installation
Anaconda Python KNIME & Orange InstallationAnaconda Python KNIME & Orange Installation
Anaconda Python KNIME & Orange InstallationGirinath Pillai
 
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...Flink Forward
 
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORS
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORSOPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORS
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORSMonica Beckwith
 
Functional programming in Scala
Functional programming in ScalaFunctional programming in Scala
Functional programming in Scaladatamantra
 
Flink Forward San Francisco 2018 keynote: Anand Iyer - "Apache Flink + Apach...
Flink Forward San Francisco 2018 keynote:  Anand Iyer - "Apache Flink + Apach...Flink Forward San Francisco 2018 keynote:  Anand Iyer - "Apache Flink + Apach...
Flink Forward San Francisco 2018 keynote: Anand Iyer - "Apache Flink + Apach...Flink Forward
 

La actualidad más candente (16)

Madeo - a CAD Tool for reconfigurable Hardware
Madeo - a CAD Tool for reconfigurable HardwareMadeo - a CAD Tool for reconfigurable Hardware
Madeo - a CAD Tool for reconfigurable Hardware
 
Functional APIs with Absinthe GraphQL
Functional APIs with Absinthe GraphQLFunctional APIs with Absinthe GraphQL
Functional APIs with Absinthe GraphQL
 
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...
Flink Forward Berlin 2017: Roberto Bentivoglio, Saverio Veltri - NSDB (Natura...
 
Introduction to Apache Hivemall v0.5.0
Introduction to Apache Hivemall v0.5.0Introduction to Apache Hivemall v0.5.0
Introduction to Apache Hivemall v0.5.0
 
Grokking Techtalk #38: Escape Analysis in Go compiler
 Grokking Techtalk #38: Escape Analysis in Go compiler Grokking Techtalk #38: Escape Analysis in Go compiler
Grokking Techtalk #38: Escape Analysis in Go compiler
 
HDF5 and Storage Resource Broker
HDF5 and Storage Resource BrokerHDF5 and Storage Resource Broker
HDF5 and Storage Resource Broker
 
Applying Concurrency Cookbook Recipes to SPEC JBB
Applying Concurrency Cookbook Recipes to SPEC JBBApplying Concurrency Cookbook Recipes to SPEC JBB
Applying Concurrency Cookbook Recipes to SPEC JBB
 
OpenJDK Concurrent Collectors
OpenJDK Concurrent CollectorsOpenJDK Concurrent Collectors
OpenJDK Concurrent Collectors
 
Supporting the "Rapi" C-laguage API in an R-compatible engine
Supporting the "Rapi" C-laguage API in an R-compatible engineSupporting the "Rapi" C-laguage API in an R-compatible engine
Supporting the "Rapi" C-laguage API in an R-compatible engine
 
Understanding Implicits in Scala
Understanding Implicits in ScalaUnderstanding Implicits in Scala
Understanding Implicits in Scala
 
Anaconda Python KNIME & Orange Installation
Anaconda Python KNIME & Orange InstallationAnaconda Python KNIME & Orange Installation
Anaconda Python KNIME & Orange Installation
 
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...
Flink Forward San Francisco 2019: Managing Flink on Kubernetes - FlinkK8sOper...
 
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORS
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORSOPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORS
OPENJDK: IN THE NEW AGE OF CONCURRENT GARBAGE COLLECTORS
 
Functional programming in Scala
Functional programming in ScalaFunctional programming in Scala
Functional programming in Scala
 
Flink Forward San Francisco 2018 keynote: Anand Iyer - "Apache Flink + Apach...
Flink Forward San Francisco 2018 keynote:  Anand Iyer - "Apache Flink + Apach...Flink Forward San Francisco 2018 keynote:  Anand Iyer - "Apache Flink + Apach...
Flink Forward San Francisco 2018 keynote: Anand Iyer - "Apache Flink + Apach...
 
Certificates
CertificatesCertificates
Certificates
 

Similar a Airframe: Lightweight Building Blocks for Scala - Scale By The Bay 2018

Reading The Source Code of Presto
Reading The Source Code of PrestoReading The Source Code of Presto
Reading The Source Code of PrestoTaro L. Saito
 
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17Muga Nishizawa
 
Revisit Dependency Injection in scala
Revisit Dependency Injection in scalaRevisit Dependency Injection in scala
Revisit Dependency Injection in scalatakezoe
 
Unifying Frontend and Backend Development with Scala - ScalaCon 2021
Unifying Frontend and Backend Development with Scala - ScalaCon 2021Unifying Frontend and Backend Development with Scala - ScalaCon 2021
Unifying Frontend and Backend Development with Scala - ScalaCon 2021Taro L. Saito
 
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...Cloud Native Day Tel Aviv
 
From shipping rpms to helm charts - Lessons learned and best practices
From shipping rpms to helm charts - Lessons learned and best practicesFrom shipping rpms to helm charts - Lessons learned and best practices
From shipping rpms to helm charts - Lessons learned and best practicesAnkush Chadha, MBA, MS
 
Перехват функций (хуки) под Windows в приложениях с помощью C/C++
Перехват функций (хуки) под Windows в приложениях с помощью C/C++Перехват функций (хуки) под Windows в приложениях с помощью C/C++
Перехват функций (хуки) под Windows в приложениях с помощью C/C++corehard_by
 
Developer insight into why applications run amazingly Fast in CF 2018
Developer insight into why applications run amazingly Fast in CF 2018Developer insight into why applications run amazingly Fast in CF 2018
Developer insight into why applications run amazingly Fast in CF 2018Pavan Kumar
 
Managing Machine Learning workflows on Treasure Data
Managing Machine Learning workflows on Treasure DataManaging Machine Learning workflows on Treasure Data
Managing Machine Learning workflows on Treasure DataAki Ariga
 
Presto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 UpdatesPresto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 UpdatesTaro L. Saito
 
Diagnose Your Microservices
Diagnose Your MicroservicesDiagnose Your Microservices
Diagnose Your MicroservicesMarcus Hirt
 
Let’s template
Let’s templateLet’s template
Let’s templateAllenKao7
 
[TGDF 2019] Mali GPU Architecture and Mobile Studio
[TGDF 2019] Mali GPU Architecture and Mobile Studio[TGDF 2019] Mali GPU Architecture and Mobile Studio
[TGDF 2019] Mali GPU Architecture and Mobile StudioOwen Wu
 
Five cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark fasterFive cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark fasterTim Ellison
 
Make your PySpark Data Fly with Arrow!
Make your PySpark Data Fly with Arrow!Make your PySpark Data Fly with Arrow!
Make your PySpark Data Fly with Arrow!Databricks
 
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020Taro L. Saito
 
Containerized MySQL OpenWorld talk
Containerized MySQL OpenWorld talkContainerized MySQL OpenWorld talk
Containerized MySQL OpenWorld talkPatrick Galbraith
 
BUD17-310: Introducing LLDB for linux on Arm and AArch64
BUD17-310: Introducing LLDB for linux on Arm and AArch64 BUD17-310: Introducing LLDB for linux on Arm and AArch64
BUD17-310: Introducing LLDB for linux on Arm and AArch64 Linaro
 
Distributed Resource Management Application API (DRMAA) Version 2
Distributed Resource Management Application API (DRMAA) Version 2Distributed Resource Management Application API (DRMAA) Version 2
Distributed Resource Management Application API (DRMAA) Version 2Peter Tröger
 
Artifactory Essentials Workshop on August 27, 2020 by JFrog
Artifactory Essentials Workshop on August 27, 2020 by JFrogArtifactory Essentials Workshop on August 27, 2020 by JFrog
Artifactory Essentials Workshop on August 27, 2020 by JFrogCloud Study Network
 

Similar a Airframe: Lightweight Building Blocks for Scala - Scale By The Bay 2018 (20)

Reading The Source Code of Presto
Reading The Source Code of PrestoReading The Source Code of Presto
Reading The Source Code of Presto
 
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17
Custom Script Execution Environment on TD Workflow @ TD Tech Talk 2018-10-17
 
Revisit Dependency Injection in scala
Revisit Dependency Injection in scalaRevisit Dependency Injection in scala
Revisit Dependency Injection in scala
 
Unifying Frontend and Backend Development with Scala - ScalaCon 2021
Unifying Frontend and Backend Development with Scala - ScalaCon 2021Unifying Frontend and Backend Development with Scala - ScalaCon 2021
Unifying Frontend and Backend Development with Scala - ScalaCon 2021
 
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...
Kubernetes is hard! Lessons learned taking our apps to Kubernetes - Eldad Ass...
 
From shipping rpms to helm charts - Lessons learned and best practices
From shipping rpms to helm charts - Lessons learned and best practicesFrom shipping rpms to helm charts - Lessons learned and best practices
From shipping rpms to helm charts - Lessons learned and best practices
 
Перехват функций (хуки) под Windows в приложениях с помощью C/C++
Перехват функций (хуки) под Windows в приложениях с помощью C/C++Перехват функций (хуки) под Windows в приложениях с помощью C/C++
Перехват функций (хуки) под Windows в приложениях с помощью C/C++
 
Developer insight into why applications run amazingly Fast in CF 2018
Developer insight into why applications run amazingly Fast in CF 2018Developer insight into why applications run amazingly Fast in CF 2018
Developer insight into why applications run amazingly Fast in CF 2018
 
Managing Machine Learning workflows on Treasure Data
Managing Machine Learning workflows on Treasure DataManaging Machine Learning workflows on Treasure Data
Managing Machine Learning workflows on Treasure Data
 
Presto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 UpdatesPresto At Arm Treasure Data - 2019 Updates
Presto At Arm Treasure Data - 2019 Updates
 
Diagnose Your Microservices
Diagnose Your MicroservicesDiagnose Your Microservices
Diagnose Your Microservices
 
Let’s template
Let’s templateLet’s template
Let’s template
 
[TGDF 2019] Mali GPU Architecture and Mobile Studio
[TGDF 2019] Mali GPU Architecture and Mobile Studio[TGDF 2019] Mali GPU Architecture and Mobile Studio
[TGDF 2019] Mali GPU Architecture and Mobile Studio
 
Five cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark fasterFive cool ways the JVM can run Apache Spark faster
Five cool ways the JVM can run Apache Spark faster
 
Make your PySpark Data Fly with Arrow!
Make your PySpark Data Fly with Arrow!Make your PySpark Data Fly with Arrow!
Make your PySpark Data Fly with Arrow!
 
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020
Journey of Migrating 1 Million Presto Queries - Presto Webinar 2020
 
Containerized MySQL OpenWorld talk
Containerized MySQL OpenWorld talkContainerized MySQL OpenWorld talk
Containerized MySQL OpenWorld talk
 
BUD17-310: Introducing LLDB for linux on Arm and AArch64
BUD17-310: Introducing LLDB for linux on Arm and AArch64 BUD17-310: Introducing LLDB for linux on Arm and AArch64
BUD17-310: Introducing LLDB for linux on Arm and AArch64
 
Distributed Resource Management Application API (DRMAA) Version 2
Distributed Resource Management Application API (DRMAA) Version 2Distributed Resource Management Application API (DRMAA) Version 2
Distributed Resource Management Application API (DRMAA) Version 2
 
Artifactory Essentials Workshop on August 27, 2020 by JFrog
Artifactory Essentials Workshop on August 27, 2020 by JFrogArtifactory Essentials Workshop on August 27, 2020 by JFrog
Artifactory Essentials Workshop on August 27, 2020 by JFrog
 

Más de Taro L. Saito

Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020
Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020
Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020Taro L. Saito
 
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020Taro L. Saito
 
Tips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTaro L. Saito
 
Learning Silicon Valley Culture
Learning Silicon Valley CultureLearning Silicon Valley Culture
Learning Silicon Valley CultureTaro L. Saito
 
Presto At Treasure Data
Presto At Treasure DataPresto At Treasure Data
Presto At Treasure DataTaro L. Saito
 
Scala at Treasure Data
Scala at Treasure DataScala at Treasure Data
Scala at Treasure DataTaro L. Saito
 
Introduction to Presto at Treasure Data
Introduction to Presto at Treasure DataIntroduction to Presto at Treasure Data
Introduction to Presto at Treasure DataTaro L. Saito
 
Workflow Hacks #1 - dots. Tokyo
Workflow Hacks #1 - dots. TokyoWorkflow Hacks #1 - dots. Tokyo
Workflow Hacks #1 - dots. TokyoTaro L. Saito
 
Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015Taro L. Saito
 
Presto As A Service - Treasure DataでのPresto運用事例
Presto As A Service - Treasure DataでのPresto運用事例Presto As A Service - Treasure DataでのPresto運用事例
Presto As A Service - Treasure DataでのPresto運用事例Taro L. Saito
 
Presto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoringPresto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoringTaro L. Saito
 
Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編Taro L. Saito
 
Weaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, TokyoWeaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, TokyoTaro L. Saito
 
Spark Internals - Hadoop Source Code Reading #16 in Japan
Spark Internals - Hadoop Source Code Reading #16 in JapanSpark Internals - Hadoop Source Code Reading #16 in Japan
Spark Internals - Hadoop Source Code Reading #16 in JapanTaro L. Saito
 
Streaming Distributed Data Processing with Silk #deim2014
Streaming Distributed Data Processing with Silk #deim2014Streaming Distributed Data Processing with Silk #deim2014
Streaming Distributed Data Processing with Silk #deim2014Taro L. Saito
 
Silkによる並列分散ワークフロープログラミング
Silkによる並列分散ワークフロープログラミングSilkによる並列分散ワークフロープログラミング
Silkによる並列分散ワークフロープログラミングTaro L. Saito
 
2011年度 生物データベース論 2日目 木構造データ
2011年度 生物データベース論 2日目 木構造データ2011年度 生物データベース論 2日目 木構造データ
2011年度 生物データベース論 2日目 木構造データTaro L. Saito
 
Relational-Style XML Query @ SIGMOD-J 2008 Dec.
Relational-Style XML Query @ SIGMOD-J 2008 Dec.Relational-Style XML Query @ SIGMOD-J 2008 Dec.
Relational-Style XML Query @ SIGMOD-J 2008 Dec.Taro L. Saito
 

Más de Taro L. Saito (19)

Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020
Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020
Scala for Everything: From Frontend to Backend Applications - Scala Matsuri 2020
 
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020
td-spark internals: Extending Spark with Airframe - Spark Meetup Tokyo #3 2020
 
Tips For Maintaining OSS Projects
Tips For Maintaining OSS ProjectsTips For Maintaining OSS Projects
Tips For Maintaining OSS Projects
 
Learning Silicon Valley Culture
Learning Silicon Valley CultureLearning Silicon Valley Culture
Learning Silicon Valley Culture
 
Presto At Treasure Data
Presto At Treasure DataPresto At Treasure Data
Presto At Treasure Data
 
Scala at Treasure Data
Scala at Treasure DataScala at Treasure Data
Scala at Treasure Data
 
Introduction to Presto at Treasure Data
Introduction to Presto at Treasure DataIntroduction to Presto at Treasure Data
Introduction to Presto at Treasure Data
 
Workflow Hacks #1 - dots. Tokyo
Workflow Hacks #1 - dots. TokyoWorkflow Hacks #1 - dots. Tokyo
Workflow Hacks #1 - dots. Tokyo
 
Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015
 
Presto As A Service - Treasure DataでのPresto運用事例
Presto As A Service - Treasure DataでのPresto運用事例Presto As A Service - Treasure DataでのPresto運用事例
Presto As A Service - Treasure DataでのPresto運用事例
 
JNuma Library
JNuma LibraryJNuma Library
JNuma Library
 
Presto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoringPresto as a Service - Tips for operation and monitoring
Presto as a Service - Tips for operation and monitoring
 
Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編Treasure Dataを支える技術 - MessagePack編
Treasure Dataを支える技術 - MessagePack編
 
Weaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, TokyoWeaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
Weaving Dataflows with Silk - ScalaMatsuri 2014, Tokyo
 
Spark Internals - Hadoop Source Code Reading #16 in Japan
Spark Internals - Hadoop Source Code Reading #16 in JapanSpark Internals - Hadoop Source Code Reading #16 in Japan
Spark Internals - Hadoop Source Code Reading #16 in Japan
 
Streaming Distributed Data Processing with Silk #deim2014
Streaming Distributed Data Processing with Silk #deim2014Streaming Distributed Data Processing with Silk #deim2014
Streaming Distributed Data Processing with Silk #deim2014
 
Silkによる並列分散ワークフロープログラミング
Silkによる並列分散ワークフロープログラミングSilkによる並列分散ワークフロープログラミング
Silkによる並列分散ワークフロープログラミング
 
2011年度 生物データベース論 2日目 木構造データ
2011年度 生物データベース論 2日目 木構造データ2011年度 生物データベース論 2日目 木構造データ
2011年度 生物データベース論 2日目 木構造データ
 
Relational-Style XML Query @ SIGMOD-J 2008 Dec.
Relational-Style XML Query @ SIGMOD-J 2008 Dec.Relational-Style XML Query @ SIGMOD-J 2008 Dec.
Relational-Style XML Query @ SIGMOD-J 2008 Dec.
 

Último

Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptx
Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptxSemiconductor Physics Background and Light Emitting Diode(LEDs)-.pptx
Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptxbhoomijyani51
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Amil baba
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical SensorTanvir Moin
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfJulia Kaye
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxSAJITHABANUS
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technologyabdulkadirmukarram03
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Amil baba
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide LaboratoryBahzad5
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...amrabdallah9
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxKISHAN KUMAR
 
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...Gaurav Singh Rajput
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabusViolet Violet
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS Bahzad5
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Apollo Techno Industries Pvt Ltd
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecTrupti Shiralkar, CISSP
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxLMW Machine Tool Division
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptxSaiGouthamSunkara
 

Último (20)

Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptx
Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptxSemiconductor Physics Background and Light Emitting Diode(LEDs)-.pptx
Semiconductor Physics Background and Light Emitting Diode(LEDs)-.pptx
 
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
Popular-NO1 Kala Jadu Expert Specialist In Germany Kala Jadu Expert Specialis...
 
Basic Principle of Electrochemical Sensor
Basic Principle of  Electrochemical SensorBasic Principle of  Electrochemical Sensor
Basic Principle of Electrochemical Sensor
 
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdfsdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
sdfsadopkjpiosufoiasdoifjasldkjfl a asldkjflaskdjflkjsdsdf
 
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptxIT3401-WEB ESSENTIALS PRESENTATIONS.pptx
IT3401-WEB ESSENTIALS PRESENTATIONS.pptx
 
The relationship between iot and communication technology
The relationship between iot and communication technologyThe relationship between iot and communication technology
The relationship between iot and communication technology
 
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
Best-NO1 Best Rohani Amil In Lahore Kala Ilam In Lahore Kala Jadu Amil In Lah...
 
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratoryدليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
دليل تجارب الاسفلت المختبرية - Asphalt Experiments Guide Laboratory
 
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
Strategies of Urban Morphologyfor Improving Outdoor Thermal Comfort and Susta...
 
Lecture 2 .pdf
Lecture 2                           .pdfLecture 2                           .pdf
Lecture 2 .pdf
 
Mohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptxMohs Scale of Hardness, Hardness Scale.pptx
Mohs Scale of Hardness, Hardness Scale.pptx
 
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...
Lifting Plan | Lifting Plan for Different Process Equipment | Gaurav Singh Ra...
 
cloud computing notes for anna university syllabus
cloud computing notes for anna university syllabuscloud computing notes for anna university syllabus
cloud computing notes for anna university syllabus
 
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS GENERAL CONDITIONS  FOR  CONTRACTS OF CIVIL ENGINEERING WORKS
GENERAL CONDITIONS FOR CONTRACTS OF CIVIL ENGINEERING WORKS
 
Litature Review: Research Paper work for Engineering
Litature Review: Research Paper work for EngineeringLitature Review: Research Paper work for Engineering
Litature Review: Research Paper work for Engineering
 
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...Technology Features of Apollo HDD Machine, Its Technical Specification with C...
Technology Features of Apollo HDD Machine, Its Technical Specification with C...
 
Lecture 4 .pdf
Lecture 4                              .pdfLecture 4                              .pdf
Lecture 4 .pdf
 
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSecGuardians and Glitches: Navigating the Duality of Gen AI in AppSec
Guardians and Glitches: Navigating the Duality of Gen AI in AppSec
 
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptxVertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
Vertical- Machining - Center - VMC -LMW-Machine-Tool-Division.pptx
 
Phase noise transfer functions.pptx
Phase noise transfer      functions.pptxPhase noise transfer      functions.pptx
Phase noise transfer functions.pptx
 

Airframe: Lightweight Building Blocks for Scala - Scale By The Bay 2018

  • 1. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Taro L. Saito, Ph.D. GitHub: @xerial Arm Treasure Data Airframe: Lightweight Building Blocks for Scala November 16th, 2018 Scale By The Bay 2018 - Unconference 1
  • 2. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. About Me: Taro L. Saito (Leo) ● An Engineer with Research Background ● Ph.D., University of Tokyo ● DBMS & Genome Science ● Leading Query Engine Team ● Active OSS Developer ● airframe ● sqlite-jdbc ■ More than 1000 GitHub stars ● snappy-java ■ Compression library used in Spark, Parquet ● sbt-sonatype ■ Used in 2000+ Scala projects ● ... 2
  • 3. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe ● Lightweight Building Blocks for Scala ● Essential libraries for building any applications ● Used in production for 2+ years ● Based on my code collection since 2009 ● Initially written in Java ● Gradually migrated to Scala ● Repackaged into wvlet.airframe in 2016 ● For maintainability ● 18 Modules ● Simplifying your daily programming in Scala 3
  • 4. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. 18 Airframe Modules ● Bootstrap ● airframe-config Configuration loader ● airframe-launcher Command-line program launcher ● Object Serialization ● airframe-codec encoder/decoder SPI + standard codecs ● airframe-msgpack pure-Scala MessagePack implementation ● airframe-tablet CSV/TSV/JSON/JDBC ResultSet <-> Object ● Monitoring & Debugging ● airframe-log Logging ● airframe-metrics Human-readable metrics for time, date, data size, etc ● airframe-jmx Object metrics provider through JMX ● Building Service Objects ● airframe Dependency injection ● airframe-surface Object type inspector ● Misc: ● airframe-control, airframe-jdbc, airframe-json, airframe-http, etc. 4
  • 5. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Utilities for Debugging Applications 5
  • 6. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Debugging Applications: Airframe Log ● Airframe Log: A Modern Logging Library for Scala (Medium Blog) ● ANSI color, source code location support 6
  • 7. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Metrics ● Human Readable Data Format (Duration, DataSize, etc.) ● Handy Time Window String Support 7 “-1d” “-1w” “-7d”
  • 8. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe JMX ● Checking the internal states of remote JVM processes ● JMX clients ● jconsole has JMX metric monitor ● Airframe JMX -> DataDog -> Dashboard 8
  • 9. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Launcher ● Using Class & Function Command Line Programs 9
  • 10. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Dependency Injection with Airframe 10
  • 11. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Today’s Goals ● Lean How to Use Airframe DI (Dependency Injection) ● Understand what can be simplified with DI ● Learn 5 Airframe DI Design Patterns ● That improve the thought processes in programming 11
  • 12. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. What is Dependency Injection (DI)? ● Many Articles… ● Inversion of Control Containers and the Dependency Injection pattern. Martin Fowler (2004) ● StackOverflow, Wikipedia, … ● Many Frameworks... ● Spring, Google Guice, Scaldi, Macwire, Grafter, Weld, etc. ● No framework approaches also exist (Pure-Scala DI) ● Recent Definition: ● Dependency Injection is the process of creating the static, stateless graph of service objects, where each service is parameterised by its dependencies. ■ What is Dependency Injection? by Adam Warski (2018) ● However, it’s still difficult to understand what is DI 12
  • 13. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Simplifying DI with Airframe ● Airframe Usage ● import wvlet.airframe._ ● Simple 3 Step DI ● bind ● design ● build 13
  • 14. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Design Patterns In Airframe ● Pattern 0 (basic): Building Service Objects with Auto-Wiring ● Pattern 1: Configuring Modules ● Pattern 2: Switching Bindings for Tests ● Pattern 3: Managing Lifecycle in FILO order ● Pattern 4: Bundling Service Traits ● Flower-bundle pattern ● Pattern 5: Binding Factory 14
  • 15. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Constructor vs In-Trait Injections ● Constructor Injection ● In-Trait Injection 15
  • 16. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Building Service Objects ● When coding A and B ● You can focus on only direct dependencies ● You can forget about indirect dependencies ● Airframe DI builds A, B, and direct/indirect dependencies on your behalf. A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 16 You can forget this part
  • 17. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Code Example ● Focus on Logic 17 A DB Client Fluentd Logger B
  • 18. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Hand Wiring vs Auto Wiring 18
  • 19. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Configuring Modules ● How to pass configuration objects to corresponding modules? ● new A(new DBClient(new ConnectionPool(new DB(new DBConfig(...)), new ConnectionPoolConfig(...), new DBClientConfig(...))) ● Things You Need to Remember ● Argument orders (or argument names in Scala) of individual modules ● How to instantiate modules A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 19 DB Config Connection Pool Config HttpClient Config
  • 20. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 1: Adding Config ● Put config to the closest place in the code 20
  • 21. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 2: Switching Bindings For Testing ● In Airframe Design ● You can replace DB and FluentdLogger to In-Memory Impl ● How to build A and B differs, but the same code can be used A Memory DB Connection Pool DB Client DB Monitor Fluentd Logger In-memory Logger B 21 Overriding Design for Testing
  • 22. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 2: Switching Bindings For Testing 22
  • 23. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 3: Managing Lifecycle ● onStart, onShutdown hooks ● JSR-250 annotations 23
  • 24. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Complex FILO Order Resource Management ● FILO := First-In Last-Out ● Airframe registers onStart and onShutdown lifecycle hooks when creating instances ● When closing sessions, onShutdown will be called in the reverse order ● Dependencies forms DAG ● Dependencies will be generated when creating new service objects A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 134 56 7 2 8 Shared Resource 24
  • 25. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 4: Bundling Service Traits ● Flower-Bundle Pattern ● Create composable services 25
  • 26. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Pattern 5: Bind Factory ● Create a factory that change partial dependencies ● e.g., creating differently configured instances of Databases 26
  • 27. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. 3 Things You Can Forget With Airframe DI ● 1. How to Build Service Objects ● config, auto-wiring, flower bundle pattern ● 2. How to Manage Resource Lifecycle ● FILO order ● 3. How to Use DI Itself (!!) ● Only need to understand bind, design, and build. 27
  • 28. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Summary: Reducing Code Complexity with Airframe DI ● You can effectively forget about: ● How to build service objects ● How to manage resources in FILO order ● How to use DI itself A DB Connection Pool DB Client DB Monitor Fluentd Logger HttpClient B 134 56 7 2 8 28 Implementation Details
  • 29. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Internals 29
  • 30. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Details ● bind[X] ● X becomes singleton ● Usually there is no need to declare bind[X].toSingleton ■ Unless you want to initialize X early with production mode ● bindInstance[X] ● When you need to call new X(d1, d2, …) every time ● bindFactory[A1 => X] ● For customizing A1 in X ● Design ● Immutable & Serializable ● design1 + design2 + …. ■ Overriding the previous design (adding order is important) ● Session ● withLazyMode/withProductionMode ● noLifeCycleLogging 30
  • 31. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Injecting Airframe Session with Scala Macros 31
  • 32. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Airframe Surface: Object Shape Inspector ● Reading Type Signatures From ScalaSig ● Scala compiler embeds Scala Type Signatures (ScalaSig) to class files ● Surface supports ● type alias, tagged type ● higher-kinded types class A (data:List[B]) class A data: List[java.lang.Object] class A data: List[java.lang.Object] ScalaSig: data:List[B] javac scalac Surface.of[A] data: List[B] scala.reflect.runtime. universe.TypeTag Type Erasure 32 ???
  • 33. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Future Work & Summary 33
  • 34. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Current State of Airframe ● Version 0.73 (As of November 2018) ● We already had 40+ releases in 2018 ● Automated Release ● Cross building libraries for Scala 2.11, 2.12, 2.13, and Scala.js ● ‘sbt release’ command took 3 hours ■ Sequential steps: ○ compile -> test -> package -> upload x 18 modules x 4 Scala versions ● Now a new version can be released in 10 minutes on Travis CI ● Blog ● 3 Tips for Maintaining Scala Projects ● Future Work ● Adding child session support ● Support multiple constructor argument blocks 34
  • 35. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Philosophy: Simplicity By Design ● “Simplicity” by Philippe Dufour ● A clock made by a legendary watchmaker in Switzerland ● Every part of the clock is built by himself ● Airframe ● Provides simplicity for application developers 35
  • 36. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Summary ● To understand DI, think about what you can simplify ● How to build objects ● How to manage resources (FILO) ● Learning DI framework itself ● 5 Airframe Design Patterns ● Pattern 1: Configuring Modules ● Pattern 2: Overriding Bindings for Tests ● Pattern 3: Managing Lifecycle in FILO order ● Pattern 4: Bundling Service Traits ■ Flower-bundle pattern ● Pattern 5: Binding Factory Don’t Forget Adding GitHub Star! wvlet/airframe 36
  • 37. Confidential © Arm 2017Confidential © Arm 2017Confidential © Arm 2017 Thank You! Danke! Merci! 谢谢! ありがとう! Gracias! Kiitos! 37