SlideShare una empresa de Scribd logo
1 de 64
Descargar para leer sin conexión
Talk Binary to Me
Maxim Zaks
@iceX33
What’s going to happen
here???
• Rant about JSON
• Explain why binary formats are better
• Save the world
I love JSON
Is this JSON Valid?
Is this JSON Valid?
Is this JSON Valid?
0
22,5
45
67,5
90
JSON JSON minified FlexBuffers FlatBuffers colfer
22
40
70
66
81
Size in bytes
WTF is colfer?
Colfer is binary serialization format
optimized for speed and size
https://github.com/pascaldekloe/colfer
https://github.com/eishay/jvm-serializers/wiki
Ser Time+Deser Time (ns)
{
"name":"Maxim Zaks",
"birthday":{
"year":1981,
"month":6,
"day":12
}
}
{
"name":"Maxim Zaks",
"birthday":{
"year":1981,
"month":6,
"day":12
}
}
package demo
type Person struct {
name text
birthday Date
}
type Date struct {
year uint16
month uint8
day uint8
}
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
0: [10] Maxim Zaks
1:
0: 1981
1: 6
2: 12
<END>
<END>
GENERATED CODE
colfer all the things?
Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data dumps
Machine to Machine
Requirements
• Small payload
• Efficient encoding & decoding
• Formalised / Typed API
• Versioning & compatibility
colfer
• Small payload 👍
• Efficient encoding & decoding 👍
• Formalised / Typed API 👍
• Versioning & compatibility 👎
can we do better?
FlatBuffers is an efficient cross platform
serialization library for games and other
memory constrained apps.
https://github.com/google/flatbuffers
Swift port from yours truly
https://github.com/mzaks/
FlatBuffersSwift
0
1500
3000
4500
6000
JSONSerialization FlexBuffersSwift FlatBuffersSwift
17335
4.771
559
848
5.794
encode in ms decode in ms
https://github.com/mzaks/FlexBuffersSwift
0
125
250
375
500
JSONSerialization FlexBuffersSwift FlatBuffersSwift
0,0111,563
235,762
77
102
436
encode in MB decode in MB
https://github.com/mzaks/FlexBuffersSwift
package demo
type Person struct {
name text
birthday Date
}
type Date struct {
year uint16
month uint8
day uint8
}
table Person {
name : string;
birthday : Date;
}
struct Date {
year : short;
month : byte;
day : byte;
}
root_type Person;
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
root table: 12
vTable size: 8
table size: 12
property offsets:
1: 4
2: 8
vTable: 8
name offset: 8
date inline:
1981
6
12
name:[10]Maxim Zaks[0]
FlatBuffers
• Small payload 👍*
• Efficient encoding & decoding 👍*
• Formalised / Typed API 👍
• Versioning & compatibility 👍
* not as small/fast as colfer
Let’s top that!
Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data dumps
Application state is interconnected
table Message {
...
}
table MessageLink {
message : Message;
nextMessages : [MessageLink];
}
table History {
seenMessages : [Message];
nextMessageLink : MessageLink;
timeStamp : double;
}
root_type History;
http://resiapp.io
Data serialisation use cases
• Machine to Machine communication
• State persistence
• Representation of configurations or data
dumps
Random value access is KING!
Show case:
find city by name or country code
https://github.com/mzaks/FindCityFB
Data dump of all cities in the world with
country code, name, normalised name and
map coordinates
150 MB as CSV
177 MB as FlatBuffers binary
table CityList {
cityByCountryCode : [City];
cityByName : [City];
}
table City {
countryCode : string;
searchName : string;
name : string;
latitude : float;
longitude : float;
}
root_type CityList;
https://youtu.be/P5tRwY39_JU
45MB214MB
table CityList {
cityByCountryCode : [City];
cityByName : [City];
}
table City {
countryCode : string;
searchName : string;
name : string;
latitude : float;
longitude : float;
}
root_type CityList;
by Name
by Country Code
Are we done?
There was one more format
0
1500
3000
4500
6000
JSONSerialization FlexBuffersSwift FlatBuffersSwift
17335
4.771
559
848
5.794
encode in ms decode in ms
Schema-less binary encoding
http://google.github.io/flatbuffers/
flexbuffers.html
https://github.com/mzaks/FlexBuffersSwift
Benefits of FlexBuffers
• No schema, no code generation needed
• Random value access
• Can easily transform JSON to FlexBuffers
• Supports every type JSON does and a bit more
• Keeps type information in the binary
let data = FlexBuffer.dataFrom(jsonData:
"{name:"Maxim", birthday:{"year": 1981, month: 6, day: 12}}"
.data(using: .utf8)!
)
let accessor = FlexBuffer.decode(data:data)
let name = accessor?["name"]?.asString
let day = accessor?["birthday"]?["day"]?.asInt
Outlook
• Don’t use JSON just because other people do, think
about your use case
• Imagine how much more responses your server could
deliver if it took 200x less time to process a request
• Start thinking in Graphs not Trees
• please ask questions after the session
• … and btw. I like to collaborate 😉 @iceX33
References
• All Gifs are powered by giphy.com
• JSON number grammar diagrams taken from json.org
• Binary Visualiser „HEX Fiend“ by ridiculous_fish http://ridiculousfish.com/hexfiend/
• https://github.com/eishay/jvm-serializers/wiki
• https://github.com/pascaldekloe/colfer
• https://github.com/google/flatbuffers
• https://github.com/mzaks/FlatBuffersSwift
• https://github.com/mzaks/FlexBuffersSwift
• https://github.com/mzaks/FindCityFB
• http://resiapp.io

Más contenido relacionado

Similar a Talk Binary to Me

WebSocket JSON Hackday
WebSocket JSON HackdayWebSocket JSON Hackday
WebSocket JSON Hackday
Somay Nakhal
 

Similar a Talk Binary to Me (20)

Beyond JSON @ Mobile.Warsaw
Beyond JSON @ Mobile.WarsawBeyond JSON @ Mobile.Warsaw
Beyond JSON @ Mobile.Warsaw
 
Json at work overview and ecosystem-v2.0
Json at work   overview and ecosystem-v2.0Json at work   overview and ecosystem-v2.0
Json at work overview and ecosystem-v2.0
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesWebscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
 
JSON Web Tokens Will Improve Your Life
JSON Web Tokens Will Improve Your LifeJSON Web Tokens Will Improve Your Life
JSON Web Tokens Will Improve Your Life
 
IT Days - Parse huge JSON files in a streaming way.pptx
IT Days - Parse huge JSON files in a streaming way.pptxIT Days - Parse huge JSON files in a streaming way.pptx
IT Days - Parse huge JSON files in a streaming way.pptx
 
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
N1QL+GSI: Language and Performance Improvements in Couchbase 5.0 and 5.5
 
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018Couchbase Tutorial: Big data Open Source Systems: VLDB2018
Couchbase Tutorial: Big data Open Source Systems: VLDB2018
 
PostgreSQL 9.4 JSON Types and Operators
PostgreSQL 9.4 JSON Types and OperatorsPostgreSQL 9.4 JSON Types and Operators
PostgreSQL 9.4 JSON Types and Operators
 
N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0N1QL: What's new in Couchbase 5.0
N1QL: What's new in Couchbase 5.0
 
OrientDB - cloud barcamp Libero Cloud
OrientDB - cloud barcamp Libero CloudOrientDB - cloud barcamp Libero Cloud
OrientDB - cloud barcamp Libero Cloud
 
JOSE Can You See...
JOSE Can You See...JOSE Can You See...
JOSE Can You See...
 
sbt, history of JSON libraries, microservices, and schema evolution (Tokyo ver)
sbt, history of JSON libraries, microservices, and schema evolution (Tokyo ver)sbt, history of JSON libraries, microservices, and schema evolution (Tokyo ver)
sbt, history of JSON libraries, microservices, and schema evolution (Tokyo ver)
 
Real-time Semantic Web with Twitter Annotations
Real-time Semantic Web with Twitter AnnotationsReal-time Semantic Web with Twitter Annotations
Real-time Semantic Web with Twitter Annotations
 
Edição de Texto Rico com React e Draft.js
Edição de Texto Rico com React e Draft.jsEdição de Texto Rico com React e Draft.js
Edição de Texto Rico com React e Draft.js
 
Spark Streaming with Cassandra
Spark Streaming with CassandraSpark Streaming with Cassandra
Spark Streaming with Cassandra
 
WebSocket JSON Hackday
WebSocket JSON HackdayWebSocket JSON Hackday
WebSocket JSON Hackday
 
MongoDB .local Toronto 2019: MongoDB Atlas Search Deep Dive
MongoDB .local Toronto 2019: MongoDB Atlas Search Deep DiveMongoDB .local Toronto 2019: MongoDB Atlas Search Deep Dive
MongoDB .local Toronto 2019: MongoDB Atlas Search Deep Dive
 
Data science at the command line
Data science at the command lineData science at the command line
Data science at the command line
 
Cassandra 3.0 - JSON at scale - StampedeCon 2015
Cassandra 3.0 - JSON at scale - StampedeCon 2015Cassandra 3.0 - JSON at scale - StampedeCon 2015
Cassandra 3.0 - JSON at scale - StampedeCon 2015
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 

Más de Maxim Zaks

Swift the implicit parts
Swift the implicit partsSwift the implicit parts
Swift the implicit parts
Maxim Zaks
 
96% macoun 2013
96% macoun 201396% macoun 2013
96% macoun 2013
Maxim Zaks
 
Diagnose of Agile @ Wooga 04.2013
Diagnose of Agile @ Wooga 04.2013Diagnose of Agile @ Wooga 04.2013
Diagnose of Agile @ Wooga 04.2013
Maxim Zaks
 
Start playing @ mobile.cologne 2013
Start playing @ mobile.cologne 2013Start playing @ mobile.cologne 2013
Start playing @ mobile.cologne 2013
Maxim Zaks
 
Under Cocos2D Tree @mdvecon 2013
Under Cocos2D Tree @mdvecon 2013Under Cocos2D Tree @mdvecon 2013
Under Cocos2D Tree @mdvecon 2013
Maxim Zaks
 
Don&rsquo;t do Agile, be Agile @NSConf 2013
Don&rsquo;t do Agile, be Agile @NSConf 2013Don&rsquo;t do Agile, be Agile @NSConf 2013
Don&rsquo;t do Agile, be Agile @NSConf 2013
Maxim Zaks
 
Test Essentials @mdevcon 2012
Test Essentials @mdevcon 2012Test Essentials @mdevcon 2012
Test Essentials @mdevcon 2012
Maxim Zaks
 
Vergiss Java konzentrier Dich auf Script @ OOP2013
Vergiss Java konzentrier Dich auf Script @ OOP2013Vergiss Java konzentrier Dich auf Script @ OOP2013
Vergiss Java konzentrier Dich auf Script @ OOP2013
Maxim Zaks
 

Más de Maxim Zaks (20)

Entity Component System - a different approach to game and app development
Entity Component System - a different approach to game and app developmentEntity Component System - a different approach to game and app development
Entity Component System - a different approach to game and app development
 
Nitty Gritty of Data Serialisation
Nitty Gritty of Data SerialisationNitty Gritty of Data Serialisation
Nitty Gritty of Data Serialisation
 
Wind of change
Wind of changeWind of change
Wind of change
 
Data model mal anders
Data model mal andersData model mal anders
Data model mal anders
 
Entity Component System - for App developers
Entity Component System - for App developersEntity Component System - for App developers
Entity Component System - for App developers
 
Beyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffersBeyond JSON - An Introduction to FlatBuffers
Beyond JSON - An Introduction to FlatBuffers
 
Beyond JSON @ dot swift 2016
Beyond JSON @ dot swift 2016Beyond JSON @ dot swift 2016
Beyond JSON @ dot swift 2016
 
Basics of Computer Science
Basics of Computer ScienceBasics of Computer Science
Basics of Computer Science
 
Entity system architecture with Unity @Unite Europe 2015
Entity system architecture with Unity @Unite Europe 2015 Entity system architecture with Unity @Unite Europe 2015
Entity system architecture with Unity @Unite Europe 2015
 
UIKonf App & Data Driven Design @swift.berlin
UIKonf App & Data Driven Design @swift.berlinUIKonf App & Data Driven Design @swift.berlin
UIKonf App & Data Driven Design @swift.berlin
 
Swift the implicit parts
Swift the implicit partsSwift the implicit parts
Swift the implicit parts
 
Currying in Swift
Currying in SwiftCurrying in Swift
Currying in Swift
 
Promise of an API
Promise of an APIPromise of an API
Promise of an API
 
96% macoun 2013
96% macoun 201396% macoun 2013
96% macoun 2013
 
Diagnose of Agile @ Wooga 04.2013
Diagnose of Agile @ Wooga 04.2013Diagnose of Agile @ Wooga 04.2013
Diagnose of Agile @ Wooga 04.2013
 
Start playing @ mobile.cologne 2013
Start playing @ mobile.cologne 2013Start playing @ mobile.cologne 2013
Start playing @ mobile.cologne 2013
 
Under Cocos2D Tree @mdvecon 2013
Under Cocos2D Tree @mdvecon 2013Under Cocos2D Tree @mdvecon 2013
Under Cocos2D Tree @mdvecon 2013
 
Don&rsquo;t do Agile, be Agile @NSConf 2013
Don&rsquo;t do Agile, be Agile @NSConf 2013Don&rsquo;t do Agile, be Agile @NSConf 2013
Don&rsquo;t do Agile, be Agile @NSConf 2013
 
Test Essentials @mdevcon 2012
Test Essentials @mdevcon 2012Test Essentials @mdevcon 2012
Test Essentials @mdevcon 2012
 
Vergiss Java konzentrier Dich auf Script @ OOP2013
Vergiss Java konzentrier Dich auf Script @ OOP2013Vergiss Java konzentrier Dich auf Script @ OOP2013
Vergiss Java konzentrier Dich auf Script @ OOP2013
 

Último

The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
heathfieldcps1
 

Último (20)

How to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POSHow to Manage Global Discount in Odoo 17 POS
How to Manage Global Discount in Odoo 17 POS
 
Fostering Friendships - Enhancing Social Bonds in the Classroom
Fostering Friendships - Enhancing Social Bonds  in the ClassroomFostering Friendships - Enhancing Social Bonds  in the Classroom
Fostering Friendships - Enhancing Social Bonds in the Classroom
 
Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024Mehran University Newsletter Vol-X, Issue-I, 2024
Mehran University Newsletter Vol-X, Issue-I, 2024
 
Interdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptxInterdisciplinary_Insights_Data_Collection_Methods.pptx
Interdisciplinary_Insights_Data_Collection_Methods.pptx
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptx
 
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
2024-NATIONAL-LEARNING-CAMP-AND-OTHER.pptx
 
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdfUnit 3 Emotional Intelligence and Spiritual Intelligence.pdf
Unit 3 Emotional Intelligence and Spiritual Intelligence.pdf
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...Kodo Millet  PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
Kodo Millet PPT made by Ghanshyam bairwa college of Agriculture kumher bhara...
 
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
 
The basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptxThe basics of sentences session 3pptx.pptx
The basics of sentences session 3pptx.pptx
 
REMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptxREMIFENTANIL: An Ultra short acting opioid.pptx
REMIFENTANIL: An Ultra short acting opioid.pptx
 
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
Sensory_Experience_and_Emotional_Resonance_in_Gabriel_Okaras_The_Piano_and_Th...
 
On_Translating_a_Tamil_Poem_by_A_K_Ramanujan.pptx
On_Translating_a_Tamil_Poem_by_A_K_Ramanujan.pptxOn_Translating_a_Tamil_Poem_by_A_K_Ramanujan.pptx
On_Translating_a_Tamil_Poem_by_A_K_Ramanujan.pptx
 
Plant propagation: Sexual and Asexual propapagation.pptx
Plant propagation: Sexual and Asexual propapagation.pptxPlant propagation: Sexual and Asexual propapagation.pptx
Plant propagation: Sexual and Asexual propapagation.pptx
 
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
NO1 Top Black Magic Specialist In Lahore Black magic In Pakistan Kala Ilam Ex...
 
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
80 ĐỀ THI THỬ TUYỂN SINH TIẾNG ANH VÀO 10 SỞ GD – ĐT THÀNH PHỐ HỒ CHÍ MINH NĂ...
 
How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17How to Create and Manage Wizard in Odoo 17
How to Create and Manage Wizard in Odoo 17
 
General Principles of Intellectual Property: Concepts of Intellectual Proper...
General Principles of Intellectual Property: Concepts of Intellectual  Proper...General Principles of Intellectual Property: Concepts of Intellectual  Proper...
General Principles of Intellectual Property: Concepts of Intellectual Proper...
 
Sociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning ExhibitSociology 101 Demonstration of Learning Exhibit
Sociology 101 Demonstration of Learning Exhibit
 

Talk Binary to Me