SlideShare a Scribd company logo
1 of 22
Download to read offline
Introduction of
Kobe Digital Labo, Inc.
Takahiro Iwase
Twitter: @okuyamaoo
http://d.hatena.ne.jp/okuyamaoo/
Distributed Key-Value Store
1.okuyama Overview
2.okuyama feature list
3.okuyama unique features
4.Information okuyama
Outline
・okuyama is Implemented in Java
>Started development in December 2009
Now Version 0.8.2
・Made from scratch
>Persistence mechanism, Network mechanism,
Replication mechanism
okuyama Overview
・Whole composition
・Client → Master Node → Data Node(×3)
Master Node
Data Node Data Node
Data Node Data Node
Client
Client
Master NodeClient
okuyama Overview
Data Node
Data Node
Data Node Data NodeData Node
Data Node Data NodeData Node
okuyama feature list
・Feature List
Persistent Type
Redundancy
Data Maintenance
Memory, Disk, Disk + Memory
Consists of multiple servers
Server can be added non-disruptively
All data is stored multiplexed
All components can be multiplexed
No okuyama signle point of failure
If servers are added or server failure
occur, data dynamic reconstruction are
automatically executed
Support Protocol Original, memcached, HTTP
Availability
okuyama feature list
・Persistece Mechanism
・Memory
The fastest
・Disk
Persistent and big data
Key and Value Management in a separate file
・Disk + Memory
The fast and persistent
Type1: After recoding file stored in memory
Type2: After recoding a file, Key only stored in
memory
okuyama feature list
・Network Mechanism
Accept
Queue
ConnectClient
Accept Queue
Worker
I/O Read
Queue
Write
Client
I/O Read Queue
Worker
Main Job
Queue
Client
Main Job Queue
Worker
Respond to
okuyama feature list
・Network Mechanism
In addition to multiplexing
Accept
Queue
ConnectClient
I/O Read
Queue
Write
Client
Main Job
Queue
Client
Respond to
Main Job
Queue
Main Job
Queue
I/O Read
Queue
I/O Read
Queue
Accept
Queue
Accept
Queue
okuyama unique features
・Feature 1.
Tags can be added to the data
okuyama unique features
・Feature 1.
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open source database”
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Tags can be added to the data
okuyama unique features
・Feature 1.
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open source database”
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open source database”
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Tag
NOSQL
RDBMS
NOSQL
Tags can be added to the data
okuyama unique features
・Feature 1.
Tags can be added to the data
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open source database”
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Tag
NOSQL
RDBMS
NOSQL
If you specify a tag …
okuyama unique features
・Feature 1.
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open source database”
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Tag
NOSQL
RDBMS
NOSQL
If you specify a tag …
{“Hibari”, “okuyama”}
You can get the keys at once
Tags can be added to the data
okuyama unique features
・Feature 2.
Sent from the client JavaScript
can run in DataNode
okuyama unique features
・Feature 2.
Sent from the client JavaScript
can run in DataNode
Main
Data Node
Client Master Node
Request
Key=“okuyama”
JavaScript=”if (dataValue == „nosql‟) { …..”
okuyama unique features
・Feature 2.
Sent from the client JavaScript
can run in DataNode
Client Master Node
Main
Data Node
Key=“okuyama”
JavaScript=”if (dataValue == „nosql‟) { …..”
Request
Key=okuyama
Value=“nosql”
Execute JavaScript
okuyama unique features
・Feature 2.
Sent from the client JavaScript
can run in DataNode
Client Master Node
Main
Data Node
Key=“okuyama”
JavaScript=”if (dataValue == „nosql‟) { …..”
Request
Key=okuyama
Value=“nosql”
Execute JavaScript
The execution result is returned
okuyama unique features
・Feature 3.
Data lock function
okuyama unique features
・Feature 3.
Data lock function
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open …..
“Hibari is implemented in Erlang”
“okuyama is implemented in Java”
Client
A
Lock
Success
Lock request
okuyama unique features
・Feature 3.
Data lock function
Key Value
Hibari
MySQL
okuyama
“The world‟s most popular open …..
“Hibari is implemented in Erlang”
“Change Value”
Client
A
Error
Set Request
“Change
Value”
Client
B
Set Request
Success
Information okuyama
・Information
UserGroup
http://groups.google.com/group/kvs_okuyama/
Development
http://sourceforge.jp/projects/okuyama/
Twitter Hashtag
#okuyama
Thank you!

More Related Content

What's hot

Building a production ready meteor app
Building a production ready meteor appBuilding a production ready meteor app
Building a production ready meteor appRitik Malhotra
 
Integrating OpenStack with Active Directory
Integrating OpenStack with Active DirectoryIntegrating OpenStack with Active Directory
Integrating OpenStack with Active Directorycjellick
 
SQLite: Light, Open Source Relational Database Management System
SQLite: Light, Open Source Relational Database Management SystemSQLite: Light, Open Source Relational Database Management System
SQLite: Light, Open Source Relational Database Management SystemTanner Jessel
 
Backup and Restore SQL Server Databases in Microsoft Azure
Backup and Restore SQL Server Databases in Microsoft AzureBackup and Restore SQL Server Databases in Microsoft Azure
Backup and Restore SQL Server Databases in Microsoft AzureDatavail
 
Introduction to Elastic Beanstalk
Introduction to Elastic BeanstalkIntroduction to Elastic Beanstalk
Introduction to Elastic BeanstalkWolfgang Schell
 
Scala play-framework
Scala play-frameworkScala play-framework
Scala play-frameworkAbdhesh Kumar
 
MySQL High Availability Deep Dive
MySQL High Availability Deep DiveMySQL High Availability Deep Dive
MySQL High Availability Deep Divehastexo
 
Moving applications to the cloud
Moving applications to the cloudMoving applications to the cloud
Moving applications to the cloudSergejus Barinovas
 
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...Sencha
 
DjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling DisqusDjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling Disquszeeg
 
Wordpress optimization
Wordpress optimizationWordpress optimization
Wordpress optimizationAlmog Baku
 
High performance java ee with j cache and cdi
High performance java ee with j cache and cdiHigh performance java ee with j cache and cdi
High performance java ee with j cache and cdiPayara
 
PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company
PyData London 2015 - How We Turned EverythingMe Into a Data Driven CompanyPyData London 2015 - How We Turned EverythingMe Into a Data Driven Company
PyData London 2015 - How We Turned EverythingMe Into a Data Driven CompanyArik Fraimovich
 
Learn to love networking on iOS
Learn to love networking on iOSLearn to love networking on iOS
Learn to love networking on iOSPaolo Tagliani
 
Practicing Continuous Deployment
Practicing Continuous DeploymentPracticing Continuous Deployment
Practicing Continuous Deploymentzeeg
 

What's hot (20)

Building a production ready meteor app
Building a production ready meteor appBuilding a production ready meteor app
Building a production ready meteor app
 
Integrating OpenStack with Active Directory
Integrating OpenStack with Active DirectoryIntegrating OpenStack with Active Directory
Integrating OpenStack with Active Directory
 
SQLite: Light, Open Source Relational Database Management System
SQLite: Light, Open Source Relational Database Management SystemSQLite: Light, Open Source Relational Database Management System
SQLite: Light, Open Source Relational Database Management System
 
Backup and Restore SQL Server Databases in Microsoft Azure
Backup and Restore SQL Server Databases in Microsoft AzureBackup and Restore SQL Server Databases in Microsoft Azure
Backup and Restore SQL Server Databases in Microsoft Azure
 
Introduction to Elastic Beanstalk
Introduction to Elastic BeanstalkIntroduction to Elastic Beanstalk
Introduction to Elastic Beanstalk
 
Scala play-framework
Scala play-frameworkScala play-framework
Scala play-framework
 
MySQL High Availability Deep Dive
MySQL High Availability Deep DiveMySQL High Availability Deep Dive
MySQL High Availability Deep Dive
 
Moving applications to the cloud
Moving applications to the cloudMoving applications to the cloud
Moving applications to the cloud
 
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
SenchaCon 2016: Building a Faceted Catalog of Video Game Assets Using Ext JS ...
 
AfNetworking vs. Native + Caching
AfNetworking vs. Native + CachingAfNetworking vs. Native + Caching
AfNetworking vs. Native + Caching
 
DjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling DisqusDjangoCon 2010 Scaling Disqus
DjangoCon 2010 Scaling Disqus
 
SQLite 3
SQLite 3SQLite 3
SQLite 3
 
Wordpress optimization
Wordpress optimizationWordpress optimization
Wordpress optimization
 
High performance java ee with j cache and cdi
High performance java ee with j cache and cdiHigh performance java ee with j cache and cdi
High performance java ee with j cache and cdi
 
Introduction to CQ5
Introduction to CQ5Introduction to CQ5
Introduction to CQ5
 
Gradle - Build System
Gradle - Build SystemGradle - Build System
Gradle - Build System
 
PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company
PyData London 2015 - How We Turned EverythingMe Into a Data Driven CompanyPyData London 2015 - How We Turned EverythingMe Into a Data Driven Company
PyData London 2015 - How We Turned EverythingMe Into a Data Driven Company
 
Azure Service Fabric Mesh
Azure Service Fabric MeshAzure Service Fabric Mesh
Azure Service Fabric Mesh
 
Learn to love networking on iOS
Learn to love networking on iOSLearn to love networking on iOS
Learn to love networking on iOS
 
Practicing Continuous Deployment
Practicing Continuous DeploymentPracticing Continuous Deployment
Practicing Continuous Deployment
 

Similar to okuyama_20101101_nosqlafternoon

2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 Geir Høydalsvik
 
Java Memory Hogs.pdf
Java Memory Hogs.pdfJava Memory Hogs.pdf
Java Memory Hogs.pdfGurbinder3
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsAchievers Tech
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlsqlhjalp
 
ELK Ruminating on Logs (Zendcon 2016)
ELK Ruminating on Logs (Zendcon 2016)ELK Ruminating on Logs (Zendcon 2016)
ELK Ruminating on Logs (Zendcon 2016)Mathew Beane
 
MySQL 5.6, news in 5.7 and our HA options
MySQL 5.6, news in 5.7 and our HA optionsMySQL 5.6, news in 5.7 and our HA options
MySQL 5.6, news in 5.7 and our HA optionsTed Wennmark
 
Oasis – data analysis platform for enterprise
Oasis – data analysis platform for enterpriseOasis – data analysis platform for enterprise
Oasis – data analysis platform for enterpriseLINE Corporation
 
MySQL Manchester TT - 5.7 Whats new
MySQL Manchester TT - 5.7 Whats newMySQL Manchester TT - 5.7 Whats new
MySQL Manchester TT - 5.7 Whats newMark Swarbrick
 
MySQL Performance Tuning at COSCUP 2014
MySQL Performance Tuning at COSCUP 2014MySQL Performance Tuning at COSCUP 2014
MySQL Performance Tuning at COSCUP 2014Ryusuke Kajiyama
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?DATAVERSITY
 
Installing your Oracle Software.ppt
Installing your Oracle Software.pptInstalling your Oracle Software.ppt
Installing your Oracle Software.pptMohammedHdi1
 
Netflix oss season 2 episode 1 - meetup Lightning talks
Netflix oss   season 2 episode 1 - meetup Lightning talksNetflix oss   season 2 episode 1 - meetup Lightning talks
Netflix oss season 2 episode 1 - meetup Lightning talksRuslan Meshenberg
 
MySQL London Tech Tour March 2015 - Embedded Database of Choice
MySQL London Tech Tour March 2015 - Embedded Database of ChoiceMySQL London Tech Tour March 2015 - Embedded Database of Choice
MySQL London Tech Tour March 2015 - Embedded Database of ChoiceMark Swarbrick
 
10 Things to Consider When Using Apache Kafka
10 Things to Consider When Using Apache Kafka10 Things to Consider When Using Apache Kafka
10 Things to Consider When Using Apache KafkaNaoto Umemori
 
Ruby and Distributed Storage Systems
Ruby and Distributed Storage SystemsRuby and Distributed Storage Systems
Ruby and Distributed Storage SystemsSATOSHI TAGOMORI
 
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014Manuel Contreras
 

Similar to okuyama_20101101_nosqlafternoon (20)

2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015 2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
2015: Whats New in MySQL 5.7, At Oracle Open World, November 3rd, 2015
 
Java Memory Hogs.pdf
Java Memory Hogs.pdfJava Memory Hogs.pdf
Java Memory Hogs.pdf
 
Profiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty DetailsProfiling and Tuning a Web Application - The Dirty Details
Profiling and Tuning a Web Application - The Dirty Details
 
PostgreSQL and MySQL
PostgreSQL and MySQLPostgreSQL and MySQL
PostgreSQL and MySQL
 
Session 307 ravi pendekanti engineered systems
Session 307  ravi pendekanti engineered systemsSession 307  ravi pendekanti engineered systems
Session 307 ravi pendekanti engineered systems
 
My sql crashcourse_intro_kdl
My sql crashcourse_intro_kdlMy sql crashcourse_intro_kdl
My sql crashcourse_intro_kdl
 
ELK Ruminating on Logs (Zendcon 2016)
ELK Ruminating on Logs (Zendcon 2016)ELK Ruminating on Logs (Zendcon 2016)
ELK Ruminating on Logs (Zendcon 2016)
 
MySQL 5.6, news in 5.7 and our HA options
MySQL 5.6, news in 5.7 and our HA optionsMySQL 5.6, news in 5.7 and our HA options
MySQL 5.6, news in 5.7 and our HA options
 
Oasis – data analysis platform for enterprise
Oasis – data analysis platform for enterpriseOasis – data analysis platform for enterprise
Oasis – data analysis platform for enterprise
 
MySQL Manchester TT - 5.7 Whats new
MySQL Manchester TT - 5.7 Whats newMySQL Manchester TT - 5.7 Whats new
MySQL Manchester TT - 5.7 Whats new
 
Collect, summarize and notify of OpenStack's log
Collect, summarize and notify of OpenStack's logCollect, summarize and notify of OpenStack's log
Collect, summarize and notify of OpenStack's log
 
MySQL Performance Tuning at COSCUP 2014
MySQL Performance Tuning at COSCUP 2014MySQL Performance Tuning at COSCUP 2014
MySQL Performance Tuning at COSCUP 2014
 
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
A Case Study of NoSQL Adoption: What Drove Wordnik Non-Relational?
 
Installing your Oracle Software.ppt
Installing your Oracle Software.pptInstalling your Oracle Software.ppt
Installing your Oracle Software.ppt
 
MySQL NoSQL APIs
MySQL NoSQL APIsMySQL NoSQL APIs
MySQL NoSQL APIs
 
Netflix oss season 2 episode 1 - meetup Lightning talks
Netflix oss   season 2 episode 1 - meetup Lightning talksNetflix oss   season 2 episode 1 - meetup Lightning talks
Netflix oss season 2 episode 1 - meetup Lightning talks
 
MySQL London Tech Tour March 2015 - Embedded Database of Choice
MySQL London Tech Tour March 2015 - Embedded Database of ChoiceMySQL London Tech Tour March 2015 - Embedded Database of Choice
MySQL London Tech Tour March 2015 - Embedded Database of Choice
 
10 Things to Consider When Using Apache Kafka
10 Things to Consider When Using Apache Kafka10 Things to Consider When Using Apache Kafka
10 Things to Consider When Using Apache Kafka
 
Ruby and Distributed Storage Systems
Ruby and Distributed Storage SystemsRuby and Distributed Storage Systems
Ruby and Distributed Storage Systems
 
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014
Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014
 

More from Takahiro Iwase

Re port aws_reinvent_161213_slideshare
Re port aws_reinvent_161213_slideshareRe port aws_reinvent_161213_slideshare
Re port aws_reinvent_161213_slideshareTakahiro Iwase
 
20151205 中国地方db勉強会 dbm_fs
20151205 中国地方db勉強会 dbm_fs20151205 中国地方db勉強会 dbm_fs
20151205 中国地方db勉強会 dbm_fsTakahiro Iwase
 
20140418 info talkセミナー資料
20140418 info talkセミナー資料20140418 info talkセミナー資料
20140418 info talkセミナー資料Takahiro Iwase
 
20131113_mysql_on_分散fsセミナー資料
20131113_mysql_on_分散fsセミナー資料20131113_mysql_on_分散fsセミナー資料
20131113_mysql_on_分散fsセミナー資料Takahiro Iwase
 
20121205 nosql(okuyama fs)セミナー資料
20121205 nosql(okuyama fs)セミナー資料20121205 nosql(okuyama fs)セミナー資料
20121205 nosql(okuyama fs)セミナー資料Takahiro Iwase
 
20120913 nosql@hikarie(okuyama fuse)
20120913 nosql@hikarie(okuyama fuse)20120913 nosql@hikarie(okuyama fuse)
20120913 nosql@hikarie(okuyama fuse)Takahiro Iwase
 
20120405 setsunaセミナー
20120405 setsunaセミナー20120405 setsunaセミナー
20120405 setsunaセミナーTakahiro Iwase
 
20120317 IT系勉強会 in 神戸
20120317 IT系勉強会 in 神戸20120317 IT系勉強会 in 神戸
20120317 IT系勉強会 in 神戸Takahiro Iwase
 
Okuyama説明資料 20120119 ss
Okuyama説明資料 20120119 ssOkuyama説明資料 20120119 ss
Okuyama説明資料 20120119 ssTakahiro Iwase
 
20110708 dist_study okuyama
20110708 dist_study okuyama20110708 dist_study okuyama
20110708 dist_study okuyamaTakahiro Iwase
 
20110519 okuyama tokyo_linuxstudy
20110519 okuyama tokyo_linuxstudy20110519 okuyama tokyo_linuxstudy
20110519 okuyama tokyo_linuxstudyTakahiro Iwase
 
20110517 okuyama ソーシャルメディアが育てた技術勉強会
20110517 okuyama ソーシャルメディアが育てた技術勉強会20110517 okuyama ソーシャルメディアが育てた技術勉強会
20110517 okuyama ソーシャルメディアが育てた技術勉強会Takahiro Iwase
 
2010/07/09 osc kansai-kvsokuyama
2010/07/09 osc kansai-kvsokuyama2010/07/09 osc kansai-kvsokuyama
2010/07/09 osc kansai-kvsokuyamaTakahiro Iwase
 

More from Takahiro Iwase (14)

Re port aws_reinvent_161213_slideshare
Re port aws_reinvent_161213_slideshareRe port aws_reinvent_161213_slideshare
Re port aws_reinvent_161213_slideshare
 
20151205 中国地方db勉強会 dbm_fs
20151205 中国地方db勉強会 dbm_fs20151205 中国地方db勉強会 dbm_fs
20151205 中国地方db勉強会 dbm_fs
 
20140418 info talkセミナー資料
20140418 info talkセミナー資料20140418 info talkセミナー資料
20140418 info talkセミナー資料
 
20131113_mysql_on_分散fsセミナー資料
20131113_mysql_on_分散fsセミナー資料20131113_mysql_on_分散fsセミナー資料
20131113_mysql_on_分散fsセミナー資料
 
20121205 nosql(okuyama fs)セミナー資料
20121205 nosql(okuyama fs)セミナー資料20121205 nosql(okuyama fs)セミナー資料
20121205 nosql(okuyama fs)セミナー資料
 
20120913 nosql@hikarie(okuyama fuse)
20120913 nosql@hikarie(okuyama fuse)20120913 nosql@hikarie(okuyama fuse)
20120913 nosql@hikarie(okuyama fuse)
 
20120405 setsunaセミナー
20120405 setsunaセミナー20120405 setsunaセミナー
20120405 setsunaセミナー
 
20120317 IT系勉強会 in 神戸
20120317 IT系勉強会 in 神戸20120317 IT系勉強会 in 神戸
20120317 IT系勉強会 in 神戸
 
Okuyama説明資料 20120119 ss
Okuyama説明資料 20120119 ssOkuyama説明資料 20120119 ss
Okuyama説明資料 20120119 ss
 
Kvs okuyama-20110818
Kvs okuyama-20110818Kvs okuyama-20110818
Kvs okuyama-20110818
 
20110708 dist_study okuyama
20110708 dist_study okuyama20110708 dist_study okuyama
20110708 dist_study okuyama
 
20110519 okuyama tokyo_linuxstudy
20110519 okuyama tokyo_linuxstudy20110519 okuyama tokyo_linuxstudy
20110519 okuyama tokyo_linuxstudy
 
20110517 okuyama ソーシャルメディアが育てた技術勉強会
20110517 okuyama ソーシャルメディアが育てた技術勉強会20110517 okuyama ソーシャルメディアが育てた技術勉強会
20110517 okuyama ソーシャルメディアが育てた技術勉強会
 
2010/07/09 osc kansai-kvsokuyama
2010/07/09 osc kansai-kvsokuyama2010/07/09 osc kansai-kvsokuyama
2010/07/09 osc kansai-kvsokuyama
 

okuyama_20101101_nosqlafternoon

  • 1. Introduction of Kobe Digital Labo, Inc. Takahiro Iwase Twitter: @okuyamaoo http://d.hatena.ne.jp/okuyamaoo/ Distributed Key-Value Store
  • 2. 1.okuyama Overview 2.okuyama feature list 3.okuyama unique features 4.Information okuyama Outline
  • 3. ・okuyama is Implemented in Java >Started development in December 2009 Now Version 0.8.2 ・Made from scratch >Persistence mechanism, Network mechanism, Replication mechanism okuyama Overview
  • 4. ・Whole composition ・Client → Master Node → Data Node(×3) Master Node Data Node Data Node Data Node Data Node Client Client Master NodeClient okuyama Overview Data Node Data Node Data Node Data NodeData Node Data Node Data NodeData Node
  • 5. okuyama feature list ・Feature List Persistent Type Redundancy Data Maintenance Memory, Disk, Disk + Memory Consists of multiple servers Server can be added non-disruptively All data is stored multiplexed All components can be multiplexed No okuyama signle point of failure If servers are added or server failure occur, data dynamic reconstruction are automatically executed Support Protocol Original, memcached, HTTP Availability
  • 6. okuyama feature list ・Persistece Mechanism ・Memory The fastest ・Disk Persistent and big data Key and Value Management in a separate file ・Disk + Memory The fast and persistent Type1: After recoding file stored in memory Type2: After recoding a file, Key only stored in memory
  • 7. okuyama feature list ・Network Mechanism Accept Queue ConnectClient Accept Queue Worker I/O Read Queue Write Client I/O Read Queue Worker Main Job Queue Client Main Job Queue Worker Respond to
  • 8. okuyama feature list ・Network Mechanism In addition to multiplexing Accept Queue ConnectClient I/O Read Queue Write Client Main Job Queue Client Respond to Main Job Queue Main Job Queue I/O Read Queue I/O Read Queue Accept Queue Accept Queue
  • 9. okuyama unique features ・Feature 1. Tags can be added to the data
  • 10. okuyama unique features ・Feature 1. Key Value Hibari MySQL okuyama “The world‟s most popular open source database” “Hibari is implemented in Erlang” “okuyama is implemented in Java” Tags can be added to the data
  • 11. okuyama unique features ・Feature 1. Key Value Hibari MySQL okuyama “The world‟s most popular open source database” “Hibari is implemented in Erlang” “okuyama is implemented in Java” Key Value Hibari MySQL okuyama “The world‟s most popular open source database” “Hibari is implemented in Erlang” “okuyama is implemented in Java” Tag NOSQL RDBMS NOSQL Tags can be added to the data
  • 12. okuyama unique features ・Feature 1. Tags can be added to the data Key Value Hibari MySQL okuyama “The world‟s most popular open source database” “Hibari is implemented in Erlang” “okuyama is implemented in Java” Tag NOSQL RDBMS NOSQL If you specify a tag …
  • 13. okuyama unique features ・Feature 1. Key Value Hibari MySQL okuyama “The world‟s most popular open source database” “Hibari is implemented in Erlang” “okuyama is implemented in Java” Tag NOSQL RDBMS NOSQL If you specify a tag … {“Hibari”, “okuyama”} You can get the keys at once Tags can be added to the data
  • 14. okuyama unique features ・Feature 2. Sent from the client JavaScript can run in DataNode
  • 15. okuyama unique features ・Feature 2. Sent from the client JavaScript can run in DataNode Main Data Node Client Master Node Request Key=“okuyama” JavaScript=”if (dataValue == „nosql‟) { …..”
  • 16. okuyama unique features ・Feature 2. Sent from the client JavaScript can run in DataNode Client Master Node Main Data Node Key=“okuyama” JavaScript=”if (dataValue == „nosql‟) { …..” Request Key=okuyama Value=“nosql” Execute JavaScript
  • 17. okuyama unique features ・Feature 2. Sent from the client JavaScript can run in DataNode Client Master Node Main Data Node Key=“okuyama” JavaScript=”if (dataValue == „nosql‟) { …..” Request Key=okuyama Value=“nosql” Execute JavaScript The execution result is returned
  • 18. okuyama unique features ・Feature 3. Data lock function
  • 19. okuyama unique features ・Feature 3. Data lock function Key Value Hibari MySQL okuyama “The world‟s most popular open ….. “Hibari is implemented in Erlang” “okuyama is implemented in Java” Client A Lock Success Lock request
  • 20. okuyama unique features ・Feature 3. Data lock function Key Value Hibari MySQL okuyama “The world‟s most popular open ….. “Hibari is implemented in Erlang” “Change Value” Client A Error Set Request “Change Value” Client B Set Request Success