SlideShare una empresa de Scribd logo
1 de 49
FB lee.hyeongchae
About me
국산 DBMSs             MobileLite
   이너비트             Embedded In-Memory DBMS
   NHN ( CUBRID )
   텔코웨어             CUBRID
   알티베이스            Object Oriented DBMS
   티베로
   리얼타임테크           Telcobase
   아키스              In-Memroy DBMS

국산 DBs               Altibase
   선재소프트            In-Memroy DBMS
   유엔젤
Agenda
 In-Memory DBMS !!
 Altibase, TimesTen, ExtremeDB, KDB+, C-ISAM …

 Why ?!
 Ultra & Extreme low latency AND Exture+

 How ?!
 NUMA, SSD, Infiniband, Compiler, MQ, Tick, CEP …

 Best ?!
 In-Memory Computing ?!

 Best Of Best !!
 High Performance Computing !!
In-Memory Architecture
In-Memory Database System

 High Performance
 Low Latency
 No Jitter




                     Disk-Based RDBMS vs Oracle TimesTen

                          ( Simple Architecture )
Modern DBMS

     ≒
In-Memory DBMS
In-Memory Database System



                    4G
              3G
         2G
    1G
Altibase HDB & XDB
장점
   국내개발 및 기술지원
   증권사 레퍼런스
   MVCC 지원


단점
   성능한계 & 메모리 이슈
   고급 엔지니어 부재
Oracle TimesTen & Coherence
장점
 오라클 & 레퍼런스
 성능
 DA ( Direct Attach )


단점
 DA ( Direct Attach ) 한계
 MVCC 미지원
 Durability 이슈
 엔지니어 및 기술지원 부제
McObject ExtremeDB & FE
장점
 성능
 해외 통신 & 증권 레퍼런스
 STAC Member Join


단점
 임베디드 전문 회사
 국내 기술지원
 FE ( Financial Edition ) 검증
KX Systems KDB+
장점
 WORLD BEST
 QL ( Q Language )
 STAC-M3 Leader


단점
 가격
 국내 기술지원
IBM Informix C-ISAM
장점
 OLD BEST
 성능 & 가격
 Zero Configuration & Admin


단점
 File DB
 ACID 미지원
 유지보수 및 기술지원
second

         ㎳
   ㎳~㎲
             ㎲



         2 digit ㎲~㎱
Ultra & Extreme Low Lantecy
Exture+ RFP
Exture+ RFP
Exture+ RFP
Exture vs Exture+
Sync vs Async
Pub-Sub Architecture
Exture+ Dev ?!
Exture+ Dev ?!
Exture+ Dev ?!
Exture+ Dev ?!
NUMA Architecture
NUMA Architecture




 Using McObject’s 64-bit eXtremeDB-64, the application creates a 1.17
 Terabyte, 15.54 billion row database on a 160-core Linux based
 SGI® Altix® 4700 server.
NUMA Architecture




It supports up to 512 * sockets or 1024 cores under one instance of
Linux and as much as 128TB of globally addressable
memory.
NUMA Architecture




 CPU-Socket-Isolation via PhysicalNIC / PhysicalCPU pairing.
 Multiple CPU sockets holding multiple CPUs should be used like multiple
 machines. Avoid inter-CPU communication
NUMA Architecture



           8-socket Nehalem-EX: architecture




     8-socket Nehalem-EX: memory bandwidth matrix
InfiniBand & 10GbE




In computer networking, Server Message Block (SMB), also known as Common
Internet File System (CIFS, /ˈsɪfs/) operates as an application-layer network protocol[1]
mainly used for providing shared access to files, printers, serial ports, and
miscellaneous communications between nodes on a network.
InfiniBand & 10GbE
SSD
SSD is very very difficult.




SSD (DRAM/NVRAM/flash/SLC/MLC) is Blah Blah !!
Compiler & *.[so|a] Library
Message Queue
ZeroMQ
   Crazy fast
   Brokerless architecture
   In-process library
   Lower latencies
   Very simple to use
   No persistence – requiring higher
    layers to manage persistence


RabbitMQ
   VMware vFabric
   AMQP compliant
   Written in erlang
   Small footprint and seemingly fewer
    lines of code in comparison to other
    AMQP compliant queue managers
OpenMAMA is high performance
Middleware Agnostic Messaging API
Tick & Time series database
eXtremeDB Financial Edition meets the specialized requirements of
handling market data with several powerful features:

Flexible data layout. eXtremeDB Financial Edition implements columnar data
layout for fields of type ‘sequence’. Sequences can be combined to form a time series, ideal
for working with tick streams, historical quotes and other sequential data. The technology
supports database designs that combine row-based and column-based layouts, to best
leverage L1/L2 cache speed.




Traditional DBMSs bring rows of data into L1/L2 cache for processing. But financial data –
such as trades and quotes – are better handled by a column-based layout that avoids
flooding the cache with unwanted data.

Vector-based statistical function library. Vector-based statistical
functions provide high efficiency by executing over all or part of one or more sequences and
supporting assembly lines of operations on sequences, for statistical/quantitative analysis




eXtremeDB Financial Edition provides a rich library of vector-basedstatistical functions that
execute over sequence to accelerate management of time series data.

 Handles real-time and historical data. eXtremeDB Financial Edition’s
in-memory storage is ideal for real-time data, while developers can easily specify persistent
tables for historical data with a simple notation in the database schema. eXtremeDB
programming skills are fully interchangeable between in-memory and persistent database
designs (developers needn’t learn two database system products for real-time and historical
data).
Complex Event Processing




우리는 대용량 고속 데이터에 대한 통찰력을 가지기 위한 복합 이벤트 처리
기술(CEP), 인-메모리 기반 분석 기술, 대용량 데이터 베이스 및 저장 기술 등 고급 기술
세트를 제공합니다. 복합 이벤트 분석(Complex Event Analytics) 솔루션은 실시간
대용량 정보 소스(Source)로부터 통찰과 함께 즉시 의사 결정을 지원하는 혁신적인
솔루션입니다.
In-Memory Computing
In-Memory Computing

       !=
No-Disk Computing
TOP500 - 1 st




 Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in
 the PFLOPS (petaFLOPS) range, with low power consumption.
High Performance Computing
HPI ( Hasso Plattner Institut )
Q?!A
FB lee.hyeongchae
hyeongchae@G+

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Amazon Aurora: Under the Hood
Amazon Aurora: Under the HoodAmazon Aurora: Under the Hood
Amazon Aurora: Under the Hood
 
Replacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDBReplacing Your Cache with ScyllaDB
Replacing Your Cache with ScyllaDB
 
Cassandra an overview
Cassandra an overviewCassandra an overview
Cassandra an overview
 
Aurora Deep Dive | AWS Floor28
Aurora Deep Dive | AWS Floor28Aurora Deep Dive | AWS Floor28
Aurora Deep Dive | AWS Floor28
 
Intro to HBase
Intro to HBaseIntro to HBase
Intro to HBase
 
Google Cloud Platform
Google Cloud PlatformGoogle Cloud Platform
Google Cloud Platform
 
Using Apache Arrow, Calcite, and Parquet to Build a Relational Cache
Using Apache Arrow, Calcite, and Parquet to Build a Relational CacheUsing Apache Arrow, Calcite, and Parquet to Build a Relational Cache
Using Apache Arrow, Calcite, and Parquet to Build a Relational Cache
 
Distributed SQL Databases Deconstructed
Distributed SQL Databases DeconstructedDistributed SQL Databases Deconstructed
Distributed SQL Databases Deconstructed
 
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of FacebookTech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
Tech Talk: RocksDB Slides by Dhruba Borthakur & Haobo Xu of Facebook
 
(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive(STG402) Amazon EBS Deep Dive
(STG402) Amazon EBS Deep Dive
 
Query logging with proxysql
Query logging with proxysqlQuery logging with proxysql
Query logging with proxysql
 
Writing Scalable Software in Java
Writing Scalable Software in JavaWriting Scalable Software in Java
Writing Scalable Software in Java
 
Amazon Aurora
Amazon AuroraAmazon Aurora
Amazon Aurora
 
Azure Database Services for MySQL PostgreSQL and MariaDB
Azure Database Services for MySQL PostgreSQL and MariaDBAzure Database Services for MySQL PostgreSQL and MariaDB
Azure Database Services for MySQL PostgreSQL and MariaDB
 
C* Summit 2013: The World's Next Top Data Model by Patrick McFadin
C* Summit 2013: The World's Next Top Data Model by Patrick McFadinC* Summit 2013: The World's Next Top Data Model by Patrick McFadin
C* Summit 2013: The World's Next Top Data Model by Patrick McFadin
 
Top 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark ApplicationsTop 5 Mistakes When Writing Spark Applications
Top 5 Mistakes When Writing Spark Applications
 
Oracle Transparent Data Encryption (TDE) 12c
Oracle Transparent Data Encryption (TDE) 12cOracle Transparent Data Encryption (TDE) 12c
Oracle Transparent Data Encryption (TDE) 12c
 
Disaster Recovery Planning for MySQL & MariaDB
Disaster Recovery Planning for MySQL & MariaDBDisaster Recovery Planning for MySQL & MariaDB
Disaster Recovery Planning for MySQL & MariaDB
 
SQL vs. NoSQL Databases
SQL vs. NoSQL DatabasesSQL vs. NoSQL Databases
SQL vs. NoSQL Databases
 
Always on in sql server 2017
Always on in sql server 2017Always on in sql server 2017
Always on in sql server 2017
 

Destacado

In-memory Database and MySQL Cluster
In-memory Database and MySQL ClusterIn-memory Database and MySQL Cluster
In-memory Database and MySQL Cluster
grandis_au
 
Main MeMory Data Base
Main MeMory Data BaseMain MeMory Data Base
Main MeMory Data Base
Siva Rushi
 

Destacado (20)

In-Memory DataBase
In-Memory DataBaseIn-Memory DataBase
In-Memory DataBase
 
In-Memory Database Platform for Big Data
In-Memory Database Platform for Big DataIn-Memory Database Platform for Big Data
In-Memory Database Platform for Big Data
 
In memory big data management and processing a survey
In memory big data management and processing a surveyIn memory big data management and processing a survey
In memory big data management and processing a survey
 
In-memory Database and MySQL Cluster
In-memory Database and MySQL ClusterIn-memory Database and MySQL Cluster
In-memory Database and MySQL Cluster
 
Transaction management for a main memory database
Transaction management for a main memory databaseTransaction management for a main memory database
Transaction management for a main memory database
 
Which DBMS and Why?
Which DBMS and Why?Which DBMS and Why?
Which DBMS and Why?
 
Using In-Memory Encrypted Databases on the Cloud
Using In-Memory Encrypted Databases on the CloudUsing In-Memory Encrypted Databases on the Cloud
Using In-Memory Encrypted Databases on the Cloud
 
Oracle Database In-Memory and the Query Optimizer
Oracle Database In-Memory and the Query OptimizerOracle Database In-Memory and the Query Optimizer
Oracle Database In-Memory and the Query Optimizer
 
Distributed architecture of oracle database in memory
Distributed architecture of oracle database in memoryDistributed architecture of oracle database in memory
Distributed architecture of oracle database in memory
 
Main MeMory Data Base
Main MeMory Data BaseMain MeMory Data Base
Main MeMory Data Base
 
IN-MEMORY DATABASE SYSTEMS.SAP HANA DATABASE.
IN-MEMORY DATABASE SYSTEMS.SAP HANA DATABASE.IN-MEMORY DATABASE SYSTEMS.SAP HANA DATABASE.
IN-MEMORY DATABASE SYSTEMS.SAP HANA DATABASE.
 
Row or Columnar Database
Row or Columnar DatabaseRow or Columnar Database
Row or Columnar Database
 
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
 
Best Practices for Genomic and Bioinformatics Analysis Pipelines on AWS
Best Practices for Genomic and Bioinformatics Analysis Pipelines on AWS Best Practices for Genomic and Bioinformatics Analysis Pipelines on AWS
Best Practices for Genomic and Bioinformatics Analysis Pipelines on AWS
 
In Memory Computing for Agile Business Intelligence
In Memory Computing for Agile Business IntelligenceIn Memory Computing for Agile Business Intelligence
In Memory Computing for Agile Business Intelligence
 
Sap technical deep dive in a column oriented in memory database
Sap technical deep dive in a column oriented in memory databaseSap technical deep dive in a column oriented in memory database
Sap technical deep dive in a column oriented in memory database
 
In-Memory Computing: How, Why? and common Patterns
In-Memory Computing: How, Why? and common PatternsIn-Memory Computing: How, Why? and common Patterns
In-Memory Computing: How, Why? and common Patterns
 
CTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive AnalyticsCTO View: Driving the On-Demand Economy with Predictive Analytics
CTO View: Driving the On-Demand Economy with Predictive Analytics
 
HUG Ireland Event Presentation - In-Memory Databases
HUG Ireland Event Presentation - In-Memory DatabasesHUG Ireland Event Presentation - In-Memory Databases
HUG Ireland Event Presentation - In-Memory Databases
 
Larry Ellison Introduces Oracle Database In-Memory
Larry Ellison Introduces Oracle Database In-MemoryLarry Ellison Introduces Oracle Database In-Memory
Larry Ellison Introduces Oracle Database In-Memory
 

Similar a in-memory database system and low latency

RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
Redis Labs
 
xTech2006_DB2onRails
xTech2006_DB2onRailsxTech2006_DB2onRails
xTech2006_DB2onRails
webuploader
 
Introduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3RIntroduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3R
Simon Huang
 

Similar a in-memory database system and low latency (20)

Wolfgang Lehner Technische Universitat Dresden
Wolfgang Lehner Technische Universitat DresdenWolfgang Lehner Technische Universitat Dresden
Wolfgang Lehner Technische Universitat Dresden
 
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
RedisConf17 - Building Large High Performance Redis Databases with Redis Ente...
 
Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...
Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...
Running Production CDC Ingestion Pipelines With Balaji Varadarajan and Pritam...
 
Building a High Performance Analytics Platform
Building a High Performance Analytics PlatformBuilding a High Performance Analytics Platform
Building a High Performance Analytics Platform
 
Experience In Building Scalable Web Sites Through Infrastructure's View
Experience In Building Scalable Web Sites Through Infrastructure's ViewExperience In Building Scalable Web Sites Through Infrastructure's View
Experience In Building Scalable Web Sites Through Infrastructure's View
 
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx7 Reasons Not to Put an External Cache in Front of Your Database.pptx
7 Reasons Not to Put an External Cache in Front of Your Database.pptx
 
VirtualStor Extreme - Software Defined Scale-Out All Flash Storage
VirtualStor Extreme - Software Defined Scale-Out All Flash StorageVirtualStor Extreme - Software Defined Scale-Out All Flash Storage
VirtualStor Extreme - Software Defined Scale-Out All Flash Storage
 
Inter connect2016 yss1841-cloud-storage-options-v4
Inter connect2016 yss1841-cloud-storage-options-v4Inter connect2016 yss1841-cloud-storage-options-v4
Inter connect2016 yss1841-cloud-storage-options-v4
 
The Pendulum Swings Back: Converged and Hyperconverged Environments
The Pendulum Swings Back: Converged and Hyperconverged EnvironmentsThe Pendulum Swings Back: Converged and Hyperconverged Environments
The Pendulum Swings Back: Converged and Hyperconverged Environments
 
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
IMCSummit 2015 - Day 1 Developer Track - Evolution of non-volatile memory exp...
 
xTech2006_DB2onRails
xTech2006_DB2onRailsxTech2006_DB2onRails
xTech2006_DB2onRails
 
Introduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3RIntroduction to NVMe Over Fabrics-V3R
Introduction to NVMe Over Fabrics-V3R
 
Ibm lenovo rack server rental
Ibm lenovo rack server rentalIbm lenovo rack server rental
Ibm lenovo rack server rental
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
 
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and ContributionsCeph on Intel: Intel Storage Components, Benchmarks, and Contributions
Ceph on Intel: Intel Storage Components, Benchmarks, and Contributions
 
Using SAS GRID v 9 with Isilon F810
Using SAS GRID v 9 with Isilon F810Using SAS GRID v 9 with Isilon F810
Using SAS GRID v 9 with Isilon F810
 
Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale Ceph: Low Fail Go Scale
Ceph: Low Fail Go Scale
 
Getting Started with Amazon Redshift
Getting Started with Amazon RedshiftGetting Started with Amazon Redshift
Getting Started with Amazon Redshift
 
EMC Symmetrix VMAX: An Introduction to Enterprise Storage: Brian Boyd, Varrow...
EMC Symmetrix VMAX: An Introduction to Enterprise Storage: Brian Boyd, Varrow...EMC Symmetrix VMAX: An Introduction to Enterprise Storage: Brian Boyd, Varrow...
EMC Symmetrix VMAX: An Introduction to Enterprise Storage: Brian Boyd, Varrow...
 
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
44spotkaniePLSSUGWRO_CoNowegowKrainieChmur
 

Más de hyeongchae lee

patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deployment
hyeongchae lee
 
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s VaultSecuring Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
hyeongchae lee
 

Más de hyeongchae lee (12)

patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deployment
 
[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Features[PGDay.Seoul 2020] PostgreSQL 13 New Features
[PGDay.Seoul 2020] PostgreSQL 13 New Features
 
[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppa[HashiTalk Korea] OCP with Super Tengen Toppa
[HashiTalk Korea] OCP with Super Tengen Toppa
 
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s VaultSecuring Databases with Dynamic Credentials and HashiCorp’s Vault
Securing Databases with Dynamic Credentials and HashiCorp’s Vault
 
OCP with super tengen toppa
OCP with super tengen toppaOCP with super tengen toppa
OCP with super tengen toppa
 
PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회PostgreSQL 정기 기술 세미나 22회
PostgreSQL 정기 기술 세미나 22회
 
PGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalkPGDay.Seoul 2016 lightingtalk
PGDay.Seoul 2016 lightingtalk
 
20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_db20141206 4 q14_dataconference_i_am_your_db
20141206 4 q14_dataconference_i_am_your_db
 
osscon_mysql_redis_plugin
osscon_mysql_redis_pluginosscon_mysql_redis_plugin
osscon_mysql_redis_plugin
 
Oracle2DBMS Notes and Comments
Oracle2DBMS Notes and CommentsOracle2DBMS Notes and Comments
Oracle2DBMS Notes and Comments
 
NewSQL
NewSQLNewSQL
NewSQL
 
eXtremeDB FE
eXtremeDB FEeXtremeDB FE
eXtremeDB FE
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Último (20)

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 

in-memory database system and low latency

  • 2. About me 국산 DBMSs MobileLite  이너비트 Embedded In-Memory DBMS  NHN ( CUBRID )  텔코웨어 CUBRID  알티베이스 Object Oriented DBMS  티베로  리얼타임테크 Telcobase  아키스 In-Memroy DBMS 국산 DBs Altibase  선재소프트 In-Memroy DBMS  유엔젤
  • 3. Agenda In-Memory DBMS !! Altibase, TimesTen, ExtremeDB, KDB+, C-ISAM … Why ?! Ultra & Extreme low latency AND Exture+ How ?! NUMA, SSD, Infiniband, Compiler, MQ, Tick, CEP … Best ?! In-Memory Computing ?! Best Of Best !! High Performance Computing !!
  • 4.
  • 6. In-Memory Database System  High Performance  Low Latency  No Jitter Disk-Based RDBMS vs Oracle TimesTen ( Simple Architecture )
  • 7. Modern DBMS ≒ In-Memory DBMS
  • 9. Altibase HDB & XDB 장점  국내개발 및 기술지원  증권사 레퍼런스  MVCC 지원 단점  성능한계 & 메모리 이슈  고급 엔지니어 부재
  • 10. Oracle TimesTen & Coherence 장점  오라클 & 레퍼런스  성능  DA ( Direct Attach ) 단점  DA ( Direct Attach ) 한계  MVCC 미지원  Durability 이슈  엔지니어 및 기술지원 부제
  • 11. McObject ExtremeDB & FE 장점  성능  해외 통신 & 증권 레퍼런스  STAC Member Join 단점  임베디드 전문 회사  국내 기술지원  FE ( Financial Edition ) 검증
  • 12. KX Systems KDB+ 장점  WORLD BEST  QL ( Q Language )  STAC-M3 Leader 단점  가격  국내 기술지원
  • 13. IBM Informix C-ISAM 장점  OLD BEST  성능 & 가격  Zero Configuration & Admin 단점  File DB  ACID 미지원  유지보수 및 기술지원
  • 14.
  • 15. second ㎳ ㎳~㎲ ㎲ 2 digit ㎲~㎱
  • 16. Ultra & Extreme Low Lantecy
  • 27.
  • 29. NUMA Architecture Using McObject’s 64-bit eXtremeDB-64, the application creates a 1.17 Terabyte, 15.54 billion row database on a 160-core Linux based SGI® Altix® 4700 server.
  • 30. NUMA Architecture It supports up to 512 * sockets or 1024 cores under one instance of Linux and as much as 128TB of globally addressable memory.
  • 31. NUMA Architecture CPU-Socket-Isolation via PhysicalNIC / PhysicalCPU pairing. Multiple CPU sockets holding multiple CPUs should be used like multiple machines. Avoid inter-CPU communication
  • 32. NUMA Architecture 8-socket Nehalem-EX: architecture 8-socket Nehalem-EX: memory bandwidth matrix
  • 33. InfiniBand & 10GbE In computer networking, Server Message Block (SMB), also known as Common Internet File System (CIFS, /ˈsɪfs/) operates as an application-layer network protocol[1] mainly used for providing shared access to files, printers, serial ports, and miscellaneous communications between nodes on a network.
  • 35. SSD
  • 36. SSD is very very difficult. SSD (DRAM/NVRAM/flash/SLC/MLC) is Blah Blah !!
  • 38. Message Queue ZeroMQ  Crazy fast  Brokerless architecture  In-process library  Lower latencies  Very simple to use  No persistence – requiring higher layers to manage persistence RabbitMQ  VMware vFabric  AMQP compliant  Written in erlang  Small footprint and seemingly fewer lines of code in comparison to other AMQP compliant queue managers
  • 39. OpenMAMA is high performance Middleware Agnostic Messaging API
  • 40. Tick & Time series database eXtremeDB Financial Edition meets the specialized requirements of handling market data with several powerful features: Flexible data layout. eXtremeDB Financial Edition implements columnar data layout for fields of type ‘sequence’. Sequences can be combined to form a time series, ideal for working with tick streams, historical quotes and other sequential data. The technology supports database designs that combine row-based and column-based layouts, to best leverage L1/L2 cache speed. Traditional DBMSs bring rows of data into L1/L2 cache for processing. But financial data – such as trades and quotes – are better handled by a column-based layout that avoids flooding the cache with unwanted data. Vector-based statistical function library. Vector-based statistical functions provide high efficiency by executing over all or part of one or more sequences and supporting assembly lines of operations on sequences, for statistical/quantitative analysis eXtremeDB Financial Edition provides a rich library of vector-basedstatistical functions that execute over sequence to accelerate management of time series data. Handles real-time and historical data. eXtremeDB Financial Edition’s in-memory storage is ideal for real-time data, while developers can easily specify persistent tables for historical data with a simple notation in the database schema. eXtremeDB programming skills are fully interchangeable between in-memory and persistent database designs (developers needn’t learn two database system products for real-time and historical data).
  • 41. Complex Event Processing 우리는 대용량 고속 데이터에 대한 통찰력을 가지기 위한 복합 이벤트 처리 기술(CEP), 인-메모리 기반 분석 기술, 대용량 데이터 베이스 및 저장 기술 등 고급 기술 세트를 제공합니다. 복합 이벤트 분석(Complex Event Analytics) 솔루션은 실시간 대용량 정보 소스(Source)로부터 통찰과 함께 즉시 의사 결정을 지원하는 혁신적인 솔루션입니다.
  • 42.
  • 44. In-Memory Computing != No-Disk Computing
  • 45.
  • 46. TOP500 - 1 st Blue Gene is an IBM project aimed at designing supercomputers that can reach operating speeds in the PFLOPS (petaFLOPS) range, with low power consumption.
  • 48. HPI ( Hasso Plattner Institut )