IBM Informix is presented as the ideal database solution for IoT architectures due to its small footprint, low memory requirements, support for time series and spatial data, and driverless operation requiring no administration. It can run on gateways to filter and analyze sensor data locally before transmitting to the cloud. In the cloud, Informix can ingest streaming data in real-time, perform operational analytics, and scale out across servers. Benchmarks show Informix outperforming SQLite for IoT workloads in areas like data loading speed, storage requirements, and analytic query speeds.
4. 4
Internet of Things Topography
Streams
Deep Analytics Zone
Device/Sensors
Smart Gateways
Sensor Analytics Zone
4
5. Simplified IoT Data Flow
Sensor Data History
Sensors
In-memory Analytics
Predictive
Analytics
Publish /
Subscribe
Cloud
Infrastructure
Real-time Analytics
Real-time Analytics Operational Analytics
Big Data
Analytics
(no gateway)
(Gateways)
HDFS / Hadoop
Big Data
Analytics
MessageSight
/ MQTT
Gateways for
local analytics
InfoSphere
Streams
Informix / Cloudant /
IOT Foundation
Service / TimeSeries
Service
Informix Warehouse
Accelerator / DashDB
PMQ / SPSS /
Cognos
Softlayer / Bluemix
Watson / DashDB /
BigInsights
Informix /
Node-Red
6. • Gateways can reduce the cost of the backend cloud
• Reduces cloud storage by filtering/aggregating/analyzing data locally
• Reduces cloud CPU requirements by precomputing values
• Reduces latency since actions can be taken immediately
• Intelligent gateways can detect and respond to local events as they
happen rather than waiting for transfer to the cloud
• Some users are not comfortable putting all their data in the cloud
• Gateways allow customers to capture and get value from their sensors
without sending data to the cloud
• Protocol Consolidation
• Cloud does not need to implement the 100’s of IoT protocols
Over time more and more of the processing will move from the cloud
to gateway devices
How Do Gateways Help IoT Solutions?
6
7. What are the Requirements for a Gateway Database?
• The database management system must:
Have a small install footprint, less than 100 MB
Run with low memory requirements – less than 256 MB
Use lossless compression or other techniques to minimize
storage space
Have built-in support for common types of IoT data like time series
and spatial/GIS data
Simple application development supporting both NoSQL and SQL
Driverless, easy access to the data
Require absolutely no administration
Ideally should be able to network multiple gateways together to
create a single distributed database
7
The database must be powerful enough to ingest, process and
analyze data in real-time
8. IBM Informix: The Ideal Database for Gateways
Simple to use
Hands-Free operation – No administration
Supports popular interfaces such as REST and Mongo as well
as ODBC/JDBC
Handles SQL and JSON data in the same database
Performance
One of a kind support for TimeSeries and Spatial data
Stream data continuously into the database
Run analytics as data arrives
Dynamically add and update analytics when needed
Storage is typically 1/3 the size compared to other vendors
Invisible
Agile
8
Informix is the only database management system
perfectly suited to run in Gateways
9. Sensor Data is TimeSeries Data
• What is a Time Series?
A logically connected set of records ordered by time
• What are the Key Strengths of Informix TimeSeries?
Much less space required
• Typically about 1/3 the space required by other vendors
Queries run orders of magnitude faster
• Unique optimized storage means codes paths are shorter and more
data fits in memory
Purpose built streaming data loader for sensor data
• Automatically run analytic and/or aggregate functions on new data
Can store structured (SQL) or unstructured (JSON) data for
quick application development
• REST/ODBC/JDBC/JSON interfaces available
100’s of functions predefined
• Programming APIs available to create your own analytics
9
10. Traditional Table Approach
Informix TimeSeries Approach
Meter_ID Time KWH Voltage ColN
1 1-1-11 12:00 Value 1 Value 2 ……… Value N
2 1-1-11 12:00 Value 1 Value 2 ……… Value N
3 1-1-11 12:00 Value 1 Value 2 ……… Value N
… … … … ……… …
1 1-1-11 12:15 Value 1 Value 2 ……… Value N
2 1-1-11 12:15 Value 1 Value 2 ……… Value N
3 1-1-11 12:15 Value 1 Value 2 ……… Value N
… … … … ……… …
Meter_ID Series
1 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
2 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
3 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
4 [(1-1-11 12:00, value 1, value 2,…, value N), (1-1-11 12:15, value 1, value 2, …, value N), …]
…
Traditional Sensor data storage vs Informix TimeSeries
Storage
10
11. IoT Requirements for SpatioTemporal Data
• Many IoT applications have a spatial
component to them
Vehicles, cell phones, even pets…
tracking is common
• In these cases both location and
time is important
Show me the vehicles that have
passed by location X in the last
hour
Where has my car been over the
last few hours?
• Informix allows you to combine Time
series and Spatial data in the same
query
11
12. 12
Both Structured and Unstructured Data is
Common in IoT Environments
JSON
Collection
SQL Driver
NoSQL Driver
SQL Data
Join Data
• Informix can store SQL and JSON data in the same database
• Write programs using SQL drivers or Mongo/NoSQL drivers
• SQL data automatically transformed into JSON documents when needed
• NoSQL data automatically transformed into SQL when needed
Embedded Device
or Database
server
Horizontal
Scale-out
with Shards
13. Informix Data Access Options
13
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• REST, MongoDB Protocol
Support
• SQLI, DRDA Protocol Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
A REST client is any
program capable of
making a HTTP request
14. Informix Data Access Options
14
MongoDB
Client
REST Client
SQLI Client
DRDA Client
Informix
DBMS
Informix NoSQL
Listener
Informix
• NoSQL ↔ SQL Translation
• REST, MongoDB Protocol
Support
• SQLI, DRDA Protocol
Support
• Relational, Collection, Time
Series, and Spatial Data
Support
Spatial
Time Series
JSON Collection
Relational Table
You can use all the client
drivers that are available
for MongoDB with the
NoSQL Listener
15. • Rapid Development
• Simple use with JSON
• Simple REST
• Simple MQTT and other adapters
• Simple visualization of TimeSeries data
• Contributor Nodes – simple to use other services
15
Use Node-Red for Quick Gateway App. Development
17. IBM IoT Smart Gateway Kit
1. Login as installing user
▪ Ex: pi
2. mkdir iot-gateway-kit-depend
3. Run git clone https://github.com/ibm-iot/iot-gateway-kit.git
4. cd iot-gateway-kit
5. Run ./iot_install
17
18. IBM IoT Smart Gateway Kit
• The iot-gateway-kit will install the following:
▪ Node.js
▪ Node-red
▪ TimeSeries nodes
▪ Bluetooth node.js application sample
18
19. Smart Gateway Kit – TI Sensor Tag
19
1. IoT gateway Kit
Designed using node-
red/node.js to work with
the TI Sensor Tag
2. Stores data in the
Timeseries database
3. Aggregate data and
push to the cloud
4. IoT Foundation or other.
23. 23
What are the IoT Requirements for the Cloud?
• Requirements - similar to gateways (but for different reasons):
• Potentially 1000’s of servers means zero administration is a must
• Data volume adds up very quickly so low storage overhead is required
• Data flows into the cloud continuously and must be processed in real-time
• Must be able to handle time series, spatial, and NoSQL data natively
• Additional requirements
• Must be able to scale-out
• Must be available as a service
The database must be able to ingest, process and analyze
data in real-time
24. 24
Why use Informix in the “Operational Zone”?
Simple to use
• Hands-Free operation
• Supports REST and Mongo APIs as well as ODBC/JDBC
• Stores SQL and JSON database in the same database
Highly Available
• Close to zero down time
• Partition or Hash your data across servers in the cloud
• Dynamically add/remove additional servers
Performance
• Continuous High Performance Analytics
• Specialized support for Time Series and Spatial data
Invisible
Agile
Resilient
25. 25
Shards: Scale-out your Database across Servers or
Gateways
• Distribute data among servers by
range or hash partitioning
• Each shard can have an associated
secondary server for high
availability
• Run queries across all shards or
a subset of the shards
• Only shards that could qualify are
searched
• Shards are searched in parallel
• Ignores shards that are offline
Shards in a
Cloud
26. 26
IoT Analytics - Operational and
Big Data Analytics
• Operational Analytics
• Needed when results are required in (near) real-time
• Real-time monitoring, situational detection, correlation of events, e.g.
• Dynamic advertising based on customer movement
• Real-time equipment failure prediction
• Operational analytics are required in gateways as well as the cloud
• Gateways need to aggregate, filter, monitor for conditions
• Analytics must run efficiently while new data is being loaded
• Must be able to dynamically add and update analytics in the cloud
and gateways
• Big Data analytics are required when you have the time to do
“Deeper/Wider” analysis
• Latency between data arrival and results not an issue
• Efficient ETL process from the operational repository is a must
27. 27
TCP/IP
Bulk Loader
SQL Queries (from apps)
Informix Warehouse Accelerator
Compressed
DB partition
Query
Processor
Data Warehouse
Informix SQL
Query Router Results
Informix Warehouse Accelerator:
• Connects to Informix via TCP/IP & DRDA
• Analyzes, compresses, and loads to memory
• Copy of (portion of) warehouse
• Processes routed SQL query and
• returns answer to Informix
Use Informix Warehouse Accelerator for
Mixed Operational/Analytic Workloads
Informix:
• Routes SQL queries to accelerator
• User need not change SQL or apps.
• Can always run query in Informix
• Too short an est. execution time
28. 28
Every IoT deployment will need to store time series data and possibly spatial data
Bluemix Cloud Services
IoT Foundation Service on Bluemix
• Quickly and simply add new sensors
• Interface for continuously loading sensor data
• Adapters for MQTT and MessageSight
30. IBM Informix and Intel Deliver Leading Edge Solutions for IoT
Informix support for Intel’s
new Quark processor, now
supporting Intel family, from
Quark to Xeon.
Quark port in the free
Informix Developer Edition
Informix Developer Edition
available as part of standard
Intel Gateway developer SDK
Combining IBM and Intel’s strength at the Edge, Gateway and in the Cloud provides
an intelligent, e2e IoT solution for rapid time to market.
Intel® Quark™ SoC / Intel®
Atom™ / Intel® Xeon™
Windriver
McAfee Security
Supports OSGI/TR69
Informix Database
30
31. Benchmark: Informix vs SQLite
31
Tests on Intel Quark Informix SQLite
Data loading – high-speed performance
(records per second)
950 / 1050
secs
(DK100 / DK50)1
700 secs
(Average)2
Storage space that is required for 1 day of data 275 MB 1200 MB
Aggregation query (seconds) 2 secs 4-25 secs
Moving average (seconds) 25 secs 2592003 secs
Missing interval search (seconds) 2 secs 14-30 secs
1. The two figures for data loading with Informix reflect a slight difference in performance between the
DK100 and DK50. DK100 had more running components causing a drop in performance vs DK50
2. Data loading with SQLite had significant variations in load performance as the database size
increased.
3. The moving average result for SQLite is a projected figure that is based on a partial result after 10
minutes.
32. 32
Metric Competitor Informix
Daily processing time
Maximum number of cores used
11 hours
62
5 hour 50 min
32
Maximum amount of memory used 192GB 192GB
Size of database per month of data 15TB 5TB
# Records processed each day 2.88 Billion 2.88 Billion
Billing determinants creation (1/21 of
the total meter population)
51,322 ~2 million reads
per second
TimeSeries Meter Data Management Benchmark
- 30 million smart meters sending data every 15 minutes
- 2.88 billion records inserted each day
- Workload: data Ingestion, data cleanup, and a daily billing cycle
33. Summary
• IBM Informix - best fit for IoT architecture
• IoT gateway
• IoT cloud analytics
• Supported on a wide array of platforms
• Best in class embeddability
• Native support for sensor data - TimeSeries & Spatial data
• Native support for unstructured (JSON) data
• Ease of application development - REST access
• Support to receive IoT data via MQTT protocol
• High availability and dynamic scaling
33
Reword to be about the requirements in general for a database in a gateway
Informix the ideal database for embedded devices
Traditional:
Each row contains one record = billions of rows in the table
All data is indexed for efficient lookups
Data is appended to the end of the table as it arrives
Meter ID’s stored in every record
No concept of a missing row
Informix:
Each row contains all the data for a single meter, data append to end of the row
Data is not indexed, only the meter ID column is indexed
Data on disk is clustered by meter id and kept ordered by time
Meter IDs stored once rather than with every record
Timestamps are not stored on disk, instead are calculated by position in series
Missing intervals are marked with a placeholder
Timestamps down to 10 microseconds supported
Currently Tested on the Pi B+ and the Pi 2. Testing on intel x86 platforms as well.
Move before slide 20, talk about requirements in the operational zone
IBM is delivering ground-breaking solutions and announcing key partnerships for gateway processing and analytics that solve these challenges.
Secure edge-to-cloud solutions to accelerate the Internet of Things
Addresses key industry barriers around interoperability, security and scalability
Interest and excitement for IoT continues, but developers and businesses face challenges in security, interoperability, and deployment complexity
IBM and Intel are working to create IoT solutions that are flexible and help drive adoption.
These partner solutions include essential Intel and IBM IOT building blocks, such as gateways, security, and data management
Together, we are providing interoperable platforms that deliver security, connectivity, manageability and analytics, enabling developers to create innovative IoT applications.
Continued Industry alignment and standards are critical. We will be working through the IIC to drive open standards and industry alignment
IBM and Intel are collaborating to deliver leading edge interoperable solutions for Internet of Things that enable the developers with GTM flexibility and ease of use. This will allow them to deliver best in-class implementations from innovation through commercialization phase
IBM Bluemix, IBM Internet of Things Foundation run optimized on Intel® Galileo and Intel® Gateway Solutions Development kits
By combining IBM’s strength in the Cloud with Intel’s strength at the Edge we are able to provide the Developers with an e2e IOT solution that is optimized for e2e security and rapid time to market.
Intel and IBM will collaborate to take these solutions to market in vertical segments such as Industrial, Retail and Fleet management
Informix provides gateways with an intelligent and highly reliable database, optimized for spatial and time series data.
Same database on the cloud and gateway, simplifying development and replicating data from gateways to the cloud and back.
Developers can develop Internet of Things apps with IBM Internet of Things Foundation and Intel’s Development kits.
The fun thing about this relationship is that it allows developers using the Intel Gateways to very quickly connect a sensor and instrument their IoT use cases without having to build a large infrastructure to support it. It also gives the developer an opportunity to gain economies of scale by using the Bluemix analytics and workflow tools to make data driven decisions on the data from the gateway.
Intel’s gateways are perfectly suited for this type of activity because they come with a pre-validated set of building blocks and a rich development environment.
IBM and Intel are collaborating to deliver leading edge interoperable solutions for Internet of Things that enable the developers with GTM flexibility and ease of use. This will allow them to deliver best in-class implementations from innovation through commercialization phase
Partners wanting to start using Informix on the Cloud can do so for free with the Bluemix Time Series Database Service.
And those partners who want to start using Informix on Gateways can get the Quark port by in the Informix Developers Edition, which again is Free. We are very excited to announce that Intel and IBM have agree to make the Informix Developers Edition available as part of the standard Intel Gateway developer SDK.