SlideShare a Scribd company logo
1 of 6
Download to read offline
Confidential
Warehousing the Data in IoT Systems
In an IoT application warehousing the data can be really challenging. Before drawing 
insights  from  data  one  needs  to  warehouse  it  in  such  a  way  that  it  can  be  easily 
retrieved for analysis. 
The multitude of available options can cause a lot of confusion. Broadly speaking, 
there are two alternatives for a database system to store the sensor data, traditional 
RDBMS systems or NoSQL. 
But  to  warehouse  periodic  sensor  data,  NoSQL’s  non­relational,  document­based 
structure is the ideal recourse. In my use case, a REST web service was exposed to a 
GPRS  module  in  the  hardware  to  send  the  sensor  data  to  our  NoSQL  database 
periodically. 
What is linear regression?
Linear  regression  is  a  supervised  learning  technique  in  which  the  variation  in  a 
continuous  dependent  variable  is  expressed  in  terms  of  coefficients  of  various 
independent  variables.  IoT devices  These  coefficients    vary  proportional  to  the 
impact, that the changes in a specific independent variable have on the value 
of the dependent variable. 
Simply put, there is a value on one side that depends on a combination of factors, 
and you need to find the line that best explains the relationship between that value 
and the factors impacting it. 
An  example  of  this  is  electricity  units  consumed  at  a  particular  house  compared 
with  the  number  of  appliances  and  the  duration  of  time.  These  two  factors 
definitely  have  an  impact  on  the  units  consumed,  with  the  consumed  units 
increasing with the number of appliances and the time duration 
while they are active. 
Spring Batch to Extract, Transform, and Load Data
Once the algorithm for the data model has been developed, the next
thing is to find a way to load the NoSQL data into R. Though there are
packages such as r Mongo for this purpose, they are not optimal and do
not support multithreading under the hood. So Spring Batch is a great
option for performing this kind of operation.
What is Spring Batch?
Spring Batch is a flavor of batch processing offered by Java’s Spring
framework. Spring Batch provides the capability to read, process, and
write data from and to multiple data sources. So it is similar to the
extract, transform, and load capabilities of BI (Business Intelligence)
platforms such as SAS.
A Spring Batch application usually includes one or more steps. These are the main 
components in 
each step of a typical Spring Batch system:
• Reader: 
A reader, as the name implies, reads chunks of data from a data source. This data source 
could be a text file, a CSV file, relational database, or a NoSQL database. The reader can be 
multithreaded with a thread pool executor managing the number of active threads. In this 
case however the sequence of the data chunks read is random. Essentially this is the extract 
step when compared to a traditional ETL job.
•Processor: 
The processor is the transform step in which the read data is processed into a the form 
required for the writer. This is the part where most of the business logic is applied to 
iot cloud the data. Once the data has been converted to the desired form, it is passed on to 
the writer and is 
ready to be written to a database or any other data source.
• Writer: 
This is the load step into globallogic. The processed data is output to a relational database or 
a NoSQL database. The writer can also be multithreaded by using a task executor. However, 
when multithreading is used, the order of writing is arbitrary and not sequential.
• Listener: 
The listener is an ad­hoc step in the process and may involve sending out emails to 
stakeholders, pushing notifications to devices, etc. It listens to time frames in the job’s life 
cycle such as before the job begins or after it ends. It allows the developer to do some 
additional tasks 
at these junctures
Confidential
Thank you

More Related Content

Viewers also liked (12)

Riesgos fìsicos
Riesgos fìsicosRiesgos fìsicos
Riesgos fìsicos
 
4º medio-guía-función-cuadrática
4º medio-guía-función-cuadrática4º medio-guía-función-cuadrática
4º medio-guía-función-cuadrática
 
CATÁLOGO BOULEVARD MONDE
CATÁLOGO BOULEVARD MONDECATÁLOGO BOULEVARD MONDE
CATÁLOGO BOULEVARD MONDE
 
CURRICULUM VITAE_
CURRICULUM VITAE_CURRICULUM VITAE_
CURRICULUM VITAE_
 
costello fish
costello fishcostello fish
costello fish
 
Integración de organismos electorales a nivel subnacional: Redes de poder y c...
Integración de organismos electorales a nivel subnacional: Redes de poder y c...Integración de organismos electorales a nivel subnacional: Redes de poder y c...
Integración de organismos electorales a nivel subnacional: Redes de poder y c...
 
Cartilha cfess final_grafica
Cartilha cfess final_graficaCartilha cfess final_grafica
Cartilha cfess final_grafica
 
презентация лекция 3 (1)
презентация лекция 3 (1)презентация лекция 3 (1)
презентация лекция 3 (1)
 
Updated Final Thesis -Post VIVA VOCE
Updated Final Thesis -Post VIVA VOCEUpdated Final Thesis -Post VIVA VOCE
Updated Final Thesis -Post VIVA VOCE
 
El computador carlos
El computador carlosEl computador carlos
El computador carlos
 
CMSC 350 PROJECT 3
CMSC 350 PROJECT 3CMSC 350 PROJECT 3
CMSC 350 PROJECT 3
 
El atletismo
El atletismoEl atletismo
El atletismo
 

Recently uploaded

Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
Joaquim Jorge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
vu2urc
 

Recently uploaded (20)

How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...Workshop - Best of Both Worlds_ Combine  KG and Vector search for  enhanced R...
Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Artificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and MythsArtificial Intelligence: Facts and Myths
Artificial Intelligence: Facts and Myths
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 

IoT technology companies in