SlideShare una empresa de Scribd logo
1 de 16
Smart data
for smart
meters
Wouter Beek
w.g.j.beek@vu.nl
www.wouterbeek.com
Context
•
•
•
•

Energy label / Energy Performance of Buildings Directive (EPBD)
Possible values: A - G
Measurements are valid for 10 years.
Requirement when buying or renting a house.
EnergyLabels dataset in numbers
• 2,354,560 entries
• Energy index
• Electricity consumption
• Gas consumption

• License: Creative Commons 0
• Dissemination date: 2012-11-05
• Updated on a daily basis

• Issued by: Energielabels Agentschap NL
• Related dataset?: Liander Open Data, approx. 1,250,000 entries.
Linked Open Data
•
•
•
•
•

Connect to existing datasets.
Connect to services.
Run queries across datasets.
Perform inference across datasets.
Easy to create mash-ups / new applications.

cheap to do all of this,
only then will Linked Data be an enabler for large-scale innovation.
If it is

(disclaimer: this is a subjective claim)
RDF files

Domain-independent data conversions
fully automated

Relational DB

Domain-dependent data conversions
domain knowledge needed

domain knowledge

Simple RDF
Link to external sources (linksets)
domain knowledge needed

XML files
depends on structure
domain knowledge

Fixing bad data
origin inconsistencies
& inaccuracies

Text files
ambiguous

Connect to services
(e.g. query interface, maps)
high level of reuse
Technological contribution
• From 3-star (published, open format) to 5-star (Linked Data, URI
identifiers, linked to BAG).
• Stored in 2.6 GB XML document containing one (1!) line :-)
• DOM is too big to hold in RAM.

• Convert to multi-line XML document.
• XML2RDF conversion infrastructure:
• Create a resource using primary/rigid properties.
• Create triples for a resource
Application based on 5-star dataset
Using Linked Data (Wouter’s Inbox)
Dear Wouter,
we gave the students of our Semantic Web class the link to the
Kadaster information, and made them enthusiastic to use it. As a result
several now have build their apps around this data. But now it has been
offline for several days.
Cheers,
Stefan.
Main difficulties (1/3)
Technical difficulties due to arbitrary data formatting.
• Publishing data in a sane way decreases the conversion costs
considerably.
• In this use case: half of all the effort went into the 1 line XML...
Main difficulties (2/3)
Institutional difficulties:
• Data publication is a short-duration visible event.
• Data maintenance is a long-duration invisible event.
“You can fool all the people some of the time, and some of the people
all the time, but you cannot fool all the people all the time.”
Abraham Lincoln

Let's make some substitutions here...

“All LOD datasets are offline some of the time, and some of the LOD
datasets are offline all of the time, but not all LOD datasets are offline
all of the time.”
Wouter Beek
Main difficulties (3/3)
Infrastructural difficulties:
• Assuming that some LOD data is online some of the time, we must
explicitly represent the network of interconnected LOD
datasets, institutions, and maintainers (DC, FOAF, VoID).
• Anticipating malfunctioning datasets should be a standard part of the
development API.
Conclusion
Only when the technical, institutional, and infrastructural
problems are solved will Linked Data become an enabler for large-scale
innovation.

Más contenido relacionado

Destacado

Intelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn ApproachIntelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn Approach
Wouter Beek
 
Introduction to AI - First Lecture
Introduction to AI - First LectureIntroduction to AI - First Lecture
Introduction to AI - First Lecture
Wouter Beek
 
DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9
Wouter Beek
 
Introduction to AI - Second Lecture
Introduction to AI - Second LectureIntroduction to AI - Second Lecture
Introduction to AI - Second Lecture
Wouter Beek
 
Introduction to AI - Third Lecture
Introduction to AI - Third LectureIntroduction to AI - Third Lecture
Introduction to AI - Third Lecture
Wouter Beek
 
Introduction to AI - Fifth Lecture
Introduction to AI - Fifth LectureIntroduction to AI - Fifth Lecture
Introduction to AI - Fifth Lecture
Wouter Beek
 
Introduction to AI - Fourth Lecture
Introduction to AI - Fourth LectureIntroduction to AI - Fourth Lecture
Introduction to AI - Fourth Lecture
Wouter Beek
 
Introduction to AI - Seventh Lecture
Introduction to AI - Seventh LectureIntroduction to AI - Seventh Lecture
Introduction to AI - Seventh Lecture
Wouter Beek
 
Filosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentieFilosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentie
Wouter Beek
 
Introduction to AI - Ninth Lecture
Introduction to AI - Ninth LectureIntroduction to AI - Ninth Lecture
Introduction to AI - Ninth Lecture
Wouter Beek
 
Proefstuderen 2011
Proefstuderen 2011Proefstuderen 2011
Proefstuderen 2011
Wouter Beek
 
Machines en procedures in de literatuur
Machines en procedures in de literatuurMachines en procedures in de literatuur
Machines en procedures in de literatuur
Wouter Beek
 
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Wouter Beek
 
Introduction to AI - Eight Lecture
Introduction to AI - Eight LectureIntroduction to AI - Eight Lecture
Introduction to AI - Eight Lecture
Wouter Beek
 
Introduction to AI - Sixth Lecture
Introduction to AI - Sixth LectureIntroduction to AI - Sixth Lecture
Introduction to AI - Sixth Lecture
Wouter Beek
 

Destacado (20)

Intelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn ApproachIntelligent Tutoring Systems: The DynaLearn Approach
Intelligent Tutoring Systems: The DynaLearn Approach
 
Rough Set Semantics for Identity Management on the Web
Rough Set Semantics for Identity Management on the WebRough Set Semantics for Identity Management on the Web
Rough Set Semantics for Identity Management on the Web
 
Introduction to AI - First Lecture
Introduction to AI - First LectureIntroduction to AI - First Lecture
Introduction to AI - First Lecture
 
Pragmatic Semantics for the Web of Data
Pragmatic Semantics for the Web of DataPragmatic Semantics for the Web of Data
Pragmatic Semantics for the Web of Data
 
DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9DynaLearn@JTEL2010_2010_6_9
DynaLearn@JTEL2010_2010_6_9
 
Introduction to AI - Second Lecture
Introduction to AI - Second LectureIntroduction to AI - Second Lecture
Introduction to AI - Second Lecture
 
Introduction to AI - Third Lecture
Introduction to AI - Third LectureIntroduction to AI - Third Lecture
Introduction to AI - Third Lecture
 
Introduction to AI - Fifth Lecture
Introduction to AI - Fifth LectureIntroduction to AI - Fifth Lecture
Introduction to AI - Fifth Lecture
 
Introduction to AI - Fourth Lecture
Introduction to AI - Fourth LectureIntroduction to AI - Fourth Lecture
Introduction to AI - Fourth Lecture
 
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
Dutch Book Trade 1660-1750: using the STCN to gain insight in publishers’ str...
 
Introduction to AI - Seventh Lecture
Introduction to AI - Seventh LectureIntroduction to AI - Seventh Lecture
Introduction to AI - Seventh Lecture
 
Filosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentieFilosofie en kunstmatige intelligentie
Filosofie en kunstmatige intelligentie
 
Introduction to AI - Ninth Lecture
Introduction to AI - Ninth LectureIntroduction to AI - Ninth Lecture
Introduction to AI - Ninth Lecture
 
Proefstuderen 2011
Proefstuderen 2011Proefstuderen 2011
Proefstuderen 2011
 
Machines en procedures in de literatuur
Machines en procedures in de literatuurMachines en procedures in de literatuur
Machines en procedures in de literatuur
 
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
Procedurele Poëzie (Cafe Scientifique, 28 maart 2011)
 
Introduction to AI - Eight Lecture
Introduction to AI - Eight LectureIntroduction to AI - Eight Lecture
Introduction to AI - Eight Lecture
 
Introduction to AI - Sixth Lecture
Introduction to AI - Sixth LectureIntroduction to AI - Sixth Lecture
Introduction to AI - Sixth Lecture
 
인공지능 기술과 서비스의 이해
인공지능 기술과 서비스의 이해 인공지능 기술과 서비스의 이해
인공지능 기술과 서비스의 이해
 
How to give a good scientific oral presentation
How to give a good scientific oral presentationHow to give a good scientific oral presentation
How to give a good scientific oral presentation
 

Similar a Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13

Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
Jeffrey T. Pollock
 

Similar a Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13 (20)

Bringing Data to the Edge
Bringing Data to the EdgeBringing Data to the Edge
Bringing Data to the Edge
 
Green Plum IIIT- Allahabad
Green Plum IIIT- Allahabad Green Plum IIIT- Allahabad
Green Plum IIIT- Allahabad
 
Flash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lonFlash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lon
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
 
Evolution from EDA to Data Mesh: Data in Motion
Evolution from EDA to Data Mesh: Data in MotionEvolution from EDA to Data Mesh: Data in Motion
Evolution from EDA to Data Mesh: Data in Motion
 
A peek into the future
A peek into the futureA peek into the future
A peek into the future
 
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014Webinar: How MongoDB is Used to Manage Reference Data - May 2014
Webinar: How MongoDB is Used to Manage Reference Data - May 2014
 
Predictions for the Future of Graph Database
Predictions for the Future of Graph DatabasePredictions for the Future of Graph Database
Predictions for the Future of Graph Database
 
Data Virtualization: An Essential Component of a Cloud Data Lake
Data Virtualization: An Essential Component of a Cloud Data LakeData Virtualization: An Essential Component of a Cloud Data Lake
Data Virtualization: An Essential Component of a Cloud Data Lake
 
ACES QuakeSim 2011
ACES QuakeSim 2011ACES QuakeSim 2011
ACES QuakeSim 2011
 
Jak konsolidovat Vaše databáze s využitím Cloud služeb?
Jak konsolidovat Vaše databáze s využitím Cloud služeb?Jak konsolidovat Vaše databáze s využitím Cloud služeb?
Jak konsolidovat Vaše databáze s využitím Cloud služeb?
 
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
DAMA & Denodo Webinar: Modernizing Data Architecture Using Data Virtualization
 
Qo Introduction V2
Qo Introduction V2Qo Introduction V2
Qo Introduction V2
 
Future of Data Strategy (ASEAN)
Future of Data Strategy (ASEAN)Future of Data Strategy (ASEAN)
Future of Data Strategy (ASEAN)
 
Managing the financial services data explosion
Managing the financial services data explosionManaging the financial services data explosion
Managing the financial services data explosion
 
Streaming is a Detail
Streaming is a DetailStreaming is a Detail
Streaming is a Detail
 
Data Mesh using Microsoft Fabric
Data Mesh using Microsoft FabricData Mesh using Microsoft Fabric
Data Mesh using Microsoft Fabric
 
Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data
Introduction to the Data Web, DBpedia and the Life-cycle of Linked DataIntroduction to the Data Web, DBpedia and the Life-cycle of Linked Data
Introduction to the Data Web, DBpedia and the Life-cycle of Linked Data
 
ITI015En-The evolution of databases (I)
ITI015En-The evolution of databases (I)ITI015En-The evolution of databases (I)
ITI015En-The evolution of databases (I)
 
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big DataVoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
VoltDB and HPE Vertica Present: Building an IoT Architecture for Fast + Big Data
 

Último

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Último (20)

Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 

Smart Data for Smart Meters - Presentation at Pilod2 Meeting 2013-11-13

  • 1. Smart data for smart meters Wouter Beek w.g.j.beek@vu.nl www.wouterbeek.com
  • 2. Context • • • • Energy label / Energy Performance of Buildings Directive (EPBD) Possible values: A - G Measurements are valid for 10 years. Requirement when buying or renting a house.
  • 3. EnergyLabels dataset in numbers • 2,354,560 entries • Energy index • Electricity consumption • Gas consumption • License: Creative Commons 0 • Dissemination date: 2012-11-05 • Updated on a daily basis • Issued by: Energielabels Agentschap NL • Related dataset?: Liander Open Data, approx. 1,250,000 entries.
  • 4. Linked Open Data • • • • • Connect to existing datasets. Connect to services. Run queries across datasets. Perform inference across datasets. Easy to create mash-ups / new applications. cheap to do all of this, only then will Linked Data be an enabler for large-scale innovation. If it is (disclaimer: this is a subjective claim)
  • 5. RDF files Domain-independent data conversions fully automated Relational DB Domain-dependent data conversions domain knowledge needed domain knowledge Simple RDF Link to external sources (linksets) domain knowledge needed XML files depends on structure domain knowledge Fixing bad data origin inconsistencies & inaccuracies Text files ambiguous Connect to services (e.g. query interface, maps) high level of reuse
  • 6. Technological contribution • From 3-star (published, open format) to 5-star (Linked Data, URI identifiers, linked to BAG). • Stored in 2.6 GB XML document containing one (1!) line :-) • DOM is too big to hold in RAM. • Convert to multi-line XML document. • XML2RDF conversion infrastructure: • Create a resource using primary/rigid properties. • Create triples for a resource
  • 7.
  • 8.
  • 9. Application based on 5-star dataset
  • 10. Using Linked Data (Wouter’s Inbox) Dear Wouter, we gave the students of our Semantic Web class the link to the Kadaster information, and made them enthusiastic to use it. As a result several now have build their apps around this data. But now it has been offline for several days. Cheers, Stefan.
  • 11. Main difficulties (1/3) Technical difficulties due to arbitrary data formatting. • Publishing data in a sane way decreases the conversion costs considerably. • In this use case: half of all the effort went into the 1 line XML...
  • 12. Main difficulties (2/3) Institutional difficulties: • Data publication is a short-duration visible event. • Data maintenance is a long-duration invisible event.
  • 13.
  • 14. “You can fool all the people some of the time, and some of the people all the time, but you cannot fool all the people all the time.” Abraham Lincoln Let's make some substitutions here... “All LOD datasets are offline some of the time, and some of the LOD datasets are offline all of the time, but not all LOD datasets are offline all of the time.” Wouter Beek
  • 15. Main difficulties (3/3) Infrastructural difficulties: • Assuming that some LOD data is online some of the time, we must explicitly represent the network of interconnected LOD datasets, institutions, and maintainers (DC, FOAF, VoID). • Anticipating malfunctioning datasets should be a standard part of the development API.
  • 16. Conclusion Only when the technical, institutional, and infrastructural problems are solved will Linked Data become an enabler for large-scale innovation.