NoSQL in research: Enipedia & AgentSpring

•

2 likes•1,369 views

Alfredas Chmieliauskas

Technology Business

What do we do at TUDelft E&I?

modelling and simulation
data analysis
economic forecasting and policy analysis
focus: big industry, infrastructure and energy
current projects: energy sector in the EU, port of Rotterdam,
etc

Alfredas Chmieliauskas NoSQL War Stories 2 / 27

Two stories

Enipedia.org
AgentSpring

Alfredas Chmieliauskas NoSQL War Stories 3 / 27

Enipedia.org

is a semantic wiki for energy data
50’000+ pages on energy infrastructure (power plants, pipes,
etc) and economics (markets, prices)
user contributed database (wiki)
analytics platform (powered by a triple store + sparql
endpoint)
12’000 unique monthly visitors, a few editors

Alfredas Chmieliauskas NoSQL War Stories 4 / 27

Enipedia.org stack

Figure: Stack

Alfredas Chmieliauskas NoSQL War Stories 5 / 27

Enipedia.org stack

Figure: Stack War Stories
Alfredas Chmieliauskas NoSQL 6 / 27

SparqlExtension: Visualisation

Alfredas Chmieliauskas NoSQL War Stories 7 / 27

SparqlExtension: Visualisation

Figure: SPARQL behind gas ﬂows

Alfredas Chmieliauskas NoSQL War Stories 8 / 27

Enipedia: Power plants

Figure: Major power plants in the EU

Alfredas Chmieliauskas NoSQL War Stories 9 / 27

Enipedia: Gas Infrastructure

Figure: Major natural gas pipelines

Alfredas Chmieliauskas NoSQL War Stories 10 / 27

SparqlExtension: Analytics

Figure: What if Japan switched from nuclear to natural gas? The result
is around 53Bcm of natural gas annually or 50% of current Japanese
imports and 18% of worlds total LNG market.

Alfredas Chmieliauskas NoSQL War Stories 11 / 27

Agent-based and graph-based simulations

things + relations between them = network (or graph)
graph + behavior = simulation

Alfredas Chmieliauskas NoSQL War Stories 12 / 27

AgentSpring

AgentSpring - an agent+graph based:
...modeling framework
...simulation engine
https://github.com/alfredas/AgentSpring

Alfredas Chmieliauskas NoSQL War Stories 13 / 27

AS stack

Neo4j
Spring + SpringDataGraph
Cypher and Gremlin graph query languages
RMI based client-server comm.
REST based web front-end
R library

Alfredas Chmieliauskas NoSQL War Stories 14 / 27

AS modeling framework: deﬁne the network

Figure: Agent+things+relationships=network

Alfredas Chmieliauskas NoSQL War Stories 15 / 27

AS modeling framework: deﬁne behavior

Figure: Behavior of one agent (modular composition)

Alfredas Chmieliauskas NoSQL War Stories 16 / 27

AS simulation engine

Figure: Running simulation - new relations are formed

Alfredas Chmieliauskas NoSQL War Stories 17 / 27

Applications

Interactions between energy markets (today)
Interactions between infrastructure and markets (next time)

Alfredas Chmieliauskas NoSQL War Stories 18 / 27

Thank You For Listening

Questions, comments?
a.chmieliauskas@tudelft.nl

Alfredas Chmieliauskas NoSQL War Stories 19 / 27

Viewers also liked

Prezentace Etwinning Tereza

test

風景南非火車之旅

Arcom 070809

AlphaTalks - Seedcamp 2009

AlphaTalks

Comenius Teenagers Year

landalo

Rozdily na školách - EN

landalo

S L I D E S H O Wprivate

deelespp

Poprve s tabletem W8.1 -TEV

landalo

Viewers also liked (9)

Prezentace Etwinning Tereza

test

風景南非火車之旅

Arcom 070809

AlphaTalks - Seedcamp 2009

Comenius Teenagers Year

Rozdily na školách - EN

S L I D E S H O Wprivate

Poprve s tabletem W8.1 -TEV

Similar to NoSQL in research: Enipedia & AgentSpring

Microsoft Innovation Center Rapperswil

mictc

Considerations for using NoSQL technology on your next IT project - Akmal Cha...

BCS Data Management Specialist Group

Talk done by Wilbert Alberts (ASML) at Sirius Day Eindhoven: ASML is the world's leading provider of lithography systems for the semiconductor industry. Such systems are controlled by more than 20 million lines of code. To improve the efficiency and quality of its software development process, ASML is using, amongst others, model-driven-engineering and associated tools and techniques. Recently, subsystems are being developed according to an architecture pattern that separates Data, Control and Algorithms (DCA). To support this pattern, the ASML software architecture group is working towards a SW development environment (ASOME). This environment consists of a set of modeling languages, associated editors that allow specification of (sub)systems according to this DCA pattern. Furthermore, it contains model-to-model transformations to (COTS) analysis tools (e.g. model checkers) and model-to-text transformation to generate (parts of) the implementation. In this presentation, I will briefly introduce ASML and the kind of (software) systems that we develop. Some aspects of the DCA architectural pattern, the languages that we are developing and the associated Sirius based editors, will be presented. For the Data part, a DSL and editor have been developed allowing the definition of various kinds of datatypes from which various kinds of repositories can be generated supporting clone based data or reference based data, modifiable and read-only entities etc. In order support the Control aspect; a language and editor have been defined that allow specification of interfaces and their realization based on state machines. A system editor allows decomposition of a system into subsystems while allowing delegation of incoming requests to internal parts. The editors are mostly Sirius based graphical editors, where the created models are persisted textually using XText. The presentation will focus on sharing some of our experiences with both the development and deployment of products based on Sirius technology. Building the ASOME environment imposes many challenges and I would like to conclude with some that specifically target the development of the front ends of this environment.

[Sirius Day Eindhoven 2018] ASML's MDE Going Sirius

Obeo

Bandwidth: Use Cases for Elastic Cloud on Kubernetes

Elasticsearch

AutomationML (Automation Markup Language) is a neutral data format based on XML for the storage and exchange of plant engineering information, which is provided as open standard. Goal of AutomationML is to interconnect the heterogeneous tool landscape of modern engineering tools in their different disciplines, e.g. mechanical plant engineering, electrical design, HMI development, PLC, robot control. This presentation provides an overview on AutomationML and a model-driven engineering view on its integration capabilities.

AutomationML: A Model-Driven View

Luca Berardinelli

Considerations for using NoSQL technology on your next IT project

Akmal Chaudhri

Considerations for using NoSQL technology on your next IT project

Akmal Chaudhri

Considerations for using NoSQL technology on your next IT project

Akmal Chaudhri

Considerations for using NoSQL technology on your next IT project

Akmal Chaudhri

Thirteen Years of SysML: A Systematic Mapping Study

swolny

Graph Databases, RDF Triplets, Ontologies, SWRL and SPARQL Queries are modern technologies for knowledge-based artificial intelligence. In this presentation, wou will see how Industry 4.0 standards such as AutomationML (AML), OPC-UA and eCl@ss create the conditions for unparalleled engineering power in manufacturing in general, and in assembly lines in particular. AI is not only data-hungry deep learning. Use what we already know before you try to guess what's new. Let's make the fourth industrial revolution elegant and energy-efficient! Weiss GmbH provides unique automation components and their Digital Twin partial models by the I4.0 standards.

Towards batch one size with industrial semantics email

Paulo Zanini

Tinder’s Quickfire Pipeline powers all things data at Tinder. It was originally built using AWS Kinesis Firehoses and has since been extended to use both Kafka and other event buses. It is the core of Tinder’s data infrastructure. This rich data flow of both client and backend data has been extended to service a variety of needs at Tinder, including Experimentation, ML, CRM, and Observability, allowing backend developers easier access to shared client side data. We perform this using many systems, including Kafka, Spark, Flink, Kubernetes, and Prometheus. Many of Tinder’s systems were natively designed in an RPC first architecture. Things we’ll discuss decoupling your system at scale via event-driven architectures include: – Powering ML, backend, observability, and analytical applications at scale, including an end to end walk through of our processes that allow non-programmers to write and deploy event-driven data flows. – Show end to end the usage of dynamic event processing that creates other stream processes, via a dynamic control plane topology pattern and broadcasted state pattern – How to manage the unavailability of cached data that would normally come from repeated API calls for data that’s being backfilled into Kafka, all online! (and why this is not necessarily a “good” idea) – Integrating common OSS frameworks and libraries like Kafka Streams, Flink, Spark and friends to encourage the best design patterns for developers coming from traditional service oriented architectures, including pitfalls and lessons learned along the way. – Why and how to avoid overloading microservices with excessive RPC calls from event-driven streaming systems – Best practices in common data flow patterns, such as shared state via RocksDB + Kafka Streams as well as the complementary tools in the Apache Ecosystem. – The simplicity and power of streaming SQL with microservices

Event Driven Architecture with a RESTful Microservices Architecture (Kyle Ben...

confluent

Machine learning at scale challenges and solutions

Stavros Kontopoulos

CAQA5e_ch1 (3).pptx

SPOCSumaLatha

We are searching the unknown. How can you find hidden and unknown relationships in unrelated data silos? How can you find relevant information in a 10^56 dimensional space? Sounds impossible? This talk will present a case study and success story about how Apache Solr has been used to build a search based business intelligence and information research application to answer these questions. The talk was delivered at the Apache: Big Data 2015 Conference in Budapast. #apachebigdataeu2015

Search-based business intelligence and reverse data engineering with Apache Solr

Mario-Leander Reimer

VILLASframework - A toolset for local and geographically distributed real-tim...

Steffen Vogel

Let's get back to the time when languages could be defined with just 6 reserved keywords or none at all, and learn how the ideas that John McCarthy and Alan Key envisioned in Lisp and Smalltalk are still alive and kicking in our serverless and message-driven world. No matter the language you are using now, and how much you are an OOP or functional guy, you'll learn how to develop better serverless applications and build extensible and decoupled workflows.

Alessandro Confetti - Oop vs functional: stop the fight and start building me...

Codemotion

Alessandro Confetti - Oop vs functional: stop the fight and start building me...

Codemotion

Oop vs functional stop the fight and start building message driven serverle...

Alessandro Confetti

The International Journal of Engineering & Science is aimed at providing a platform for researchers, engineers, scientists, or educators to publish their original research results, to exchange new ideas, to disseminate information in innovative designs, engineering experiences and technological skills. It is also the Journal's objective to promote engineering and technology education. All papers submitted to the Journal will be blind peer-reviewed. Only original articles will be published.

The International Journal of Engineering and Science (The IJES)

theijes

Similar to NoSQL in research: Enipedia & AgentSpring (20)

Microsoft Innovation Center Rapperswil

Considerations for using NoSQL technology on your next IT project - Akmal Cha...

[Sirius Day Eindhoven 2018] ASML's MDE Going Sirius

Bandwidth: Use Cases for Elastic Cloud on Kubernetes

AutomationML: A Model-Driven View

Considerations for using NoSQL technology on your next IT project

Thirteen Years of SysML: A Systematic Mapping Study

Towards batch one size with industrial semantics email

Event Driven Architecture with a RESTful Microservices Architecture (Kyle Ben...

Machine learning at scale challenges and solutions

CAQA5e_ch1 (3).pptx

Search-based business intelligence and reverse data engineering with Apache Solr

VILLASframework - A toolset for local and geographically distributed real-tim...

Alessandro Confetti - Oop vs functional: stop the fight and start building me...

Oop vs functional stop the fight and start building message driven serverle...

The International Journal of Engineering and Science (The IJES)

Recently uploaded

Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf

FIDO Alliance

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...

ScyllaDB

The Metaverse: Are We There Yet?

Mark Billinghurst

Portal Kombat : extension du réseau de propagande russe

中央社

In the ever-evolving landscape of data management, Zero-ETL is an approach that is reshaping how businesses handle and integrate their data. This webinar explores Zero-ETL, a paradigm shift from the traditional Extract, Transform, Load (ETL) process, offering a more streamlined, efficient, and real-time data integration method. We will begin with an introduction to the concept of Zero-ETL, including how it allows direct access to data in its native environment and real-time data transformation, providing up-to-date information with significantly reduced data redundancy. Next, we'll take you through several demonstrations showing how Zero-ETL can deliver real-time data and enable the free movement of data between systems. We will also discuss the various tools that support all aspects of Zero-ETL, providing attendees with an understanding of how they can adopt this innovative approach in their organizations. Lastly, the session will conclude with an interactive Q&A segment, allowing participants to gain deeper insights into how Zero-ETL can be tailored to their specific business needs and how they can get started today. Join us to discover how Zero-ETL can elevate your organization's data strategy.

The Zero-ETL Approach: Enhancing Data Agility and Insight

Safe Software

Join me in this session where I'll share our journey of building a fully serverless application that flawlessly managed check-ins for an event with a staggering 80 thousand registrations. We'll dive into three key strategies that made this possible. Firstly, by harnessing DynamoDB global tables, we ensured global service availability and data replication across regions, boosting performance and disaster recovery. Next, we'll explore how we seamlessly integrated real-time updates into the app using Appsync subscriptions, making the experience dynamic and engaging for users. Finally, I'll discuss how provisioned concurrency not only improved performance but also kept costs in check, highlighting the cost-effectiveness of serverless architectures. Through these strategies and the inherent scalability of serverless technology, our application effortlessly handled massive user loads without manual intervention. This session is a real world example to the power and efficiency of modern cloud-based solutions in enabling seamless scalability and robust performance with Serverless

How we scaled to 80K users by doing nothing!.pdf

Srushith Repakula

Generative AI refers to a class of machine learning algorithms that are designed to generate new data samples that are similar to those in the training data. Unlike traditional AI models that are trained to recognize patterns and make predictions, generative AI models have the ability to create entirely new data based on the patterns they have learned. This is achieved through techniques such as generative adversarial networks (GANs), variational autoencoders (VAEs), and transformer architectures, among others.

Generative AI Use Cases and Applications.pdf

alexjohnson7307

Explore the latest trends and insights on JavaScript usage with Pixlogix's informative blog. Discover key statistics and facts about JavaScript's role in web development, its popularity among developers, and its impact on modern websites. Stay updated with the evolving landscape of JavaScript frameworks and libraries, and learn how they're shaping the future of web development. Gain valuable insights to enhance your JavaScript skills and stay ahead in the digital realm.

JavaScript Usage Statistics 2024 - The Ultimate Guide

Pixlogix Infotech

Extensible Python: Robustness through Addition - PyCon 2024

Patrick Viafore

Event-Driven Architecture Masterclass: Challenges in Stream Processing

ScyllaDB

Ruby has a lot of standard libraries from Ruby 1.8. I promote them democratically with GitHub today via default and bundled gems. So, I'm working to extract them for Ruby 3.4 continuously and future versions. It's long journey for me. After that, some versions may suddenly happen LoadError at require when running bundle exec or bin/rails, for example matrix or net-smtp. We need to learn what's difference default/bundled gems with standard libraries. In this presentation, I will introduce what's the difficult to extract bundled gems from default gems and the details of the functionality that Ruby's require and bundle exec with default/bundled gems. You can learn how handle your issue about standard libraries.

Long journey of Ruby Standard library at RubyKaigi 2024

Hiroshi SHIBATA

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx

FIDO Alliance

Working together SRE & Platform Engineering

Marcus Vechiato

Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside

Stefan Dietze

ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...

FIDO Alliance

Ivanti’s Patch Tuesday breakdown goes beyond patching your applications and brings you the intelligence and guidance needed to prioritize where to focus your attention first. Catch early analysis on our Ivanti blog, then join industry expert Chris Goettl for the Patch Tuesday Webinar Event. There we’ll do a deep dive into each of the bulletins and give guidance on the risks associated with the newly-identified vulnerabilities.

2024 May Patch Tuesday

Ivanti

WebRTC and SIP not just audio and video @ OpenSIPS 2024

Lorenzo Miniero

Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...

FIDO Alliance

Webinar Recording: https://www.panagenda.com/webinars/alles-neu-macht-der-mai-wir-durchleuchten-den-verbesserten-notes-eigenschaftendialog/ Haben Sie sich schon einmal über den zu kleinen Eigenschaftendialog in Notes geärgert? Mussten Sie einen Agenten oder eine Aktion erstellen, um schnell mal ein Feld zu ändern? Haben Sie jedes mal endlos nach dem zu vergleichenden Feld gesucht, nachdem Sie ein neues Dokument ausgewählt haben? Wollten Sie das verdammte Ding einfach nur größer machen? Zum Glück gibt es dafür eine Lösung – und sie ist wahrscheinlich bereits installiert! Mit dem kostenlosen panagenda Document Properties (Pro) erhalten Sie den Eigenschaftendialog, den Sie schon immer haben wollten. Größer, anpassbar, und im Volltext durchsuchbar. Sehen Sie mehrere Dokumente gleichzeitig oder vergleichen Sie mit einem Diff-Viewer. Ändern Sie beliebige Felder und haben Sie endlich eine einfache Möglichkeit, Profildokumente für alle Benutzer zu verwalten. Entdecken Sie mit HCL Ambassador Marc Thomas, wie Document Properties Ihre Arbeit vereinfachen und Sie bei der täglichen Verwendung von Domino-Anwendungen unterstützen kann – im Client oder im Designer. Sie werden es nicht bereuen! Für Sie in diesem Webinar - Was Document Properties ist, welche Editionen es gibt und wo es in Notes und Domino Designer zu finden ist - Wie Sie nach einem beliebigen Feld suchen und es bearbeiten, Dokumente vergleichen oder alle Daten per CSV exportieren können - Suchen, Bearbeiten und auch Löschen von Profildokumenten - Welche Konfigurationseinstellungen verfügbar sind, um Funktionen anzupassen - Wie Ihre Endbenutzer davon profitieren - Sehen Sie alles in einer Live-Demo

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...

panagenda

Design Guidelines for Passkeys 2024.pptx

FIDO Alliance

Recently uploaded (20)

Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf

Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...

The Metaverse: Are We There Yet?

Portal Kombat : extension du réseau de propagande russe

The Zero-ETL Approach: Enhancing Data Agility and Insight

How we scaled to 80K users by doing nothing!.pdf

Generative AI Use Cases and Applications.pdf

JavaScript Usage Statistics 2024 - The Ultimate Guide

Extensible Python: Robustness through Addition - PyCon 2024

Event-Driven Architecture Masterclass: Challenges in Stream Processing

Long journey of Ruby Standard library at RubyKaigi 2024

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx

Working together SRE & Platform Engineering

Collecting & Temporal Analysis of Behavioral Web Data - Tales From The Inside

ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...

2024 May Patch Tuesday

WebRTC and SIP not just audio and video @ OpenSIPS 2024

Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...

Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...

Design Guidelines for Passkeys 2024.pptx

NoSQL in research: Enipedia & AgentSpring

1. NoSQL in research: two stories Alfredas Chmieliauskas TU Delft, TBM, Energy & Industry Section April 11, 2012 Alfredas Chmieliauskas NoSQL War Stories 1 / 27

2. What do we do at TUDelft E&I? modelling and simulation data analysis economic forecasting and policy analysis focus: big industry, infrastructure and energy current projects: energy sector in the EU, port of Rotterdam, etc Alfredas Chmieliauskas NoSQL War Stories 2 / 27

3. Two stories Enipedia.org AgentSpring Alfredas Chmieliauskas NoSQL War Stories 3 / 27

4. Enipedia.org is a semantic wiki for energy data 50’000+ pages on energy infrastructure (power plants, pipes, etc) and economics (markets, prices) user contributed database (wiki) analytics platform (powered by a triple store + sparql endpoint) 12’000 unique monthly visitors, a few editors Alfredas Chmieliauskas NoSQL War Stories 4 / 27

5. Enipedia.org stack Figure: Stack Alfredas Chmieliauskas NoSQL War Stories 5 / 27

6. Enipedia.org stack Figure: Stack War Stories Alfredas Chmieliauskas NoSQL 6 / 27

7. SparqlExtension: Visualisation Alfredas Chmieliauskas NoSQL War Stories 7 / 27

8. SparqlExtension: Visualisation Figure: SPARQL behind gas ﬂows Alfredas Chmieliauskas NoSQL War Stories 8 / 27

9. Enipedia: Power plants Figure: Major power plants in the EU Alfredas Chmieliauskas NoSQL War Stories 9 / 27

10. Enipedia: Gas Infrastructure Figure: Major natural gas pipelines Alfredas Chmieliauskas NoSQL War Stories 10 / 27

11. SparqlExtension: Analytics Figure: What if Japan switched from nuclear to natural gas? The result is around 53Bcm of natural gas annually or 50% of current Japanese imports and 18% of worlds total LNG market. Alfredas Chmieliauskas NoSQL War Stories 11 / 27

12. Agent-based and graph-based simulations things + relations between them = network (or graph) graph + behavior = simulation Alfredas Chmieliauskas NoSQL War Stories 12 / 27

13. AgentSpring AgentSpring - an agent+graph based: ...modeling framework ...simulation engine https://github.com/alfredas/AgentSpring Alfredas Chmieliauskas NoSQL War Stories 13 / 27

14. AS stack Neo4j Spring + SpringDataGraph Cypher and Gremlin graph query languages RMI based client-server comm. REST based web front-end R library Alfredas Chmieliauskas NoSQL War Stories 14 / 27

15. AS modeling framework: deﬁne the network Figure: Agent+things+relationships=network Alfredas Chmieliauskas NoSQL War Stories 15 / 27

16. AS modeling framework: deﬁne behavior Figure: Behavior of one agent (modular composition) Alfredas Chmieliauskas NoSQL War Stories 16 / 27

17. AS simulation engine Figure: Running simulation - new relations are formed Alfredas Chmieliauskas NoSQL War Stories 17 / 27

18. Applications Interactions between energy markets (today) Interactions between infrastructure and markets (next time) Alfredas Chmieliauskas NoSQL War Stories 18 / 27

19. Thank You For Listening Questions, comments? a.chmieliauskas@tudelft.nl Alfredas Chmieliauskas NoSQL War Stories 19 / 27

NoSQL in research: Enipedia & AgentSpring

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

Similar to NoSQL in research: Enipedia & AgentSpring

Similar to NoSQL in research: Enipedia & AgentSpring (20)

Recently uploaded

Recently uploaded (20)

NoSQL in research: Enipedia & AgentSpring