The document discusses enterprise integration patterns (EIPs) and how they are relevant for integrating applications and systems in an era of big data and the internet of things. It provides an agenda covering application integration, EIPs, modeling, frameworks and tools. It then discusses how EIPs apply to big data, IoT, microservices, and the future of integration. Real-time integration is highlighted as important for a world with more connected devices and data sources.
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
Enterprise Integration Patterns Revisited (again) for the Era of Big Data, Internet of Things and Microservices
1. Enterprise Integration Patterns
Revisited for an Era of Big Data and Internet of Things
Kai Wähner
kontakt@kai-waehner.de
@KaiWaehner
www.kai-waehner.de
LinkedIn / Xing à Please connect!
2. Key Take-Aways
– Integration is the key factor for success of business in the future!
– Enterprise Integration Patterns are used everywhere!
– Real time integration is the game changer in a Big Data and IoT world!
3. Agenda
Ø Application Integration
Ø Enterprise Integration Patterns
Ø Modeling
Ø Frameworks and Tools
Ø Big Data, Internet of Things, Microservices
Ø Future
4. Agenda
Ø Application Integration
Ø Enterprise Integration Patterns
Ø Modeling
Ø Frameworks and Tools
Ø Big Data, Internet of Things, Microservices
Ø Future
5. Growing IT Infrastructure Everywhere
• Applications
• Interfaces
• Technologies
• Products
• Cloud
• Mobile
• Internet of Things
10. ... vs. using Patterns, Frameworks, Tools
• Standardized Modeling
• Efficient Realization
• Automatic Testing
11. Agenda
Ø Application Integration
Ø Enterprise Integration Patterns
Ø Modeling
Ø Frameworks and Tools
Ø Big Data, Internet of Things, Microservices
Ø Future
12. “A software design pattern, in the sense of a template,
is a general solution to a problem in programming. A
design pattern provides a reusable architectural outline
that may speed the development of many computer
programs.”
http://en.wikipedia.org/wiki/Design_pattern
What is a Design Pattern?
14. “The goal of EIPs is to document
technology-independent design
guidance that helps developers and
architects describe and develop
robust INTEGRATION solutions.”
http://www.eaipatterns.com
Enterprise Integration Patterns (EIP)
15. History of EIPs
• Talk from Gregor Hohpe
– WSO2 Con 2011
– „History, present, future of EIP“
– A lot about past (ideas, creation of
book, ...)
– http://www.youtube.com/watch?
v=Xwi1DU6KoQ4
• Today‘s Talk
– Not too much about history
– Just about present and future
28. Kai Wähner - Enterprise Integration Patterns Revisited
Complexity
of Integration
Integra6on
Products
Integra6on
Suite
Low High
Integration
Framework
INTEGRATION
Connectivity
Routing
Transformation
When to use an Integration Framework?
34. Kai Wähner - Enterprise Integration Patterns Revisited
Connectivity
Routing
Transformation
INTEGRATION
Tooling
Monitoring
Support
Complexity
of Integration
Integra6on
Products
Integra6on
Suite
Low High
Integration
Framework
+
When to use an Enterprise Service Bus?
37. Adaptive Integration Approaches
Self-Service Integration for
DIY Integrators (iSaaS)
Business Integration
Cloud Integration
Integration Platform as a
Service (iPaaS)
Enterprise
Business
Users
Department
Audience
Enterprise Integration
Platform
(on-premise and cloud)
Enterprise Service Bus
40. Enterprise Service Bus (ESB)
Intuitive graphical design environment
Streamlines time and cost of development and training
Graphical
Process
Modeling
Native
Standards based
XSLT Mapper
Drag-n-Drop
Access to
Resources
Fully Integrated
Test Environment
Adapters, Services, Processes,
Deployment, Management
42. iSaaS: Integration for the “Prosumer”
Subscription cloud service that
makes it easy to automate moving
of data between apps on the Web
43. iSaaS: Integration for the “Prosumer”
• Connectors
– SaaS APIs baked in a common
user interface
• Recipes
– Combinations of connectors
preconfigured as “apps”
– Just configure to your accounts
– Customize as needed
• Drag and Drop ease
44. Agenda
Ø Application Integration
Ø Enterprise Integration Patterns
Ø Modeling
Ø Frameworks and Tools
Ø Big Data, Internet of Things, Microservices
Ø Future
46. Evolution of Integration
Microservices
Web
APIs
Real-‐6me
Web
Oriented
Architecture
Enabling
Technologies
In-‐Memory,
Mul6core
REST,
JSON
Demand
Drivers
Mobile,
Cloud
Fast
Data,
IoT
Services
Web
Services
Real-‐6me
Service
Oriented
Architecture
Enabling
Technologies
XML,
SOAP,
WS-‐*
Process
Modeling
Demand
Drivers
E-‐Commerce
BPM
Interfaces
Adapters
Real-‐6me
Enterprise
Applica6on
Integra6on
Enabling
Technologies
Client-‐Server
Messaging
Middleware
Demand
Drivers
ERP
Analy6cs
Records
Batch
Jobs
Non-‐real6me
Enabling
Technologies
Mainframe
ETL,
Databases
Data
Integra6on
Demand
Drivers
Data
Processing
MIS
Are
we
there
yet?
Level-‐Up
by
u=lizing
the
lessons,
assets
and
prac=ces
of
the
previous
Level
Accelera=ng
Produc=vity
&
Agility
47. Enterprise Integration Patterns (EIPs) …
• … are needed everywhere!
• Not just in an Integration Framework
or Integration Product!
49. EIPs for Streaming Analytics
Stream Processing for Real Time Processing of Big Masses of Data
à Fast Data
50. EIPs for Open API and API Management
Your
Enterprise
Closed
APIs
EDI
Web
B2B
SOA
FTP
Pre-‐defined
integra6on
points
Limited,
trusted
partners
Strictly
constrained
interac6ons
Enterprise
friendly
Open
APIs
Opportunis6c
access
points
Many
partners,
untrusted
Encourage
new
ideas
Developer
friendly
Your
Enterprise
Partners
Innovators
Known
Par6es
Consumers
Unknown
Par6es
Suppliers
Coope66on
Employees
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
API
53. EIPs for Business Logic
E.g. with Microservices:
• Services implementing a limited set of business functions
• Services developed, deployed and scaled independently
Integration
Service
Monolith application SOA
Integration
Service
Business
Service
Business
Service
Business
Service
Micro
Service
API Gateway
SaaS Business
Service
54. EIPs for Integration of “Internet of Everything”
à Real Time Integration with Things, APIs, Business Processes, Cloud, Social, etc.
hep://blogs.cisco.com/ioe/how-‐the-‐internet-‐of-‐everything-‐will-‐change-‐the-‐worldfor-‐the-‐beeer-‐infographic
56. One or more frameworks / vendors?
Frameworks or Products from different Vendors
• No license costs (for open source frameworks)
• A lot of glue code
• Testing / Bugfixing
• No (combined) support
Frameworks or Products from a single Vendor
• All integrated within one framework or product? à Vision!
• Not integrated? à Often reality!
• Loosely coupled, highly integrated frameworks or products? à Best choice!
57. Agenda
Ø Application Integration
Ø Enterprise Integration Patterns
Ø Modeling
Ø Frameworks and Tools
Ø Big Data, Internet of Things, Microservices
Ø Future
58. Future of EIPs: The BAD News
• New Patterns (since release of the book)? ß NOT YET
– Conversations
– Complex Event Processing
– Error Handling
• More Details: „EIP Visions“ ß TO BE DETERMINED
– EIP I: Messaging à done
– EIP II: Conversations à 50 page paper
– EIP III: Process à tbd
– EIP IV: Event Processing à tbd
http://www.eaipatterns.com/ramblings/72_eipvolumes.html
59. Future of EIPs: The GOOD News
• New Patterns not necessary! ß OFTEN
– EIPs were based on Messaging already, which is a good thing (usually)
– Conversations == State == Anti Pattern (often)
– Integration scenarios can be solved with existing patterns (usually)
• Better Tooling ß YES
– More frameworks and products (e.g. Stream Processing for Big Data / Fast
Data, Internet of Things)
– Stable, mature tools
– New standards (MQTT, OMG's DDS, Eclipse Open IoT Stack, etc.)
– Support for new hardware (iBeacons, OSIsoft Pi, Google Glass, etc.)
– Ease of use (no coding)
– Better scalability
60. Future of Integration and EIPs
Buy
this
book
if
you
do
not
own
it
already!
Integration will get even more important in the future
than it is today!
The number of different data sources and technologies increases even more than in the past:
• CRM, ERP, Mainframe, B2B (EDIFACT), etc. will not disappear
• DWH, Hadoop cluster, event / streaming server, In-Memory DB – all of them have to
communicate
• Cloud, Mobile, APIs, Big Data, Internet of Things are no option, but our future!
EVERYTHING HAS TO BE INTEGRATED!
62. Integration is the key factor for success of business in the future!
Enterprise Integration Patterns are used everywhere!
Real time integration is the game changer in a Big Data and IoT world!
Recap: Key Take-Aways