AWS based Enterprise Digital Transformation Platform (EDTP) is architected as an Event Processing Digital Center around which all Businesses’ current and future data sources, consumers, services, and processes interact. The purpose of the Platform is to enable business innovation and agility by providing semantically-cohesive and structurally-flexible harmonized data across processes and systems and by bringing functions and capabilities to data instead of moving data.
2. Agenda
• Business’ Ecosystems: Data and Integration Challenges
ü Digital Transformation Drivers
ü Digital Transformation Inhibitors
ü Digital Transformation Platform Northern Star Attributes
• How Platform Computing and EDTP Address Challenges
ü Enterprise Digital Transformation Platform
ü EDTP Layered Architecture
ü EDTP Layered Distributed Architecture
• EDTP Deep Dive
ü EDTP Fully Installed Through Automation on AWS
• Demo 1: Business Agility Through Automation of the
EDTP Full Stack Deployment
ü Demo 1a: ”Core Platform Pipeline”
ü Demo 1b: “Tech Components Pipeline”
ü Demo 1c: “Service Templates Pipeline”
• Demo 2: Accelerated Deployment of Digital Transformation
Solutions Based on the EDTP Data and Integration Services
ü Demo 2a: “Data Quality Through Harmonization”
ü Demo 2b: “Diverse Analytics”
ü Demo 2c: “Microservices Through Templating”
• Sample Use Cases:
ü Enterprise Logging Framework
ü Enterprise Interactions Logging Framework
ü Next Generation Campaign Management
ü Insurance Forms Automation
• Discussion and Next Steps
2
December 12, 2020 Enterprise Digital Transformation Platform
4. Digital Transformation Drivers
Businesses may take on Digital
Transformation for several reasons. But
by far, the most likely reason is that
they have to: It is a survival issue.
A business’ ability to adapt quickly to
disruptions from incumbents and
startups, to time to market pressures,
and to rapidly changing customer
expectations has become critical.
It is about evolutions and changes
4
December 12, 2020 Enterprise Digital Transformation Platform
5. Digital Transformation Inhibitors
Data and integration challenges
hindering businesses’ ability to
transform:
1. Data Quality and Consistency,
2. Coupling on Data & BL levels,
3. Point-to-Point Integration n-
square problem,
4. Point-to-Point with imbedded
transformations,
5. Multiple runtime environments.
5
December 12, 2020 Enterprise Digital Transformation Platform
6. Digital Transformation Platform North Star Attributes
• Decoupling
• Adaptability:
⎯ Service Layer
⎯ Data Layer
⎯ Integration Layer
• Agility
• Elasticity
• Experimenting
• Fast-failing
6
December 12, 2020 Enterprise Digital Transformation Platform
7. How Platform Computing and
EDTP Address Challenges
7
December 12, 2020 Enterprise Digital Transformation Platform
8. Enterprise Digital Transformation Platform
To address the challenges we
developed EDTP for accelerated
development and deployment of a
wide range of agile Services and
Data Digital Transformation
Solutions (DTS) that scale.
The solutions are based on the
modern 4-tier architectural styles
including cloud, containers,
microservices, events, streaming,
and sync & async processing.
8
December 12, 2020 Enterprise Digital Transformation Platform
9. EDTP Layered Architecture
EDTP is built on top of core
technologies including Docker
Kubernetes and OpenShift:
• Docker provides container services,
• Kubernetes run and scale containers
for production,
• OpenShift provides developers with
the features to manage their DevOps,
• EDTP builds on to of it to deliver full-
stack automation and accelerators for
delivery of DTS.
9
December 12, 2020 Enterprise Digital Transformation Platform
10. EDTP Layered Distributed Architecture
EDTP has a microservices-based architecture of smaller, decoupled
units that work together. It runs on top of OpenShift and Kubernetes
clusters. The services are broken down by function:
• EDTP Technology Components and Services are deployed in
Containers - a virtual boundary of compute and memory resources
assigned to the components. The containers spin-up from Docker
images.
• OpenShift leverages the Kubernetes concept of a POD, which is one or
more Containers deployed together on one Worker Node, and the
smallest compute unit that can be defined, deployed, and managed.
• A Kubernetes Service serves as an internal load balancer. It exposes
an applications running on a set of PODs as a network service.
• An OpenShift Route is a way to expose a Service by giving it an
externally reachable hostname .
• The control plane, which is composed of Master Nodes, manages the
OpenShift cluster.
10
December 12, 2020 Enterprise Digital Transformation Platform
13. Demo 1:Business Agility Through
Automation of the EDTP Full
Stack Deployment
13
December 12, 2020 Enterprise Digital Transformation Platform
14. Demo 1: Scope
• Demo 1a – “Core Platform Pipeline”: deployment of the EDTP core platform components
(cloud agnostic infrastructure, Docker, Kubernetes and OpenShift) using Terraform.
• Demo 1b – “Tech Components Pipelines”: walkthrough the Jenkins pipelines for
deployments of the EDTP pluggable digital technology components (Ambassador, Nifi,
Zookeeper, Kafka, MongoDB, Istio and Elastic Stack).
• Demo 1c – “Service Templates Pipelines”: walkthrough the Jenkins pipelines for
deployments of the EDTP data and integration service templates (microservices, data
services, data streaming services, aggregation services, event processing services and
several integration patterns). The templates are used for accelerated deployments of
digital transformation solutions.
14
December 12, 2020 Enterprise Digital Transformation Platform
15. Demo 1a: Core Platform Pipeline
15
EDTP Core Components deployed.
December 12, 2020 Enterprise Digital Transformation Platform
16. Demo 1a: From Empty VPC
16
December 12, 2020 Enterprise Digital Transformation Platform
17. Demo 1a: To Core EDTP
17
December 12, 2020 Enterprise Digital Transformation Platform
18. Demo 1b: Tech Components Pipeline
18
Technology Components added to
the Core Platform
EDTP Core Components deployed.
December 12, 2020 Enterprise Digital Transformation Platform
19. 19
Demo 1b: From Core EDTP
December 12, 2020 Enterprise Digital Transformation Platform
20. Demo 1b: To Core EDTP & Tech Components
20
December 12, 2020 Enterprise Digital Transformation Platform
21. Demo 1c: Service Templates Pipeline
21
Technology Components added to
the Core Platform
EDTP Core Components deployed.
Services added to the stack
December 12, 2020 Enterprise Digital Transformation Platform
22. Demo 1c: Keep Adding Services
22
December 12, 2020 Enterprise Digital Transformation Platform
23. Demo 2: “Accelerated Deployment of Digital
Transformation Solutions Based on the EDTP
Data and Integration Service Templates”
23
December 12, 2020 Enterprise Digital Transformation Platform
24. Demo 2: Scope
• Demo 2a – “Data Quality Through Harmonization”: deployment of a
data harmonization solution based on the EDTP data streaming
service template.
• Demo 2b – “Diverse Analytics”: deployment of a diverse analytics
solution based on the EDTP data streaming service template for
loading MongoDB JSON data to Snowflake tables for analytics.
• Demo 2c – “Microservices”: deployment of scalable microservices
from the EDTP templates.
24
December 12, 2020 Enterprise Digital Transformation Platform
25. Demo 2a: Data Quality Through Harmonization
We demonstrate how EDTP ingress a stream of raw records, harmonizes and materializes records in
MongoDB collections for consumption by others utilizing egress services. The solution is fully
configurable via AVRO schemas (raw, harmonized, and materialize). Several EDTP features are
highlighted during the demo:
• Processing of batch and streaming data including CDC,
• Data quality & master data through harmonization,
• Transforming streaming data in flight via configurable DSL,
• Data-Vault pattern and full audit trail,
• Meta-data management,
• Flexibility of JSON/AVRO schemas as opposed to complex CDM,
• Flexibility of an elastic search index for cataloging, browsing and searching data.
25
December 12, 2020 Enterprise Digital Transformation Platform
26. Demo 2a: EDTP Component Interactions
26
December 12, 2020 Enterprise Digital Transformation Platform
27. Demo 2b: Diverse Analytics
We demonstrate a diverse analytics solution based on the
EDTP data streaming service template for loading MongoDB
JSON data to Snowflake tables for analytics. The demo is an
extension of Demo 2a and it highlights several EDTP
integration features including:
• Distributed, evolutionary architecture,
• Extensibility via connectors – Snowflake Connector for Kafka,
• Integration via loose coupling of the right systems for the job.
27
December 12, 2020 Enterprise Digital Transformation Platform
28. Demo 2: Component Interaction View
1. Raw records
dropped into S3
1
2 3
4
2. Nifi service processes
records and publishes
on Raw topic
3. Harmonization service
processes raw data into
MongoDB and publishes
them on Harmonized
topic
4. Kafka Snowflake
Connector Service loads
data into Snowflake
table
28
December 12, 2020 Enterprise Digital Transformation Platform
29. Demo 2: From S3 to MongoDB to Snowflake
1. Raw records dropped into
S3
2. Nifi service processes
records and publishes on
Raw topic
3. Harmonization service
processes raw data into
MongoDB and publishes
them on Harmonized topic
4. Kafka Snowflake
Connector Service loads
data into Snowflake table
1
2
3
4
29
December 12, 2020 Enterprise Digital Transformation Platform
30. Demo 2c: Microservices Through Templating
We demonstrate deployment of scalable microservices from the EDTP templates. The demonstration
highlights several EDTP features including:
• Microservices architectural style,
• Decoupling of software components on both business capability and data levels (smart endpoints dumb pipes),
• Exposing services as REST APIs – EDTP as API Economy Platform,
• Four tier digital transformation platform (services tier, aggregation tier, delivery tier and client tier),
• Simplified integration due to loose coupling of software components including mainframe and legacy applications,
• Processing patterns: synchronous and asynchronous with or without retries,
• Event-Driven Architecture Patterns: event notification, event-carried state transfer, event-sourcing and CQRSP (command
query responsibility segregation pattern),
• Handlers – for implementation of client specific business logic,
• Full-stack CI/CD/CT and DevOps.
30
December 12, 2020 Enterprise Digital Transformation Platform
31. Demo 2c: Templating Idea
31
In Software Development we develop classes and
use them to instantiate objects.
In EDTP Service Development we develop
Templates and Registration Services for a given
Service Class and use them to instantiate new
services of the same class.
December 12, 2020 Enterprise Digital Transformation Platform
32. Demo 2c: Service Classes based on Patterns
Templating idea is especially powerful when combined with an idea of
using Patterns to define Service Classes
32
From Pattern to Service Class Template to Class Instances
December 12, 2020 Enterprise Digital Transformation Platform
33. Demo 2c: Examples of Patterns
EDTP provides templates for many Patterns including:
33
Synchronous Pattern Asynchronous Pattern Asynchronous Pattern with Retries
December 12, 2020 Enterprise Digital Transformation Platform
34. Demo 2c: Instantiating Service Instances Demo
We demonstrate how to instantiate new Service from Asynchronous Pattern
with Retries
34
Asynchronous Pattern with Retries EDTP Service Class Template
Generic Schema
Demo will create
A Service Instance
Generic schema
includes the required
fields, such as:
• TenantID,
• Version #,
• CorelationID.
• Section for Service
Data.
December 12, 2020 Enterprise Digital Transformation Platform
40. Next Step: Request Demonstration
Email Request to: ssipcic@gmail.com
40
December 12, 2020 Enterprise Digital Transformation Platform
41. Request Demonstration
The following EDTP features can be demonstrate on request:
1. Automated Deployment of EDTP in Client’s AWS environment including:
a. Infrastructure,
b. Technology Components,
c. Services.
2. EDTP Data and Integration Capabilities including:
a. Ingestion of streaming and batch data,
b. Harmonization and Materialization of the streaming data,
c. Integration based on the EDTP Connector Services – such integration with Snowflake,
d. Service templating for accelerated deployment of enterprise services.
3. EDTP Support for Pattern Based Service Development including:
a. Event Notification,
b. Event-Carried State Transfer,
c. Event-Sourcing,
d. CQRS,
e. Asynchronous Processing,
f. Service Retries Processing,
g. and many more…
41
December 12, 2020 Enterprise Digital Transformation Platform