SlideShare una empresa de Scribd logo
1 de 12
GoodData Confidential. 2013 GoodData Corporation. All rights
reserved.
Platform Architecture Deep Dive - External
The GoodData Open Analytics Platform
Operating System Infrastructure: Virtualized OpenStack environment
Hardware Infrastructure: high performance, redundant, networking
Health &
Performance
Monitoring
Cloud Control
Center (C3)
Data
Sources
Cloud
Sources
Shared
Storage
Cloud Computing Fabric
BI Automation SDK
Significant GoodData IP
Open Source or Commercial Software
Managed & Operated by GoodData
Web
Server &
Proxy
Data
Backup
ADS
Agile Data
Warehouse
Service
Clustered
Columnar
Database
BDS
Data Mart
MICCache
Columnar
Database
LDM
Metadata
DIS
Data
Integration
Services
Visualization
Export
Services
XAE
Extensible
Analytics
Engine
< / >
API’s
Visualize
Analytic
Designer
Browser
Cloud
Connect
SQL Client
Automation
SDK
SQL
JDBC
WebDav
4
GoodData Confidential. 2014 GoodData Corporation. All rights
Operating System Infrastructure: Virtualized OpenStack environment
Hardware Infrastructure: high performance, redundant, networking
Health & Performance
Monitoring
PIPE, Splunk
Cloud Control Center (C3)
• Projects, users, services, their
associations, and access
• Spans multiple clusters
• Stored in MySQL Database
Shared Storage
• GlusterFS/NetApp
• Passes data between
stateless workers
• Data Input  WebDav/FTP
• Garbage collection
Cloud Computing Fabric
• Asynchronous task distribution  Can add workers (scale) at any time
• Workflow orchestration & dependencies  UI will wait for a Report Task to Run
• Load distribution  spread workers across any number of computers
Cloud Computing Fabric
Coordinate and do any type of work quickly and at scale
TECHNICAL DIFFERENTIATION
 Scale & Flexibility
 HOT-HOT-HOT Redundancy
BI Automation SDK Reusable Framework: DIS, ADS, XAE
Example: query, transformation execution, data exporter
etc. Interoperable - different programming languages
5
GoodData Confidential. 2014 GoodData Corporation. All rights
API & Proxy Framework
Deliver a fast, customizable, and secure experience
TECHNICAL DIFFERENTIATION
 100% of API’s are Public!
 REST = Easy Web Integration
 Synchronous & Asynchronous 
Responsive UI no longer how long queries take
Internal Services
Exporters, Connectors
Authenticate
Web Server & Proxy
REST
API’s
Browser
< / >
• Login
• Project
• User
• Data Model
• Data Upload
• Notifications
• Report
• Dashboard
• Email
• Metadata
• Mandatory
User Filters
Automation
SDK
Customer
Applications
Cloud
Connect
C3
Proxy
Firewall
Load Balancers
HTTP
Accelerator
Web
App
Servers
Operating System Infrastructure
Hardware Infrastructure
Automation
SDK
Cloud
Connect
C3
User Verified
via API
Shared Storage
Raw Data Input and
Staging Area from
External Sources
Cloud Computing
Fabric
< / >
API’s
ADS
Data
Sources
Cloud
Sources
WebDav
Data Integration Services (DIS)
My data lives everywhere – bring it together
Security
All custom code can only
access external API’s
User’
s
Script
Secure Container
Extract Load
Data
Normalization
Transform
DIFFERENTIATION
 Drag and Drop GUI
 Can embed custom
code in cloud
 Can source data
locally or cloud
 Data Normalization
 Garbage Collection
Data Marts
Columnar
Database
LDM
MICCache
7
GoodData Confidential. 2014 GoodData Corporation. All rights
Data Mart & Analytics Engine (XAE, MAQL)
Give me data access (queries) better than I could write myself
TECHNICAL DIFFERENTIATION
 Benefits of OLAP Cube but w/ High Performance
 MAQL is robust to PDM changes
Data Base
Vertica or Postgres
Physical Data Model (PDM) +
Stored Data
Auto-generate SQL
SELECT * FROM WHERE
Optimize Query
w + h = a
Algebraically Prove Query
2w + 2h = 2a
Simplified Query Language
No Knowledge of LDM needed - MAQL
What & How?
Metadata
Report Definitions, Dashboards, etc.
Cloud
Connect
Analytic
Designer
 No Run away Queries
 Simplified Analyst Data Access
Logical Data Model (LDM)
Multi-level Intelligent
Computing Cache
• Fast Access to both full & partial prior used
reports, metrics, sub-metrics, cross tabs, etc.
• No need to re-query slow DB to do recalc
Operating System Infrastructure
Hardware Infrastructure
Agile Data Warehouse Service (ADS)
Store Everything. Create a Data Transformation Architecture
Automation
SDK
Cloud
Connect
SQL Client
C3: User Verified
for both JDBC & API
Shared Storage
Raw storage, import,
& Vertica backup
Cloud Computing Fabric
BI Automation SDK
Clustered Columnar DB #1
Instance A
Instance B
Instance C
Clustered Columnar DB #2
Instance D
Customer X
Customer Y
Customer Z
TECHNICAL DIFFERENTIATION
 Only Multi-Tenant Vertica in market
 Provisioning via REST API
 SQL & JDBC w/ SSL cert. auth
 Isolation/security via SQL Validator
 Can Scale to Petabytes
< / >
API’s
SQL
JDBC
Output Data
Direct SQL
Commands &
Queries for
Data
Transformation
Data Marts
9
GoodData Confidential. 2014 GoodData Corporation. All rights
Shared Storage
NetApp/GlusterFS
Rackspace: Elk Grove, IL, USA
ISO 27001, SOC2
Rackspace: Europe (Roadmap)
Data Location & Redundancy
Where does my Data go and… Don’t Lose it!
TECHNICAL DIFFERENTIATION
 Fully Automated & Monitored
 Computing Fabric is HOT-HOT-HOT
AWS (S3)
Simple Storage Service
ISO 27001,SOC2
S3 US East Zone
Multiple Locations
EncryptedData
Sources
Cloud
Sources
WebDav
Browser
ADS DB’s
SQL Client
Data Marts
10
GoodData Confidential. 2014 GoodData Corporation. All rights
Security
Be more secure than I ever could be
TECHNICAL DIFFERENTIATION
 Single Entry/Exit point for all Customers 
Easier to Secure/Control vs. a whole Enterprise
 All data in/out through encrypted channels
 SSO for Platform, Customer’s Info
 Private Cloud
 Stateless Workers = no exe code
Shared Storage
Self Encrypting Disks
Rackspace: ISO 27001, SOC2
Rackspace: Europe (Roadmap)
Data Marts
Data
Sources
Cloud
Sources
WebDav
Browser
SQL Client
ADS DB’sJDBC
SSL
HTTPS
Encrypted
File
Systems
HTTPS
HTTPS
Cloud & Physical Security
• 24/7/365 security, phys. access control
• Proactively monitors all components
• Alerting chain all the way to the CEO
Data Security
• Security zones, protocol enforcement,
intrusion detection, penetration testing
• Data isolated/encrypt in transport/at rest
Operational Security
• Access is under strict control
• Admins allowed to invoke platform-
specific functions but not permitted to
interact with the platform components
• Admin Employee Background Checks
similar to US Top Secret
11
GoodData Confidential. 2014 GoodData Corporation. All rights
Infrastructure Health & Performance Monitoring
GoodData Architecture – Scalable Multi-Tenancy
GoodData
HTML5
Client
Cloud Connect
Modeler
Resource
Manager
HTTPS,
WebDav
Data
Connector
Framework
ETL Server
Processes
Hadoop Data
Expansion Clustered
Columnar
DB
Data
Integration
Services
Columnar
DB
JDBC to ADS BI Automation SDK REST APIs
Common Metadata: Sources, ADS, LDM, Marts
Data Warehouse Management
Data
Integration
Service
Console
Extensible
Analytics
Engine
Visualization
Export
Services:
Chart & PDF
Data Mart:
Metadata
LDM &
cache
Data Orchestration Services
Cloud Control Center: Real-time, Multi-tenant resource manager
including Tenants, Projects and Users across distributed environment
Shared Storage: Redundant staging & archiving of raw data & files
Operating System Infrastructure: Virtualized OpenStack environment
Hardware Infrastructure: Custom-tailored high performance, redundant environment and networking
Sources
Significant GoodData IP Integration Framework Open Source & Commercial Software
GoodData Computing Fabric
Business
User
Experience
Analyst
Discovery
Experience
IT
Control
Experience
Cloud
Sources
12
GoodData Confidential. 2014 GoodData Corporation. All rights
Data Pipeline

Más contenido relacionado

La actualidad más candente

Oracle Cloud infrastructure
Oracle Cloud infrastructureOracle Cloud infrastructure
Oracle Cloud infrastructureWajid Hassan
 
Oracle Database Migration to Oracle Cloud Infrastructure
Oracle Database Migration to Oracle Cloud InfrastructureOracle Database Migration to Oracle Cloud Infrastructure
Oracle Database Migration to Oracle Cloud InfrastructureSinanPetrusToma
 
Oracle database in cloud, dr in cloud and overview of oracle database 18c
Oracle database in cloud, dr in cloud and overview of oracle database 18cOracle database in cloud, dr in cloud and overview of oracle database 18c
Oracle database in cloud, dr in cloud and overview of oracle database 18cAiougVizagChapter
 
High Availability & Disaster Recovery on Oracle Cloud Infrastructure
High Availability & Disaster Recovery on Oracle Cloud InfrastructureHigh Availability & Disaster Recovery on Oracle Cloud Infrastructure
High Availability & Disaster Recovery on Oracle Cloud InfrastructureSinanPetrusToma
 
KoprowskiT_session1_SDNEvent_WASDforBeginners
KoprowskiT_session1_SDNEvent_WASDforBeginnersKoprowskiT_session1_SDNEvent_WASDforBeginners
KoprowskiT_session1_SDNEvent_WASDforBeginnersTobias Koprowski
 
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...Tammy Bednar
 
Implementing Security on a Large Multi-Tenant Cluster the Right Way
Implementing Security on a Large Multi-Tenant Cluster the Right WayImplementing Security on a Large Multi-Tenant Cluster the Right Way
Implementing Security on a Large Multi-Tenant Cluster the Right WayDataWorks Summit
 
DBA Tasks in Oracle Autonomous Database
DBA Tasks in Oracle Autonomous DatabaseDBA Tasks in Oracle Autonomous Database
DBA Tasks in Oracle Autonomous DatabaseSinanPetrusToma
 
Autonomous Database Security Features
Autonomous Database Security FeaturesAutonomous Database Security Features
Autonomous Database Security FeaturesSinanPetrusToma
 
Oracle Autonomous Data Warehouse Cloud Webex - with Demo
Oracle Autonomous Data Warehouse Cloud Webex - with DemoOracle Autonomous Data Warehouse Cloud Webex - with Demo
Oracle Autonomous Data Warehouse Cloud Webex - with DemoIslam Gohar
 
2019 - OOW - Database Migration Methods from On-Premise to Cloud
2019 - OOW - Database Migration Methods from On-Premise to Cloud2019 - OOW - Database Migration Methods from On-Premise to Cloud
2019 - OOW - Database Migration Methods from On-Premise to CloudMarcus Vinicius Miguel Pedro
 
6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure6 Ways to Get More From Your Azure
6 Ways to Get More From Your AzureHolly Plude
 
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade Compliance
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade ComplianceAvoid the Oracle SE2 Trap with EnterpriseDB & Palisade Compliance
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade ComplianceEDB
 
Mastering the move
Mastering the moveMastering the move
Mastering the moveTrivadis
 
Oracle database 12c_and_DevOps
Oracle database 12c_and_DevOpsOracle database 12c_and_DevOps
Oracle database 12c_and_DevOpsMaria Colgan
 

La actualidad más candente (20)

Oracle Cloud infrastructure
Oracle Cloud infrastructureOracle Cloud infrastructure
Oracle Cloud infrastructure
 
Oracle Database Migration to Oracle Cloud Infrastructure
Oracle Database Migration to Oracle Cloud InfrastructureOracle Database Migration to Oracle Cloud Infrastructure
Oracle Database Migration to Oracle Cloud Infrastructure
 
Azure IoT Suite
Azure IoT Suite Azure IoT Suite
Azure IoT Suite
 
Oracle database in cloud, dr in cloud and overview of oracle database 18c
Oracle database in cloud, dr in cloud and overview of oracle database 18cOracle database in cloud, dr in cloud and overview of oracle database 18c
Oracle database in cloud, dr in cloud and overview of oracle database 18c
 
High Availability & Disaster Recovery on Oracle Cloud Infrastructure
High Availability & Disaster Recovery on Oracle Cloud InfrastructureHigh Availability & Disaster Recovery on Oracle Cloud Infrastructure
High Availability & Disaster Recovery on Oracle Cloud Infrastructure
 
KoprowskiT_session1_SDNEvent_WASDforBeginners
KoprowskiT_session1_SDNEvent_WASDforBeginnersKoprowskiT_session1_SDNEvent_WASDforBeginners
KoprowskiT_session1_SDNEvent_WASDforBeginners
 
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...
Database@Home : Data Driven Apps - Data-driven Microservices Architecture wit...
 
Implementing Security on a Large Multi-Tenant Cluster the Right Way
Implementing Security on a Large Multi-Tenant Cluster the Right WayImplementing Security on a Large Multi-Tenant Cluster the Right Way
Implementing Security on a Large Multi-Tenant Cluster the Right Way
 
DBA Tasks in Oracle Autonomous Database
DBA Tasks in Oracle Autonomous DatabaseDBA Tasks in Oracle Autonomous Database
DBA Tasks in Oracle Autonomous Database
 
Oracle ADWC
Oracle ADWCOracle ADWC
Oracle ADWC
 
Pci multitenancy exalogic at AMIS25
Pci multitenancy exalogic at AMIS25Pci multitenancy exalogic at AMIS25
Pci multitenancy exalogic at AMIS25
 
Autonomous Database Security Features
Autonomous Database Security FeaturesAutonomous Database Security Features
Autonomous Database Security Features
 
Exadata Cloud Service Overview(v2)
Exadata Cloud Service Overview(v2) Exadata Cloud Service Overview(v2)
Exadata Cloud Service Overview(v2)
 
Oracle Autonomous Data Warehouse Cloud Webex - with Demo
Oracle Autonomous Data Warehouse Cloud Webex - with DemoOracle Autonomous Data Warehouse Cloud Webex - with Demo
Oracle Autonomous Data Warehouse Cloud Webex - with Demo
 
2019 - OOW - Database Migration Methods from On-Premise to Cloud
2019 - OOW - Database Migration Methods from On-Premise to Cloud2019 - OOW - Database Migration Methods from On-Premise to Cloud
2019 - OOW - Database Migration Methods from On-Premise to Cloud
 
6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure
 
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade Compliance
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade ComplianceAvoid the Oracle SE2 Trap with EnterpriseDB & Palisade Compliance
Avoid the Oracle SE2 Trap with EnterpriseDB & Palisade Compliance
 
Mastering the move
Mastering the moveMastering the move
Mastering the move
 
Oracle database 12c_and_DevOps
Oracle database 12c_and_DevOpsOracle database 12c_and_DevOps
Oracle database 12c_and_DevOps
 
Oracle Cloud
Oracle CloudOracle Cloud
Oracle Cloud
 

Similar a Platform Deep Dive

Connect your datacenter to Microsoft Azure
Connect your datacenter to Microsoft AzureConnect your datacenter to Microsoft Azure
Connect your datacenter to Microsoft AzureK.Mohamed Faizal
 
Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeKent Graziano
 
Get started with Cloudera's cyber solution
Get started with Cloudera's cyber solutionGet started with Cloudera's cyber solution
Get started with Cloudera's cyber solutionCloudera, Inc.
 
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft CloudEuropean Collaboration Summit
 
Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019George Walters
 
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdfData & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdfChris Bingham
 
SQL Server Ground to Cloud.pptx
SQL Server Ground to          Cloud.pptxSQL Server Ground to          Cloud.pptx
SQL Server Ground to Cloud.pptxsaidbilgen
 
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Codit
 
How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsHow to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsInformatica
 
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa Toroman
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa ToromanO365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa Toroman
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa ToromanNCCOMMS
 
Azure Community Tour 2019 - AZUGDK
Azure Community Tour 2019 - AZUGDKAzure Community Tour 2019 - AZUGDK
Azure Community Tour 2019 - AZUGDKPeter Selch Dahl
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudMark Kromer
 
Azure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar PresentationAzure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar PresentationMatthew W. Bowers
 
KoprowskiT_SQLSatMoscow_WASDforBeginners
KoprowskiT_SQLSatMoscow_WASDforBeginnersKoprowskiT_SQLSatMoscow_WASDforBeginners
KoprowskiT_SQLSatMoscow_WASDforBeginnersTobias Koprowski
 
Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshJeffrey T. Pollock
 
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...Amazon Web Services
 

Similar a Platform Deep Dive (20)

Connect your datacenter to Microsoft Azure
Connect your datacenter to Microsoft AzureConnect your datacenter to Microsoft Azure
Connect your datacenter to Microsoft Azure
 
Delivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with SnowflakeDelivering Data Democratization in the Cloud with Snowflake
Delivering Data Democratization in the Cloud with Snowflake
 
Get started with Cloudera's cyber solution
Get started with Cloudera's cyber solutionGet started with Cloudera's cyber solution
Get started with Cloudera's cyber solution
 
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud
[Toroman/Kranjac] Red Team vs. Blue Team in Microsoft Cloud
 
6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure
 
6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure6 Ways to Get More From Your Azure
6 Ways to Get More From Your Azure
 
Vue d'ensemble Dremio
Vue d'ensemble DremioVue d'ensemble Dremio
Vue d'ensemble Dremio
 
Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019Customer migration to Azure SQL database, December 2019
Customer migration to Azure SQL database, December 2019
 
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdfData & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
Data & Analytics ReInvent Recap [AWS Basel Meetup - Jan 2023].pdf
 
SQL Server Ground to Cloud.pptx
SQL Server Ground to          Cloud.pptxSQL Server Ground to          Cloud.pptx
SQL Server Ground to Cloud.pptx
 
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017) Integration of Things (Sam Vanhoutte @Iglooconf 2017)
Integration of Things (Sam Vanhoutte @Iglooconf 2017)
 
How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data AnalyticsHow to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
How to Architect a Serverless Cloud Data Lake for Enhanced Data Analytics
 
Ibm db2update2019 icp4 data
Ibm db2update2019   icp4 dataIbm db2update2019   icp4 data
Ibm db2update2019 icp4 data
 
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa Toroman
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa ToromanO365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa Toroman
O365Con18 - Red Team vs Blue Team - Sasha Kranjac & Mustafa Toroman
 
Azure Community Tour 2019 - AZUGDK
Azure Community Tour 2019 - AZUGDKAzure Community Tour 2019 - AZUGDK
Azure Community Tour 2019 - AZUGDK
 
Azure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the CloudAzure Data Factory ETL Patterns in the Cloud
Azure Data Factory ETL Patterns in the Cloud
 
Azure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar PresentationAzure Synapse 101 Webinar Presentation
Azure Synapse 101 Webinar Presentation
 
KoprowskiT_SQLSatMoscow_WASDforBeginners
KoprowskiT_SQLSatMoscow_WASDforBeginnersKoprowskiT_SQLSatMoscow_WASDforBeginners
KoprowskiT_SQLSatMoscow_WASDforBeginners
 
Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to Mesh
 
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...
Expedite the development lifecycle with MongoDB and serverless - DEM17-S - Ne...
 

Último

Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Alkin Tezuysal
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Mark Goldstein
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxLoriGlavin3
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsNathaniel Shimoni
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentPim van der Noll
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditSkynet Technologies
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...AliaaTarek5
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersRaghuram Pandurangan
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersNicole Novielli
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityIES VE
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Farhan Tariq
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality AssuranceInflectra
 

Último (20)

Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
Unleashing Real-time Insights with ClickHouse_ Navigating the Landscape in 20...
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
Arizona Broadband Policy Past, Present, and Future Presentation 3/25/24
 
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptxA Deep Dive on Passkeys: FIDO Paris Seminar.pptx
A Deep Dive on Passkeys: FIDO Paris Seminar.pptx
 
Time Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directionsTime Series Foundation Models - current state and future directions
Time Series Foundation Models - current state and future directions
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native developmentEmixa Mendix Meetup 11 April 2024 about Mendix Native development
Emixa Mendix Meetup 11 April 2024 about Mendix Native development
 
Manual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance AuditManual 508 Accessibility Compliance Audit
Manual 508 Accessibility Compliance Audit
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
(How to Program) Paul Deitel, Harvey Deitel-Java How to Program, Early Object...
 
Generative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information DevelopersGenerative AI for Technical Writer or Information Developers
Generative AI for Technical Writer or Information Developers
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
A Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software DevelopersA Journey Into the Emotions of Software Developers
A Journey Into the Emotions of Software Developers
 
Decarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a realityDecarbonising Buildings: Making a net-zero built environment a reality
Decarbonising Buildings: Making a net-zero built environment a reality
 
Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...Genislab builds better products and faster go-to-market with Lean project man...
Genislab builds better products and faster go-to-market with Lean project man...
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance[Webinar] SpiraTest - Setting New Standards in Quality Assurance
[Webinar] SpiraTest - Setting New Standards in Quality Assurance
 

Platform Deep Dive

  • 1. GoodData Confidential. 2013 GoodData Corporation. All rights reserved. Platform Architecture Deep Dive - External
  • 2. The GoodData Open Analytics Platform
  • 3. Operating System Infrastructure: Virtualized OpenStack environment Hardware Infrastructure: high performance, redundant, networking Health & Performance Monitoring Cloud Control Center (C3) Data Sources Cloud Sources Shared Storage Cloud Computing Fabric BI Automation SDK Significant GoodData IP Open Source or Commercial Software Managed & Operated by GoodData Web Server & Proxy Data Backup ADS Agile Data Warehouse Service Clustered Columnar Database BDS Data Mart MICCache Columnar Database LDM Metadata DIS Data Integration Services Visualization Export Services XAE Extensible Analytics Engine < / > API’s Visualize Analytic Designer Browser Cloud Connect SQL Client Automation SDK SQL JDBC WebDav
  • 4. 4 GoodData Confidential. 2014 GoodData Corporation. All rights Operating System Infrastructure: Virtualized OpenStack environment Hardware Infrastructure: high performance, redundant, networking Health & Performance Monitoring PIPE, Splunk Cloud Control Center (C3) • Projects, users, services, their associations, and access • Spans multiple clusters • Stored in MySQL Database Shared Storage • GlusterFS/NetApp • Passes data between stateless workers • Data Input  WebDav/FTP • Garbage collection Cloud Computing Fabric • Asynchronous task distribution  Can add workers (scale) at any time • Workflow orchestration & dependencies  UI will wait for a Report Task to Run • Load distribution  spread workers across any number of computers Cloud Computing Fabric Coordinate and do any type of work quickly and at scale TECHNICAL DIFFERENTIATION  Scale & Flexibility  HOT-HOT-HOT Redundancy BI Automation SDK Reusable Framework: DIS, ADS, XAE Example: query, transformation execution, data exporter etc. Interoperable - different programming languages
  • 5. 5 GoodData Confidential. 2014 GoodData Corporation. All rights API & Proxy Framework Deliver a fast, customizable, and secure experience TECHNICAL DIFFERENTIATION  100% of API’s are Public!  REST = Easy Web Integration  Synchronous & Asynchronous  Responsive UI no longer how long queries take Internal Services Exporters, Connectors Authenticate Web Server & Proxy REST API’s Browser < / > • Login • Project • User • Data Model • Data Upload • Notifications • Report • Dashboard • Email • Metadata • Mandatory User Filters Automation SDK Customer Applications Cloud Connect C3 Proxy Firewall Load Balancers HTTP Accelerator Web App Servers
  • 6. Operating System Infrastructure Hardware Infrastructure Automation SDK Cloud Connect C3 User Verified via API Shared Storage Raw Data Input and Staging Area from External Sources Cloud Computing Fabric < / > API’s ADS Data Sources Cloud Sources WebDav Data Integration Services (DIS) My data lives everywhere – bring it together Security All custom code can only access external API’s User’ s Script Secure Container Extract Load Data Normalization Transform DIFFERENTIATION  Drag and Drop GUI  Can embed custom code in cloud  Can source data locally or cloud  Data Normalization  Garbage Collection Data Marts Columnar Database LDM MICCache
  • 7. 7 GoodData Confidential. 2014 GoodData Corporation. All rights Data Mart & Analytics Engine (XAE, MAQL) Give me data access (queries) better than I could write myself TECHNICAL DIFFERENTIATION  Benefits of OLAP Cube but w/ High Performance  MAQL is robust to PDM changes Data Base Vertica or Postgres Physical Data Model (PDM) + Stored Data Auto-generate SQL SELECT * FROM WHERE Optimize Query w + h = a Algebraically Prove Query 2w + 2h = 2a Simplified Query Language No Knowledge of LDM needed - MAQL What & How? Metadata Report Definitions, Dashboards, etc. Cloud Connect Analytic Designer  No Run away Queries  Simplified Analyst Data Access Logical Data Model (LDM) Multi-level Intelligent Computing Cache • Fast Access to both full & partial prior used reports, metrics, sub-metrics, cross tabs, etc. • No need to re-query slow DB to do recalc
  • 8. Operating System Infrastructure Hardware Infrastructure Agile Data Warehouse Service (ADS) Store Everything. Create a Data Transformation Architecture Automation SDK Cloud Connect SQL Client C3: User Verified for both JDBC & API Shared Storage Raw storage, import, & Vertica backup Cloud Computing Fabric BI Automation SDK Clustered Columnar DB #1 Instance A Instance B Instance C Clustered Columnar DB #2 Instance D Customer X Customer Y Customer Z TECHNICAL DIFFERENTIATION  Only Multi-Tenant Vertica in market  Provisioning via REST API  SQL & JDBC w/ SSL cert. auth  Isolation/security via SQL Validator  Can Scale to Petabytes < / > API’s SQL JDBC Output Data Direct SQL Commands & Queries for Data Transformation Data Marts
  • 9. 9 GoodData Confidential. 2014 GoodData Corporation. All rights Shared Storage NetApp/GlusterFS Rackspace: Elk Grove, IL, USA ISO 27001, SOC2 Rackspace: Europe (Roadmap) Data Location & Redundancy Where does my Data go and… Don’t Lose it! TECHNICAL DIFFERENTIATION  Fully Automated & Monitored  Computing Fabric is HOT-HOT-HOT AWS (S3) Simple Storage Service ISO 27001,SOC2 S3 US East Zone Multiple Locations EncryptedData Sources Cloud Sources WebDav Browser ADS DB’s SQL Client Data Marts
  • 10. 10 GoodData Confidential. 2014 GoodData Corporation. All rights Security Be more secure than I ever could be TECHNICAL DIFFERENTIATION  Single Entry/Exit point for all Customers  Easier to Secure/Control vs. a whole Enterprise  All data in/out through encrypted channels  SSO for Platform, Customer’s Info  Private Cloud  Stateless Workers = no exe code Shared Storage Self Encrypting Disks Rackspace: ISO 27001, SOC2 Rackspace: Europe (Roadmap) Data Marts Data Sources Cloud Sources WebDav Browser SQL Client ADS DB’sJDBC SSL HTTPS Encrypted File Systems HTTPS HTTPS Cloud & Physical Security • 24/7/365 security, phys. access control • Proactively monitors all components • Alerting chain all the way to the CEO Data Security • Security zones, protocol enforcement, intrusion detection, penetration testing • Data isolated/encrypt in transport/at rest Operational Security • Access is under strict control • Admins allowed to invoke platform- specific functions but not permitted to interact with the platform components • Admin Employee Background Checks similar to US Top Secret
  • 11. 11 GoodData Confidential. 2014 GoodData Corporation. All rights Infrastructure Health & Performance Monitoring GoodData Architecture – Scalable Multi-Tenancy GoodData HTML5 Client Cloud Connect Modeler Resource Manager HTTPS, WebDav Data Connector Framework ETL Server Processes Hadoop Data Expansion Clustered Columnar DB Data Integration Services Columnar DB JDBC to ADS BI Automation SDK REST APIs Common Metadata: Sources, ADS, LDM, Marts Data Warehouse Management Data Integration Service Console Extensible Analytics Engine Visualization Export Services: Chart & PDF Data Mart: Metadata LDM & cache Data Orchestration Services Cloud Control Center: Real-time, Multi-tenant resource manager including Tenants, Projects and Users across distributed environment Shared Storage: Redundant staging & archiving of raw data & files Operating System Infrastructure: Virtualized OpenStack environment Hardware Infrastructure: Custom-tailored high performance, redundant environment and networking Sources Significant GoodData IP Integration Framework Open Source & Commercial Software GoodData Computing Fabric Business User Experience Analyst Discovery Experience IT Control Experience Cloud Sources
  • 12. 12 GoodData Confidential. 2014 GoodData Corporation. All rights Data Pipeline