Más contenido relacionado La actualidad más candente La actualidad más candente (20) Similar a Oracle GoldenGate and ODI Integration Overview Similar a Oracle GoldenGate and ODI Integration Overview (20) Más de Dr. Wilfred Lin (Ph.D.) Más de Dr. Wilfred Lin (Ph.D.) (20) Oracle GoldenGate and ODI Integration Overview2. Data Integration Trends
Data Integration Tools Today Need to Do More
Big Data is the new normal
Real-time data jumps to the forefront
Application data now lives in the Cloud and
in a multitude of disparate environments
Conventional ETL tools are struggling to
meet performance demands
2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
3. New Requirements for Data Integration
Integrated and
High Productivity
Tooling
Any Data Latency,
Real-time
Any Data,
Any Source
3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
High Performance,
High Availability
Cloud Ready
4. Oracle Data Integration 12c
Delivering Real-Time Integration for Big Data and Cloud
Cloud
Oracle Data Integrator
Apps
Oracle GoldenGate
Database
Oracle Enterprise
Data Quality
Big Data
Oracle Data
Service Integrator
Future Ready
Extreme Performance
4
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Fast Time to Value
5. New Features in Data Integrator 12c
Improved developer productivity through
declarative flow based mapping
Extreme performance through parallelism
and low session overheads
Enterprise integration through big data and
business analytics capabilities
5
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
6. Enhanced Experience for Improved Productivity
Flow Based Modeling
Declarative, flow based mapping to easily
model complex transformation logic combined
with technology specific out of the box
knowledge modules
Component re-usability to capture shared
logic throughout mappings (sources, targets
and transformations)
Step–by–step debugger with complete user
control / access to uncommitted data, and
defining break points
6
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
7. Performance Optimizations
Faster Loads for Improved Run-time Caching
TRADITIONAL
In-session parallelism
SrcSet1
SrcSet2
Integration
– Parallel task execution
– Narrower execution windows
SrcSet3
Total Execution Time
– When executing repeated scenarios, restarting
ODI 12.1.2
sessions
SrcSet1
SrcSet2
Integration
SrcSet3
Total Execution Time
7
Low overhead session during execution
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
– Efficient micro-batch loading for real time data
– Reduced logging, better cache utilization reduce
network traffic for entire IT system
8. Integration Between GoldenGate and ODI 12c
Enable GoldenGate journalizing on
interfaces without restructuring.
Manage GoldenGate Capture/Delivery
instances in ODI Topology
Set common GoldenGate parameters
graphically
Bulk Load Scenario
Capture
8
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Real Time Scenario
Delivery
Deploy GoldenGate processes via ODI
Support real-time and bulk from the
same interface
9. Enterprise Manager Cloud Control Integration
Overview
Discovery of ODI components
– Agents: Standalone and JEE
– Repositories
– Source and Target databases
Monitoring
– Availability Monitoring
ODI homepage to quickly assess ODI infrastructure status
Graphical representation of ODI infrastructure
– Performance Monitoring
ODI components performance
Scenarios, Load Plans, Sessions execution statistics
Historical metrics
9
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
10. Enterprise Manager Cloud Control Integration
Other Features
Performance Monitoring
– SLA Management
Alerts creation
– SQL Drill Down
Navigate from ODI session to Database session statistics (leveraging DB EMGC
Diagnostics and Tuning Packs)
10
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
11. Loading and Transforming Big Data
Access
Transform
Supports Hadoop standards
Reverse engineer Hadoop
metadata
Check, validate and ensure
Oracle Data
Integrator
data integrity with Hadoop
Loads
Load data into HDFS/Hive
Generate HiveQL and execute
in Hadoop
Leverage existing Hadoop
transformations
11
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
12. Oracle Warehouse Builder Integration
Overview
Goal
Provide a migration path to OWB users
Integrate OWB and ODI runtime for
phased migration
How
OWB repositories can now be managed
in an ODI topology
OWB jobs can be executed and
monitored in ODI
OWB to ODI Migration Utility
12
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
13. Key New Features in GoldenGate 12c
Advancing Oracle’s Leadership in Data Replication
Intelligent and Integrated Delivery for
higher performance and IT productivity
Optimization for Database 12c
Expanded heterogeneity and solutions for
Oracle Applications
13
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
14. Current Approach to High Volume Replication
Customer
defines
split
SQL
Delivery
Delivery
SQL
Trail
Files
SQL
Delivery
SQL
Delivery
14
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
15. Intelligent & Integrated Delivery for Oracle
Capture
Trail
Files
Delivery
Lightweight
Streaming
API
Oracle Database
Unique, next generation, high-speed data apply for Oracle Database
Leverages the Oracle Database parallel apply servers for auto dependency-
aware parallel delivery
Simplifies deployment while doubling performance
15
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
16. Coordinated Delivery for Non-Oracle
Single replication
process for customer
Delivery
SQL
Thread
Capture
Trail
Files
Thread
Thread
Customer
Defines
Split
SQL
SQL
Non- Oracle Databases
GoldenGate coordinates threads to
maintain ordering
Orchestrates multiple apply threads
Simplifies configuration and increases performance effortlessly
16
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
17. Optimization for Oracle Database 12c
Oracle 12c Container Database
Oracle GoldenGate
Oracle GoldenGate
Multitenant
Supports traditional non-consolidated
databases and the new multitenant
architecture, Database12c specific
features and data types
Enables enterprises to move to private
cloud without downtime and meet
availability SLAs
Reduces overhead by capturing changed
data at the container level
Enhanced security for credentials and
encryption keys
Integrated with Data Guard and FSFO for
Oracle & Non-Oracle Database(s)
17
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
automated failover and high availability
18. Expanded Heterogeneity
Capture & Delivery
Oracle
IBM DB2 LUW
IBM DB2 z Series
IBM DB2 i Series /AS400
Microsoft SQL Server
Sybase ASE
Teradata
C-Tree
Enscribe
SQL/MP
SQL/MX
MySQL
JMS Message Queues
18
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Delivery Only
Netezza
New in Oracle GoldenGate 12c
PostgreSQL
•
•
•
GreenPlum
-------------------------------------------------------
HP Neoview
•
•
•
TimesTen
Flat File
ETL products
MySQL 5.6
MySQL Cluster 7.1 and 7.2
Sybase 15.7
DB2 LUW 10.1
Teradata 14.10
SQL Server 2012
Coherence
Platform Specific Upgrades
• SQL Server Compressed Back-Up
• MySQL Clusters (NDB)
19. Integration with Oracle Coherence
Oracle Coherence Grid Edition 12.1.2
Tight Integration with Oracle Coherence
Oracle TopLink
enables real-time updates to Coherence
cache
Coherence
Adapter
Refreshes invalidated object in the
Capture
Oracle & Non-Oracle Database(s)
19
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Trail
Files
Coherence cache when the database is
directly modified
Coherence users can access real-time
data without any changes required to the
source system
20. New Certified Oracle Application Solutions
Database Switchover
Real-Time Change Data Feeds via GoldenGate
Capture
Source
E-Business
Database
Trail
Files
Trail
Files
LAN / WAN / Internet Over TCP/IP
Oracle to Oracle Initial Data Load
Low-downtime Oracle E-Business Suite database migration
ATG Web Commerce active-active data center synchronization
20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Delivery
Target
E-Business
Database
21. Emerging Capabilities in Oracle GoldenGate
Management, Monitoring, Adapters
Advanced Monitoring
Veridata: Heterogeneous database
comparison and repair
Integration with Hadoop
Cloud friendly replication
21
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
22. Management Pack for Oracle GoldenGate
What’s Coming
Enhanced Functionality for
Enterprise Manager Plug-in
Oracle preferred monitoring
tool, covers entire Oracle stack
– database, storage, FMW etc
Extends command and control
functions; start and stopping of
processes
22
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
23. Oracle GoldenGate Veridata
What’s Coming
Oracle Weblogic Server
integration
Streamlined Navigation
Compare
Compare
Compare Group XML Import
Repair
Repair Out of Sync
Data
23
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
24. GoldenGate for Integration with Hadoop
Enabling Coexistence of Relational Data with Big Data Systems
Stream transactional data into
Hadoop without source impact
and in real time
Capture
Oracle &
Non-Oracle
Database(s)
24
Trail
Files
Pump
Pump
Parameter
File
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Custom
Adapter
Adapter
Properties
File
Implement custom adapter or
handler for the big data system
using Oracle GoldenGate's Java
Adapter
Full control of data delivery to
any Java exposed Big Data
technology
25. Enterprise-Class Data Integration with Oracle
Multiple Use-Cases for Heterogeneous Systems
Real-Time Data
Warehousing, Siebel
CRM Upgrades, Zero
Downtime Database
Migrations
Real-Time Data from
Heterogeneous
Sources to ODS
Supporting
Application Services
Real-Time Data
Feed from SQL
Server Sources to
Teradata DW for
Finance & Marketing
analysis
Data integration from
Heterogeneous Sources
for SOA and BAM
25
100% Faster Speed
into Enterprise Data
Warehouse on Oracle
Exadata
Operational Reporting
for E-Business Suite
and Data Services for
SOA
Real-Time Data
Warehousing and Hot
Standby for Disaster
Recovery
Data Warehousing
for Operations from
Restocking to
Marketing
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
26. Ameristar Casinos
Real-Time Heterogeneous Data Replication for Data Warehousing
Real-Time Visibility for Customer Service
Real-time data integration from 14
gaming and promotional management
systems on SQL Server in 8 properties
Delivers to Teradata DW for fast
analysis of financial data and customer
promotions for improved customer
experience
Reduced operational costs due to ease
of deployment and management
26
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
27. StarWood Hotels & Resorts
Near Real-Time Reporting on Oracle Exadata
Timely Insight , Globally
Real-time data for revenue, central
reservations, and rate plan reports running
on Oracle Exadata
100% improvement in data loading speed,
288X reduction in latency, down to 5 min.
Access to up-to-date data globally
Uses timely data for increasing efficiency
and improving customer service via
visibility into room occupancy
27
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
28. Data Integration on Engineered Systems
Oracle GoldenGate is the best way to
migrate data to Exadata and Exalytics, for
zero downtime
Oracle Data Integrator transformations run
natively and use Exadata optimized SQL
Oracle Enterprise Data Quality leverages
both Exalogic and Exadata extreme deduplication performance
Simplifies operations for big data and
analytics, while maximizing performance
28
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
30. Join the Data Integration Community
Twitter
twitter.com/ORCLGoldenGate
LinkedIn
“Oracle Data Integration”
Facebook
facebook.com/OracleDataIntegration
Oracle’s Data Integration blog
blogs.oracle.com/dataintegration
Oracle GoldenGate YouTube channel
youtube.com/oraclegoldengate
Oracle
Technolog
y Network
Oracle’s Public Help Forum
http://forums.oracle.com/forums/forum.j
spa?forumID=860&start=0
Oracle.com/goto/dataintegration
30
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.