It can help you do much more. You can use DMS to consolidate multiple databases into a single database or split a single database into multiple databases. You can also use DMS for data distribution to multiple systems. For both of these use cases your source database can be outside of AWS (on premises) or in AWS (EC2 or RDS). DMS can also be used for near real-time replication of data. Replication can be done to one or more targets within AWS, in the same region or across regions. You can also replicate data from databases within AWS to databases outside of AWS. In this session we will discuss all these usage patterns and help you try them out yourselves.
Prerequisites:
You should have good database knowledge and at least some experience with Amazon RDS or Amazon Aurora.
Participants should have an AWS account established and available for use during the workshop.
Please bring your own laptop.
AWS re:Invent 2016: Simplify Cloud Migration with AWS Server Migration Servic...
Similar a AWS re:Invent 2016: Workshop: Using the Database Migration Service (DMS) for Database Consolidation, Data Distribution and Replication (DAT321)
Similar a AWS re:Invent 2016: Workshop: Using the Database Migration Service (DMS) for Database Consolidation, Data Distribution and Replication (DAT321) (20)
6. • Quickly provision databases
• Multiple Availability Zones
• Rapid scaling
• Automated patching
• Easy read replica creation
• High durability
• Point in time recovery
• Detailed metrics
• Single-click encryption at rest
Amazon
RDS
Why AWS?
7. • How will my on-premises data migrate to the cloud?
• How can I make it transparent to my users?
• How will on-premises and cloud data interact?
• How can I integrate my data assets within AWS?
• How can I move off of commercial databases?
How?
8. Migration Options
• Lift and shift
• Leverage Amazon EC2 and Amazon S3
• Keep existing DB engine but migrate to Amazon RDS
• For example Oracle on-premises to RDS Oracle
• Migrate database engine
• Commercial engine to open source
• Maintenance window
• Maintenance window duration vs. CDC with 0 downtime
12. AWS Schema Conversion Tool
Features
• Converts schema of one database engine to another
• Database Migration Assessment report for choosing the best target engine
• Code browser that highlights places where manual edits are required
The AWS Schema Conversion Tool helps
automate many database schema and code
conversion tasks when migrating from Oracle
and SQL Server to open source database
engines.
14. Components of the Console
1. Source Schema
2. Action Items
3. Target Schema
4. Schema Element Details
5. Edit Window
15. Database Migration Assessment
1. Connect Schema
Conversion Tool to
source and target
databases.
2. Run Assessment
Report.
3. Read Executive
Summary.
4. Follow detailed
instructions.
16. Supported Conversions
Source Database Target Database
Microsoft SQL Server Amazon Aurora, Microsoft SQL Server, MySQL, PostgreSQL
MySQL MySQL, PostgreSQL
Oracle Amazon Aurora, MySQL, Oracle, PostgreSQL
Oracle Data Warehouse Amazon Redshift
PostgreSQL Amazon Aurora, MySQL, PostgreSQL
Teradata Amazon Redshift
17. $0
for software license
Allowed Use
Use Schema Conversion Tool to migrate database
schemas to Amazon RDS, Amazon Redshift, or Amazon
EC2–based databases
To use Schema Conversion Tool to migrate schemas to
other destinations, contact for special pricing
Pricing
Free software license
For active AWS customers with
accounts in good standing
Pricing, Terms & Conditions
29. • Start your first migration in 10 minutes or less
• Keep your apps running during the migration
• Replicate within, to, or from Amazon EC2 or RDS
• Move data to the same or a different database engine
AWS
Database Migration
Service
(AWS DMS)
30. Customer
premises
Application users
AWS
Internet
VPN
• Start a replication instance
• Connect to source and target
databases
• Select tables, schemas, or
databases
Let AWS DMS create tables,
load data, and keep them in
sync
Switch applications over to
the target at your convenience
Keep your apps running during the migration
AWS
DMS
31. Multi-AZ option for high availability
Customer
premises
or AWS
AWS
Internet
VPN
AWS DMS
AWS DMS
32. AWS Database Migration service pricing
T2 for developing and periodic data migration tasks
C4 for large databases and minimizing time
T2 pricing starts at $0.018 per hour for T2.micro
C4 pricing starts at $0.154 per hour for C4.large
50 GB GP2 storage included with T2 instances
100 GB GP2 storage included with C4 instances
Data transfer inbound and within AZ is free
Data transfer across AZs starts at $0.01 per GB
34. On-Premises Migration Scenarios
• An on-premises database to a database on Amazon RDS DB
instance
• An on-premises database to a database on an Amazon EC2
instance
• Migration from an on-premises database to another on-premises
database is not supported.
35. RDS Migration Scenarios
• A database on an Amazon RDS DB instance to an on-premises
database
• A database on an Amazon RDS DB instance to a database on an
Amazon RDS DB instance
• A database on an Amazon RDS DB instance to a database on an
Amazon EC2 instance
36. EC2 Migration Scenarios
• A database on an Amazon EC2 instance to an on-premises
database
• A database on an Amazon EC2 instance to a database on an
Amazon EC2 instance
• A database on an Amazon EC2 instance to a database on an
Amazon RDS DB instance
38. Replication Instances
• Performs the work of the migration
• Tasks run on instances
• Can support multiple tasks
• AWS DMS currently supports T2 and C4 instance classes for
replication instances
39. Public and Private Replication Instances
• A replication instance should have a public IP address if the source
or target database is located in a network that is not connected to
the replication instance's VPC by using a virtual private network
(VPN), AWS Direct Connect, or VPC peering.
• A replication instance should have a private IP address when both
the source and target databases are located in the same network
that is connected to the replication instance's VPC by using a VPN,
AWS Direct Connect, or VPC peer.
40. Sources for AWS Database Migration Service
Customers use the following databases as a source for data migration using AWS
DMS:
On-premises and Amazon EC2 instance databases:
• Oracle Database 10g – 12c
• Microsoft SQL Server 2005 – 2014
• MySQL 5.5 – 5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.4 – 9.5
• SAP ASE 15.7+
RDS instance databases:
• Oracle Database 11g – 12c
• Microsoft SQL Server 2008R2 - 2014. CDC operations are not supported yet.
• MySQL versions 5.5 – 5.7
• MariaDB (MySQL-compatible data source)
• PostgreSQL 9.4 – 9.5
• Amazon Aurora (MySQL-compatible data source)
41. Targets for AWS Database Migration Service
Customers can use the following databases as a target for data replication using
AWS DMS:
On-premises and EC2 instance databases:
• Oracle Database 10g – 12c
• Microsoft SQL Server 2005 – 2014
• MySQL 5.5 – 5.7
• MariaDB (MySQL-compatible data target)
• PostgreSQL 9.3 – 9.5
• SAP ASE 15.7+
RDS instance databases:
• Oracle Database 11g – 12c
• Microsoft SQL Server 2008 R2 - 2014
• MySQL 5.5 – 5.7
• MariaDB (MySQL-compatible data target)
• PostgreSQL 9.3 – 9.5
• Amazon Aurora (MySQL-compatible data target)
• Amazon Redshift
42. Tasks Overview
• Run on a replication instance
• Contain two and only two endpoints (source and target)
• Different migration methods available
• Specify selection and/or transformation rules
• Can run multiple tasks
43. Migration Methods
• Migrate existing data
• Migrate existing data and replicate ongoing changes
• Replicate data changes only
44. DMS – Change Data Capture (CDC)
“No Touch” design
• Reads recovery log of source database
• Using the engine’s native change data capture API
• No agent required on the source
Some requirements
• Oracle: Supplemental logging required
• MySQL: Full image row level bin logging required
• SQL Server: Recovery model bulk logged or full
• Postgres: wal_level = logical; max_replication_slots >= 1; max_wal_Senders >=1;
wal_sender_timeout = 0
Changes captured and applied as units of single committed transactions
Activated when load starts
No changes are applied until load completes, then applied as soon as possible in near real-time