4. Need of Database migration
Database
Application
Development
setup
Database
Application
QA setup
Database
Application
Integration setup
Database
Application
Production setup
Different environments
Real world scenario
5. Comparison between source code
and database
Source code
•Version control tools are
available
•Maven, Jenkins, Hudson
reproducible builds and
continuous integration
•Maven release plug-ins for
tagging and for newer versions
Database
•Manually applying
database scripts
•What is current state of
database?
•Which scripts are applied?
•If quick fix is applied on
production after dev
machine?
•How to setup new DB
instance?
6. Solution is Database migrations
Allows to recreate a database
from scratch / update from
any previously migrated state
Make it clear at all times
what state a database is in
Migrate in a deterministic
way from your current
version of the database to a
newer one
8. How Flyway works
SCHEMA_VERSION
Flyway first creates single empty table
SCHEMA_VERSION by default:
This table will be used for tracking state of
database.
Once SCHEMA_VERSION is created,
Flyway will start scanning classpath for
migrations.
Migrations are sorted based on version
number and applied in order
10. API- Flyway can be configured directly into application.
Flyway checks database version and before actual
application starts, applies newer migrations
Command-line tool- CLI is for users:
Who wish to migration their database from the command-
line without having to install Maven or Ant
Flyway comes with plugin for
Maven
Gradle Plugin
SBT
ANT- Ant Tasks are provided as an AntLib
Different ways to use flyway
13. Demo- first migration- API
First migration:
Migration directory: src/main/resources/db/migration
First migration: src/main/resources/db/migration/V1__Create_person_table.sql
Create table PERSON (
ID int not null,
NAME varchar (100) not null
);
Executing our program
bar> mvn package exec:java -Dexec.mainClass=foobar.App
Check the output
INFO: Creating Metadata table: "PUBLIC"."schema_version"
INFO: Current version of schema "PUBLIC": << Empty Schema >>
INFO: Migrating schema "PUBLIC" to version 1
INFO: Successfully applied 1 migration to schema "PUBLIC"
(execution time 00:00.062s).
14. Demo- Second migration- API
Second migration:
src/main/resources/db/migration/V2__Add_people.sql
insert into PERSON (ID, NAME) values (1, 'Axel');
insert into PERSON (ID, NAME) values (2, 'Mr. Foo');
insert into PERSON (ID, NAME) values (3, 'Ms. Bar');
Executing our program
bar> mvn package exec:java -Dexec.mainClass=foobar.App
Check the output:
INFO: Current version of schema "PUBLIC": 1
INFO: Migrating schema "PUBLIC" to version 2
INFO: Successfully applied 1 migration to schema "PUBLIC"
(execution time 00:00.090s).
16. Demo- migrations Existing database
Existing database:
Migration directory: src/main/resources/db/migration
migrations: src/main/resources/db/migration/V1__Base_version.sql
Check the output:
INFO: Schema "PUBLIC" is up to date. No migration necessary.
production status:
State: SUCCESS version:1
INFO: Creating Metadata table: "PUBLIC"."schema_version“
INFO: Current version of schema "PUBLIC": << Empty Schema >>
INFO: Migrating schema "PUBLIC" to version 1.0
INFO: Successfully applied 1 migration to schema "PUBLIC“
Sells status:
State: SUCCESS version:1.0
Executing our program
bar> mvn package exec:java -Dexec.mainClass=foobar.ExistingDB
We have production db, we need to replicate database on other databases
Generate a sql script that includes the entire DDL (including indexes, triggers, procedures, ...) of the
production database.
Add all insert scripts of data available on production.
This script will form your base migration.
17. Demo- Prerequisites & Setup- CLI
Prerequisites: Java 5+
Download flyway command line distribution
http://repo1.maven.org/maven2/com/googlecode/flyway/flyway-
commandline/2.3/flyway-commandline-2.3.zip
Download H2 jars
http://repo1.maven.org/maven2/com/h2database/h2/1.3.170/
h2-1.3.170.jar
Setup
Create flyway-commandline-2.3 and required directories
Extract flyway-commandline-2.3.zip. It will create necessary structure
Configuring Flyway
Edit conf/flyway.properties
flyway.url= jdbc:h2:file:///E:/PROJECTS/flywaydb/flyway-2.3/commandLineDB
flyway.user=SA
18. Demo- first migration- CLI
First migration: sql/V1__Create_person_table.sql
Create table PERSON (
ID int not null,
NAME varchar (100) not null
);
Executing our program
flyway-2.3> flyway migrate
Check the output
Flyway (Command-line Tool) v.2.3
Creating Metadata table: "PUBLIC"."schema_version"
Current version of schema "PUBLIC": << Empty Schema >>
Migrating schema "PUBLIC" to version 1
Successfully applied 1 migration to schema "PUBLIC" (execution
time 00:00.194s).
19. Demo- Second migration- CLI
Second migration:
sql/V2__Add_people.sql
insert into PERSON (ID, NAME) values (1, 'Axel');
insert into PERSON (ID, NAME) values (2, 'Mr. Foo');
insert into PERSON (ID, NAME) values (3, 'Ms. Bar');
Executing our program
flyway-2.3> flyway migrate
Check the output:
Flyway (Command-line Tool) v.2.3
Current version of schema "PUBLIC": 1
Migrating schema "PUBLIC" to version 2
Successfully applied 1 migration to schema "PUBLIC"
(execution time 00:00.126s).
20. Demo- Prerequisites & Setup- Maven
We will use the project as earlier created just change the pom.xml
Adding the dependencies
<project ...> ...
<build>
<plugins><plugin>
<groupId>com.googlecode.flyway</groupId>
<artifactId>flyway-maven-
plugin</artifactId> <version>2.3</version>
<configuration>
<url>jdbc:h2:file:///E:/PROJECTS/flywaydb/f
lyway-maven/database/mavenDB</url>
<user>sa</user>
</configuration>
<dependencies> <dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>1.3.170</version>
</dependency> </dependencies>
</plugin> </plugins>
</build>
21. Demo- first migration- Maven
First migration:
Migration directory: src/main/resources/db/migration
First migration: src/main/resources/db/migration/V1__Create_person_table.sql
Create table PERSON (
ID int not null,
NAME varchar (100) not null
);
Executing our program
bar> mvn compile flyway:migrate
Check the output
[INFO] Creating Metadata table: "PUBLIC"."schema_version"
[INFO] Current version of schema "PUBLIC": << Empty Schema >>
[INFO] Migrating schema "PUBLIC" to version 1
[INFO] Successfully applied 1 migration to schema "PUBLIC"
(execution time 00:00
.171s).
22. Demo- Second migration- Maven
Second migration:
src/main/resources/db/migration/V2__Add_people.sql
insert into PERSON (ID, NAME) values (1, 'Axel');
insert into PERSON (ID, NAME) values (2, 'Mr. Foo');
insert into PERSON (ID, NAME) values (3, 'Ms. Bar');
Executing our program
bar> mvn compile flyway:migrate
Check the output:
[INFO] --- flyway-maven-plugin:2.3:migrate (default-cli) @ bar
---
[INFO] Current version of schema "PUBLIC": 1
[INFO] Migrating schema "PUBLIC" to version 2
[INFO] Successfully applied 1 migration to schema "PUBLIC"
(execution time 00:00
.075s).
23. Demo- Prerequisites & Setup- ANT
Prerequisites: Java 5+, A working Ant install
Create directories
# mkdir flyway-antlibs flyway-antmigrations
Download ant distribution and H2 jars
http://repo1.maven.org/maven2/com/googlecode/flyway/flyway-
ant/2.3/flyway-ant-2.3.zip
Create build.xml as below
<project name="foobar" default="migrate-db”
xmlns:flyway="antlib:com.googlecode.flyway.ant">
<target name="migrate-db“> <taskdef
uri="antlib:com.googlecode.flyway.ant"
resource="com/googlecode/flyway/ant/antlib.xml">
<classpath>
<pathelement location="libs/flyway-core-2.3.jar" />
<pathelement location="libs/flyway-ant-2.3.jar" />
</classpath>
</taskdef>
<path id="flyway.classpath">
<fileset dir="./libs" includes="h2-1.3.170.jar" />
</path>
<property name="flyway.locations" value="filesystem:./migrations" />
<flyway:migrate url="jdbc:h2:file:///E:/PROJECTS/flywaydb/flyway-
ant/database/antDB" user="SA" />
</target>
</project>
24. Demo- first migration- ANT
First migration:
Migration directory: migration
First migration: migration/V1__Create_person_table.sql
Create table PERSON (
ID int not null,
NAME varchar (100) not null
);
Executing our program
flyway-ant > ant
Check the output
[flyway:migrate] Creating Metadata table:
"PUBLIC"."schema_version"
[flyway:migrate] Current version of schema "PUBLIC": << Empty
Schema >>
[flyway:migrate] Migrating schema "PUBLIC" to version 1
[flyway:migrate] Successfully applied 1 migration to schema
"PUBLIC" (execution
time 00:00.161s).
25. Demo- Second migration- ANT
Second migration:
migration/V2__Add_people.sql
insert into PERSON (ID, NAME) values (1, 'Axel');
insert into PERSON (ID, NAME) values (2, 'Mr. Foo');
insert into PERSON (ID, NAME) values (3, 'Ms. Bar');
Executing our program
flyway-ant > ant
Check the output:
[flyway:migrate] Current version of schema "PUBLIC": 1
[flyway:migrate] Migrating schema "PUBLIC" to version 2
[flyway:migrate] Successfully applied 1 migration to schema
"PUBLIC" (execution
time 00:00.085s).
package foobar;import com.googlecode.flyway.core.Flyway; public class App { public static void main(String[] args) { // Create the Flyway instanceFlyway flyway = new Flyway(); // Point it to the database flyway.setDataSource("jdbc:h2:file:target/foobar", "sa", null); // Start the migration flyway.migrate(); } }
package foobar;import com.googlecode.flyway.core.Flyway;import com.googlecode.flyway.core.api.MigrationInfo;public class ExistingDB { public static void main(String[] args) { // Create the Flyway instance Flyway flyway = new Flyway();flyway.setInitVersion("1.0");flyway.setInitDescription("Base Version");flyway.setInitOnMigrate(true); // Point it to the databaseflyway.setDataSource("jdbc:h2:file:///E:/PROJECTS/flywaydb/flyway-api/database/existingProductionDB", "sa", null); // Start the migrationflyway.migrate();MigrationInfo[] existingProdInfos=flyway.info().all();System.out.println("production status: "); for (inti = 0; i < existingProdInfos.length; i++) {MigrationInfomigrationInfo = existingProdInfos[i];System.out.println("State: "+migrationInfo.getState()+" version:"+migrationInfo.getVersion()); } // Point it to the databaseflyway.setDataSource("jdbc:h2:file:///E:/PROJECTS/flywaydb/flyway-api/database/SellsDB", "sa", null); // Start the migrationflyway.migrate();MigrationInfo[] existingSellsInfos=flyway.info().all();System.out.println("Sells status: "); for (inti = 0; i < existingSellsInfos.length; i++) {MigrationInfomigrationInfo = existingSellsInfos[i];System.out.println("State: "+migrationInfo.getState()+" version:"+migrationInfo.getVersion()); } }}