Migrating an Oracle database to Postgres is never an automated operation. And it rarely (never?) involve just the database. Experience brought us to develop an agile methodology for the migration process, involving schema migration, data import, migration of procedures and queries up to the generation of unit tests for QA.
Pitfalls, technologies and main migration opportunities will be outlined, focusing on the reduction of total costs of ownership and management of a database solution in the middle-long term (without reducing quality and business continuity requirements).
2. Agile Oracle to PostgreSQL
migrations
PGConf.EU 2013, Dublin, Nov 1st
Gabriele Bartolini
2ndQuadrant Italia / PostgreSQL Europe
gabriele.bartolini@2ndQuadrant.it
@_GBartolini_
3. Gabriele Bartolini
• Co-Founder and Manager of 2ndQuadrant Italia
• Data Architect, Business critical environments
• Data warehousing
• Co-Founder Italian PostgreSQL Users Group
• Co-Founder PostgreSQL Europe
• PostgreSQL Contributor and Advocate
11. License
• Very simple and clear
• Applies to every part
(module, product, option) of
Postgres
• server, client, libraries,
additional modules
• 100% open source
• No need to hire a lawyer to
understand it
• Highly Permissive
• Not subject to monopoly
nor acquisitions
• Fosters market competition
in professional services
offer
12. Oracle DBA, do not be afraid!
• Transferrable knowledge
• PostgreSQL focused professional training
• 100% open source
19. Comparison with Oracle upgrade
Testing
Processes
Training
Licensing
fictitious data
Dev
25,00%
50,00%
20. External applications
• Force your supplier to provide Postgres support for their
application
• You set the rules
• Typical case of the public sector
• e.g.: Land registry
• Financial crisis and competition are on your side
21. Agile approach
• Requirements change
• Every context is different from the other
• No one-size-fits-all scenario
• Be dynamic:
• SCRUM (with the customer)
• Kanban (internally)
• DevOps
26. Schema major differences
• USER = schema/database
• UPPERCASE = lowercase
• Partitioning (*)
• Tablespaces
• SYNONYMs
• search_path / auto updatable views / foreign tables
27. Queries
• ANSI/ISO SQL
• PostgreSQL has no hints
(optimiser)
• OUTER JOINs
• (+)
• use LEFT/RIGHT JOIN
• Major differences:
• DUAL
• NULL = NULL
• ROWNUM and ROWID
• Postgres: NULL
• ...
• Oracle: TRUE
28. Data
• Might require specialised
consulting activities
• Major solutions:
• ETL tools (e.g. Kettle)
• External scripts (e.g.
ora2pg or custom)
• COPY t FROM stdin;
• Foreign tables with Oracle
FDW
• INSERT INTO t SELECT *
FROM ft;
• Cut-over time
•>0
•~0
29. Stored Procedures
• Most critical and complex component of the whole migration
• Package = extension (schema)
• Two major ways:
• 1:1 conversion (not always possible)
• complete rewriting (optimised for PostgreSQL)
• It might required specialised consulting activities
30. PL/SQL vs PL/pgSQL
• Syntactical differences
• PROCEDURE = FUNCTION that returns VOID
• TRIGGER = TRIGGER FUNCTION + TRIGGER
• Lack of autonomous transactions
• Lack of global variables
• ...
31. Test
• Vital component for a successful migration process
• It measures the QUALITY of the project
• It can be reused in the future for PostgreSQL upgrades
• strategic investment in the long period
33. ora2pg
• URL: http://ora2pg.darold.net/
• Written in Perl - GNU GPL 3
• Reads the information schema from Oracle
• Generates DDL instructions for PostgreSQL
• Highly configurable (mapping data type conversions)
• Basic support for PL/SQL (regexp)
34. orafce
• URL: http://orafce.projects.pgfoundry.org/
• C and SQL - BSD licensed
• Set of functions, objects and modules that are part of Oracle:
• NVL function (COALESCE)
• DBMS_ALERT, DBMS_OUTPUT, UTL_FILE, ...
• DUAL
35. Oracle FDW
• URL: http://oracle-fdw.projects.pgfoundry.org/
• SQL/MED
• uses the API for Foreign Data Wrapper
• CREATE FOREIGN TABLE
• Currently READ ONLY
• Useful for ETL
36. Example of Oracle FDW
...
CREATE FOREIGN TABLE oracle_table (
id integer NOT NULL,
...
) SERVER oradb OPTIONS
(schema 'ORAUSER', table 'ORATAB');
...
-- pushes down the WHERE clause
SELECT * FROM oracle_table WHERE id=10;
37. PgTap
• Unit tests suite for PostgreSQL
• Extensions (functions)
• Produces TAP output
• Test Driven Development in databases
38. Vagrant
• Open Source software
• Creation and configuration of virtual environments
• Provisioning: Puppet, Chef
• Useful for local testing
39. PostgreSQL
• Every major release of Postgres reduces the technology gap
with Oracle
• 5 years ago, Postgres did not have hot standby, streaming/
synchronous/cascading replication, window functions, CTEs,
column permissions, foreign tables, serialisable isolation
level, XML, JSON, updatable and materialised views ...
• Is it called “OraGIS” or “PostGIS”?
• Postgres does have transactional DDL
43. Cost analysis of a database solution on a
period of 3 years for 1 server
License
Support
300.000
270.000
240.000
210.000
64.000
150.000
Costs in Euro
180.000
140.000
120.000
32.000
90.000
60.000
30.000
0
70.000
16.000
35.000
1 CPU Socket
2 CPU Socket
4 CPU Socket
* PostgreSQL with 24/7 support from us
7.500
PostgreSQL*
44. Cost analysis of a migration to PostgreSQL
in a 3 year period
Oracle
PostgreSQL
150.000
135.000
120.000
50.000
105.000
Costs in euro
90.000
75.000
60.000
45.000
70.000
30.000
2.500
0
Anno 1
2.500
16.000
15.000
16.000
Anno 2
Anno 3
* PostgreSQL with 24/7 support from us
45. Conclusions
• Final solution: 100% open
source
• reduced TCO
• No vendor lock-in
• Migration must be focused
on costs evaluation
• Supervised process (not
automated)
• If a feature is missing, you
can simply sponsor it
• If needed, professional
companies can help you
before, during and after the
migration process (you
won’t be alone)