Oracle to Postgres Migration - PGCon
Oracle to Postgres Migration - PGCon
Oracle to Postgres Migration - PGCon
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Oracle</strong> <strong>to</strong> <strong>Postgres</strong><br />
<strong>Migration</strong><br />
<br />
Presented by Gurjeet Singh<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Agenda<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Schema <strong>Migration</strong><br />
<br />
<br />
<br />
A.K.A “User” in <strong>Oracle</strong><br />
<br />
<br />
<strong>Oracle</strong> gives every user her own schema, by default<br />
<br />
<br />
<br />
<br />
<br />
<br />
Names of schema, tables, columns, functions, …<br />
<br />
<br />
<br />
<strong>Oracle</strong> converts them <strong>to</strong> UPPER CASE, unless quoted<br />
<strong>Postgres</strong> converts them <strong>to</strong> lower case, unless quoted<br />
You're safe if application quotes/does not quote the identifiers<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Schema <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
CREATE TABLE is mostly compatible, except<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Virtual Columns: Use views<br />
Data Types <br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Schema <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Primary Key, Foreign Key, Unique, CHECK, NOT NULL<br />
<br />
<br />
<br />
<br />
<br />
<br />
Btree / Descending: Works<br />
Reverse Key / Bitmap / Join: Not implemented (yet)<br />
Global: Feature not available<br />
<br />
<br />
<br />
<br />
Hash, List, Range<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Schema <strong>Migration</strong><br />
<br />
<br />
<br />
Not really the same thing as <strong>Oracle</strong>, but serves the same purpose<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Data Type <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Convert <strong>to</strong> VARCHAR or TEXT<br />
<br />
<br />
Convert <strong>to</strong> CHAR<br />
<br />
<br />
Convert <strong>to</strong> VARCHAR or TEXT<br />
<br />
<br />
<br />
<br />
Totally transparent <strong>to</strong> application.<br />
Size limit 2^30-1 (1 GB)<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Data Type <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
BIGINT, INT, SMALLINT, REAL, REAL, DOUBLE PRECISION<br />
<br />
<br />
NUMERIC<br />
<br />
<br />
<br />
<br />
<br />
<br />
Convert <strong>to</strong> INTEGER, FLOAT, …<br />
<br />
<br />
Convert <strong>to</strong> BYTEA; requires additional work in application<br />
migration<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Data Type <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
DATE or TIMESTAMP<br />
Also consider timezone effects; TIMESTAMP WITH TIMEZONE<br />
<br />
<br />
<br />
<br />
DATE + integer<br />
<br />
<br />
ORAFCE provides last_day, add_months, …<br />
TIMESTAMP – TIMESTAMP: <strong>Oracle</strong>: NUMBER, <strong>Postgres</strong>: INTERVAL<br />
<br />
<br />
<br />
<br />
Controls output of TO_CHAR and TO_DATE functions<br />
In <strong>Postgres</strong>, controlled by locale settings<br />
Note: DateStyle GUC variable<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Data <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
Use GUI <strong>to</strong>ols<br />
<br />
<br />
<br />
<br />
Use ETL style<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Data <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Extract sequence_name.nextval<br />
Use <strong>Postgres</strong>' setval('sequence_name', value)<br />
<br />
<br />
<br />
Avoid transaction logging (WAL), as noted previously<br />
Defer Index creation until after data load<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
RETURN becomes RETURNS<br />
EXECUTE IMMEDIATE becomes EXECUTE<br />
SELECT without INTO becomes PERFORM<br />
<br />
<br />
You must chose a language<br />
<br />
<br />
<br />
<br />
%TYPE, %ROWTYPE: works<br />
cursor_name%ROWTYPE: Doesn't work; Use RECORD<br />
REFCURSORS: No replacement; Use Set-Returning-Functions<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Au<strong>to</strong>nomous transactions<br />
<br />
<br />
<br />
Ability <strong>to</strong> COMMIT/ROLLBACK within procedures (only)<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
REVERSE LOOPs require switching the start/end conditions<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
Split them in<strong>to</strong> trigger function and the trigger<br />
<br />
<br />
CREATE OR REPLACE FUNCTION my_trig_fn() RETURNS TRIGGER<br />
AS $$ ... $$ LANGUAGE xxx;<br />
CREATE TRIGGER tbl1_trig1 BEFORE UPDATE ON table<br />
EXECUTE PROCEDURE my_trig_fn();<br />
<br />
<br />
<br />
:NEW, :OLD<br />
<br />
<br />
UPDATING, INSERTING => Use TG_OP; consider TG_* variables<br />
Don't forget <strong>to</strong> RETURN NEW in BEFORE triggers<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Execute a trigger only if a condition matches<br />
<strong>Postgres</strong> has it.<br />
<br />
<br />
<br />
<br />
<strong>Postgres</strong> has only functions<br />
Use RETURNS VOID<br />
May need application changes<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
RETURN becomes RETURNS<br />
Should provide parentheses () even for empty parameter list<br />
<br />
<br />
DEFAULT values for parameters<br />
<br />
<br />
Can return pseudo type RECORD<br />
<br />
<br />
Can return set of records; RETURNS SETOF type<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
A group of variables, functions and procedures<br />
Use schema <strong>to</strong> group functions<br />
Use (temporary) tables <strong>to</strong> replace variables<br />
No substitute for private functions, and variables<br />
Package Body initialization code: not very often used<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Business Logic <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
Functions within functions, oh my...<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Feature not available in <strong>Postgres</strong>; use normal functions<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved.
(C) EnterpriseDB Corporation 2011. All Rights Reserved.
Other Objects<br />
<br />
<br />
<br />
<br />
<br />
Feature not avaialable<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Feature not available<br />
Use the dblink contrib module, and views<br />
<br />
<br />
<br />
<br />
<br />
Use WITH RECURSIVE; SQL compliant and very flexible<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Other Objects<br />
<br />
<br />
<br />
<br />
<br />
<br />
Create wrapper views<br />
Jonathan Gardner<br />
<br />
<br />
Dan Chak – Materialized Views that Work<br />
<br />
<br />
<br />
<br />
Roll your own using Inheritance, Triggers, CHECK<br />
constraints, and constraint_exclusion<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Other Objects<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Work pretty much the same way as in <strong>Oracle</strong>.<br />
NOCACHE becomes CACHE 1 (or remove this clause)<br />
MAXVALUE <br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Other Objects<br />
<br />
<br />
<br />
NO{CACHE|MINVALUE|MAXVALUE|CYCLE}<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application Connectivity<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Works<br />
Works<br />
Consider turning off the au<strong>to</strong>commit flag in driver<br />
Npgsql<br />
Used by Pro*C programs<br />
<strong>Oracle</strong> Forms<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
Names of schema, tables, columns, functions, …<br />
<strong>Oracle</strong> converts them <strong>to</strong> UPPER CASE, unless quoted<br />
<strong>Postgres</strong> converts them <strong>to</strong> lower case, unless quoted<br />
You're safe if application quotes/does not quote the identifiers<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
In <strong>Oracle</strong>, WHERE clause entries mark the NULL augmented side<br />
with a (+)<br />
<strong>Oracle</strong> was ahead of the SQL Standards Committee<br />
<strong>Postgres</strong> provides SQL Standard syntax {LEFT|RIGHT|FULL}<br />
[OUTER] JOIN; and so does <strong>Oracle</strong>.<br />
SELECT e.name, d.name FROM emp e, dept d WHERE e.deptno<br />
= d.deptno (+)<br />
SELECT e.name, d.name FROM emp e LEFT JOIN dept d ON<br />
e.deptno = d.deptno<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
Becomes EXCEPT<br />
<br />
<br />
=> becomes :=<br />
<br />
<br />
<br />
<br />
For example:<br />
var = fn( c => 10, a => 'xyz', b => 2.5);<br />
becomes<br />
var = fn( c := 10, a := 'xyz', b := 2.5);<br />
Just a 1-row x 1-column table for expression evaluation<br />
Orafce provides this table.<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Use ROW_NUMBER() windowing function<br />
Use as a wrapper around the main query, if needed.<br />
Use CTID system column<br />
<br />
<br />
Use OID column<br />
<br />
<br />
<br />
<br />
<br />
<strong>Postgres</strong> doesn't have them, and doesn't want them.<br />
Discard, or keep for future reference; they won't bite you<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Application <strong>Migration</strong><br />
<br />
<br />
<br />
<br />
<strong>Oracle</strong> treats empty string '' as NULL. Non-standard and<br />
confusing.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Needs careful examination of queries comparing empty string<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Builtin functions<br />
<br />
<br />
<br />
<br />
<br />
<br />
Provided by Orafce<br />
Or use SQL standard COALESCE()<br />
<br />
<br />
<br />
<br />
<br />
Use the SQL Standard CASE clause<br />
<strong>Postgres</strong> now has VARIADIC; it might be possible <strong>to</strong> implement<br />
this where all parameters' data types are same.<br />
<br />
<br />
<br />
<strong>Postgres</strong> has this, but not very robust; requires testing of queries.<br />
Orafce provides the 1-argument version<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Builtin functions<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<strong>Postgres</strong> provides this.<br />
<strong>Postgres</strong> also provides SQL standards compliant syntax<br />
Use current_timestamp<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
DBA <strong>Migration</strong><br />
<br />
<br />
<br />
Have them attend some of Bruce's talks :)<br />
<br />
<br />
<br />
<br />
<br />
<br />
No Rollback Segments<br />
SGA => ~ shared_buffers<br />
PGA => ~ work_mem<br />
PMON => Postmaster<br />
TNS Listener => Postmaster<br />
GRANT/REVOKE => Almost the same; mostly syntax change<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Porting Tools<br />
<br />
<br />
<br />
<br />
<br />
A lot of <strong>Oracle</strong> compatibility functions<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
DUAL table<br />
Packages for various platforms (RPM, .deb)<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Porting Tools<br />
<br />
<br />
<br />
Pretty advanced schema and data extraction<br />
<br />
<br />
<br />
<br />
Extracts PL/SQL <strong>to</strong>o; Packages, Functions, Procedures<br />
Tries <strong>to</strong> convert PL/SQL<br />
Export <strong>to</strong> file, multiple files, compressed<br />
Export directly <strong>to</strong> <strong>Postgres</strong><br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Porting Tools<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Perl module<br />
Develop your own extraction <strong>to</strong>ols<br />
Ora2pg uses this<br />
Packages available for different platforms<br />
<br />
<br />
<br />
<br />
<br />
<br />
Developed by EnterpriseDB<br />
Mainly for <strong>Oracle</strong> <strong>to</strong> <strong>Postgres</strong> Plus Advanced Server migration<br />
May help in <strong>Oracle</strong> <strong>to</strong> <strong>Postgres</strong> migration<br />
Does not convert PL/SQL code<br />
Maps data types<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
Tables<br />
<br />
<br />
<br />
<br />
Views<br />
<br />
<br />
<br />
<br />
<br />
Sequences<br />
<br />
<br />
Indexes<br />
<br />
<br />
GRANT<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
Range<br />
<br />
List<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
No Hash partitions (yet)<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
<br />
Export <strong>to</strong> a single file<br />
Export <strong>to</strong> multiple files<br />
Compress output files using gzip or bzip<br />
Export directly <strong>to</strong> <strong>Postgres</strong> (not recommended as first step)<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
Everything is specified in a config file<br />
<br />
<br />
Define <strong>Oracle</strong>'s connection paramters<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
Define objects <strong>to</strong> export<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
Define objects <strong>to</strong> export (continued)<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
Modify structure<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
DATA_LIMIT: Limit number of incoming rows in memory<br />
OUTPUT: output file name; .gz or .bz2<br />
OUTPUT_DIR: Where <strong>to</strong> put output file(s)<br />
BZIIP2: Location of bzip2 executable<br />
FILE_PER_TABLE: One output file per table<br />
FILE_PER_FUNCTION: One function/trigger per file<br />
TRUNCATE_TABLE: Truncate the table before loading;<br />
DATA/COPY mode only<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
PG_DSN<br />
<br />
<br />
PG_USER<br />
PG_PWD<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
SKIP: List of schema constraint type <strong>to</strong> skip<br />
<br />
<br />
KEEP_PKEY_NAMES<br />
<br />
<br />
FKEY_DEFERRABLE<br />
<br />
<br />
DEFER_FKEY<br />
<br />
<br />
DROP_FKEY<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
<br />
DROP_INDEXES<br />
<br />
<br />
DISABLE_TABLE_TRIGGERS: 0/USER/ALL<br />
<br />
<br />
<br />
DISABLE_SEQUENCE<br />
<br />
<br />
DATA_TYPE<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
CASE_SENSITIVE<br />
<br />
<br />
ORA_RESERVED_WORDS<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
Ora2pg<br />
<br />
<br />
<br />
<br />
<br />
NLS_LANG<br />
<br />
<br />
<br />
<br />
BINMODE<br />
<br />
<br />
<br />
<br />
CLIENT_ENCODING<br />
<br />
<br />
<br />
<br />
<br />
<br />
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential
(C) EnterpriseDB Corporation 2011. All Rights Reserved. Company Confidential