SlideShare una empresa de Scribd logo
1 de 15
1

DB2 Migration – Tips, Tricks, and Pitfalls
1.1 Overview
The following sections include tips and tricks for migrating from DB2 environments to
Teradata. The concepts put forth here were derived (and in some cases updated) from two
web casts that are available on Teradata University. They are:
Migrating Your DB2 Database to Your Teradata Database Webcast (about 74 minutes
long)
Migrating your DB2 Applications to your Teradata Data Warehouse Webcast (about 61
minutes long)
If you wish to view these web casts in their entirety, log on to Teradata University and search
for “DB2”. These courses will appear in the search results. You can then choose to enroll and
take the courses.

1.2 Key Information
The following is a list of information that is important to gather in advance of planning
your DB2 to Teradata Migration.
Production Job Schedule – It is important to understand what each production job you
need to convert is. You should understand the types of jobs (autoloader, optiload, cube
build, etc), from what platform they run, and what tools are employed if any.
Concurrency/Dependency of Load Jobs (How many at once and to what tables in what
tablespaces).
Load / Batch Windows, SLAs, and procedures if SLAs are missed.
What Scheduler and Version Control software is being used and on what platform?
What kind of connectivity is available for Teradata to the Source (e.g. Channel Connect –
ESCON, etc.)
Is there any use of native CLI?
Reverse engineered Erwin Models from all warehouses
Are there any application / Non-DBMS (or coexisting processes on same node) node
analysis, End User Access
What processes are being performed on these nodes, ETL, reporting?
Representative sample of applications and queries from user community.
What user tools / applications are running against which systems?
What access methods (ODBC, JDBC, SQLJ, SPUFI, QMF, QMF Windows, mainframe,
etc…) are used?
What is the level of ad-hoc usage vs. canned queries, What are these queries/reports
written in? (COBOL, Java, Power Builder, Visual Basic, Cognos, etc.)
Is there and Data Replication (DPROP/DPROPR)
Which tables, when, what sort of process?
ETL Phase and Methodology
o Processed on the mainframe/what flavor of
Tool/combination?
o Are SQL Processes used in ETL?
o Is there INSERT, UPDATE, DELETE, UPSERT logic?
o Are Temp Tables?
o Does old data roll off, when or how often, how much?

Unix/Windows/ETL

What is the Refresh Methodology currently
o Monthly?
o Daily?
o Continual?
o Triggers, MQSeries?
o RI?
 Across all of history or just insert new data?
Relative Table sizing in rows and gigabytes
Partition key analysis for DB2 – What tables belong to what tablespaces which existing
on which node groups?

1.3 Migration Overview
A DB2 Migration consists of three distinct functions:
Establishing the Teradata Data Warehouse Infrastructure
Migrating the existing DB2 Environment to the new Teradata Environment
Establishing a production environment
1.3.1 Establishing the Teradata Data Warehouse Infrastructure
This phase establishes the hardware and software environment that will support the new
warehouse environment. This includes Teradata and non-Teradata environments for
development, test, QA, and production. Major activities include:
Prepare data center
Establish Teradata environment
Establish BAR environment
Establish network connectivity
Establish ETL server(s) and cloning of old environment
Establish BI server(s) and cloning of old environment
1.3.2 Migrating the existing DB2 Environment to the new
Teradata Environment
This involves migrating all the data, processes and applications associated with the DB2
Environment to Teradata. Major areas of focus include:
Physical Database – Design and Build of the physical database with modifications made
to take into consideration the differences between DB2 and Teradata
Database Security – Design and Build of the security views. Possibly implementation of
LDAP
ETL – Migration of the ETL that will maintain the data. Additionally, all supporting
process and scheduling will migrate
Post Load Processes – Migration of processes that are performed after the load is
complete. For example: post load reports
Historical Data Migration – The migration and validation of the historical data that is in
the current DB2 system
Reinstatement of Archive data – It is normal process to archive data to tape. This activity
sets up the process to allow retrieval of the DB2 archive data on to the new EDW.
User Table Migration – Migrate the tables and possibly the data created by users
BI Migration – Migrate the BI applications to Teradata
Regression / Benchmark Test Suite – Establish a set of scripts that will be utilized to
determine the impact of upgrades or changes in the new environment
Continuity of Business (COB) – Design and build the COB solution for the new
warehouse
Backup, Archive and Recovery (BAR) – Design and build the BAR solution for the new
warehouse
Testing – Define the test and execute the different levels of testing:
o Unit Test
o Integration Test
o System Test
o User Validation Test
o User Parallel Testing
Production Install – Establishing the new EDW into the production environment
Decommission – Decommission the existing DB2 system and infrastructure
1.3.3 Establishing a production environment
This involves establishing the processes and procedures that will be used to maintain the new
Teradata environment and assisting the end users in the migration. Major areas of focus
include:
Establish operational processes and procedures
User Change Management – Defines and Builds the infrastructure that will be used to
communicate to the end user of the EDW. This activity may also include establishing a
help desk
1.4 DB2 Database Migration Considerations
1.4.1 Physical Data Base Conversion
1.4.1.1 Table Conversion
The first thing to do when converting the physical database is to collect an inventory of DB2
DDL and Indexes. There are three basic methods of converting DB2 DDL to Teradata DDL.
1.4.1.2 Using a Script to Convert DB2 DDL
This is the best approach if there are a large number of tables to convert. There is a set of Perl
scripts that have been developed as part of the DB2 to Teradata Migration tool set that are
discussed later is this offer guide. These scripts will read the DB2 catalog and create DDL.
Please refer to that section for more detail on this option. Note that in some cases these
scripts may need to be modified slightly to work for your particular client. If this is the case,
and if you do not have anyone on your team who is comfortable modifying Perl scripts you
may want to choose one of the other options listed below for migrating tables.
1.4.1.3 Table Conversion Using Erwin
To use this method the steps are as follows:
Reverse Engineer the DB2 database(s) into an Erwin Model.
Then in Erwin:
Make the target database Teradata. This will change the data types for you
Go to the Tasks menu and forward engineer to generate the schema.
Do a Preview or Report, outline all, and Ctl-C (copy) to copy the text
Open up a text editor, Paste
Remove objects that are not to be ported
Removechange primary index names
If there are many tables, consider doing the above using
scripting language
Do object renames as necessary
Insert a logon statement before CREATE TABLE statement
Using the text file and Bteq, create the objects
When using this method you may want to use a scripting language for certain modification
tasks, especially when dealing with a large number of tables. Below is an example of a Perl
script that removes index names (assuming that is what you wanted to do) from the DDL.
#!/usr/bin/perl
$FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ;
$FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ;
open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!n"; return; };
open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!n"; return; };
while (<FILE>) {

a
@Fld = split( /s+/, $_, 100);
if (($Fld[1] eq "PRIMARY" ) && ($Fld[2] eq "INDEX" ))
{ print FOUT "$Fld[1] $Fld[2] Fld[4]n"; }
elsif (($Fld[1] eq "UNIQUE" ) && ($Fld[2] eq "PRIMARY" ))
{print FOUT "$Fld[1] $Fld[2] $Fld[3] $Fld[5]n";}
elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "INDEX" ))
{print FOUT "$Fld[0] $Fld[1] $Fld[3]n";}
elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "UNIQUE" ))
{print FOUT "$Fld[0] $Fld[1] $Fld[2] $Fld[4]n";}
elsif ($Fld[1] eq "INDEX" ) {print FOUT "$Fld[1] $Fld[3]n";}
else {print FOUT $_; }
}
close FILE; close FOUT;
Here is the before and after picture of the DDL that was modified by this scripts

Before
CREATE TABLE accts (
CUST_ID
FORMAT '-(10)9',

After
CREATE TABLE accts (

INTEGER NOT NULL

CUST_ID
FORMAT '-(10)9',

INTEGER NOT NULL

ACCT_TYPE

CHAR(2) NOT NULL,

ACCT_TYPE

CHAR(2) NOT NULL,

ACCT_NBR

CHAR(16) NOT NULL,

ACCT_NBR

CHAR(16) NOT NULL,

ACCT_START_DATE
FORMAT 'YYYYMMDD',
ACCT_END_DATE
'YYYYMMDD'

DATE NOT NULL

ACCT_START_DATE
FORMAT 'YYYYMMDD',

DATE FORMAT

)

ACCT_END_DATE
'YYYYMMDD'

DATE FORMAT

)
PRIMARY INDEX XIE1accts (

PRIMARY INDEX (

CUST_ID,

CUST_ID,

ACCT_TYPE

ACCT_TYPE

)
;

DATE NOT NULL

)
;

1.4.1.4 DB2LOOK Command
The third method that can be employed is using the DB2LOOK command. This command is
similar to the SHOWTABLE command in Teradata. IBM uses this command in order to
move objects from a production database to a test database but it can be used to extract DDL
as well. One caveat is that it is not always installed in all environments so you should check
with the DB2 administrator to determine if it has been on your system. If not it is a relatively
simple process to get it installed (similar to running a dip script). Once available you can run
this command and loop through getting DDL for each table and placing it in a text file where
it can be modified. Of course you can either do this manually or you can use the scripting
language of your choice to make the modifications, as mentioned above.
1.4.1.5 Index Conversion
For the indexes, you should focus on the primary index selection first. Secondary indexes in
DB2 are not necessarily comparable to Teradata Secondary Indexes so it is usually best to
select a primary index and the select secondary indexes based solely on Teradata criteria.
1.4.1.6 View Conversion
Erwin can be used to reverse engineer the views out of db2 as long as they are fairly simple
and straight forward. It does not, however, excel at converting more complex views. It does
not do well with the following:
Converting views that contain multiple joins
Converting all of the DB2 functions to proper Teradata functions
Converting view that contain derived tables
For these more complex views it would be best to build a conversion tool of your own using
your favorite scripting language or to do them manually. Even when using a scripting
language you may need to make some manual adjustments to some views in order to get
them to perform adequately.
1.4.1.7 UDFs and UDTs
These are being encountered more and more in conversions. They will require special
attention because while Teradata supports UDFs the languages and features are different.
Converting may not always be the answer. Each of these should be taken on a case by case
basis. Consider the following alternatives.
Port UDFs to Macros or Stored Procedures
Port UDTs to data containers in Stored Procedures
Consider if UDF/UDT logic can be replaced by a Teradata function thereby eliminating
the need for it.
1.4.1.8 Stored Procedures
Stored procedures have been out on IBM for a few years now. On IBM platforms they can be
written in Java, C, or any other platform supported language, as well as in SPL (Stored
Procedure Language). Terdata internal stored procedures must be written in ANSI Standard
SPL. External Stored Procedures may be written in C.
This is one of those conversion categories that you must grind out manually. One tip is to
look for alternative ways of accomplishing the same task. For example many Stored
Procedures make use of sequential looping. There may be a way to accomplish the same
thing in a different way using a parallel process in Teradata.
1.5 Historic Data Migration
1.5.1 Historic Data Migration Approaches
There are three basic approaches to migrating history. They are outlined below.
1.5.1.1 Unload DB2 / Load Teradata via Script
There are a set of Perl scripts that are a part of the DB2 to Teradata Migration toolkit that
will read the DB2 catalog and produce DB2 Unloads and corresponding FASTLOAD or
MULTILOAD scripts. These scripts run on UNIX and at this time will operate only against
DB2 EEE or other UNIX flavor of DB2. There is also a tool that generates MULTILOAD
scripts for the OS390 version of DB2. For more information on this option, refer to the
section on tools with in this guide.
1.5.1.2 Use an Existing ETL Tool.
If there is an ETL tool such as Informatica, Ab Initio, etc. available, this may provide a
viable method for moving history from DB2 to Teradata.
1.5.1.3 Manual Unload / Load
This is the least desirable option as it is the most labor intensive. It may, however be
necessary because the options above are not suitable or because the data must be physically
moved from one site to another.
If this is the method that is chosen, the first step is to create the exports. There are a number
of methods to do this. You should use the one that is most familiar to you and keep the
following in mind:
Sorting by any column is not desirable
Export numeric data types as displayable, sign preceding
Export dates as “YYYYMMDD”
Try to export fixed length rows, adding blanks to fill out char strings (OS390)
If VARCHAR exports are necessary, you need a 2 byte binary length preceding the
VARCHAR
All data must be in the host format of the EXPORT requestor, if OS390 then EBCDIC
(except for 2 byte lengths, which must be gig endian, little endian)
When creating ASCII flat files from a DB2 EEE environment the records can be fixed or
delimited either can be handled by Teradata. Here are some guidelines.
If Delimited
Choose a delimiter – Standard Pip “|” or other
Do not use a cancellation delimiter as they are different in DB2 than in Teradata.
A delimiter followed by a delimiter (e.g. ||) indicates a null value
There should be no delimiter at the end of a line
If Fixed
Determine the null indicator that you will use
For Both
Date - format „YYYY-mm-dd‟
Time w/timezone – format HH:MI:SS.S(F)Z
Timestamp w/timezone – format YYYY-MM-DDBHH:MI:SS.S(F)Z
Once you have created the extracts you must create FASTLOAD scripts. The following Perl
script will take text version of the DDL from Erwin and create a Fastload script.
#!/usr/bin/perl
$FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ;
$FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ;
open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!n"; return; };
open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!n"; return; };
while (<FILE>) {
@Fld = split( /s+/, $_, 100);
if (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "TABLE" )) {
print FOUT "sessions 2;nerrlimit 25;nlogontdpid/username,password;n";
print FOUT "set record unformatted;ndefinen";
$TNAME = $Fld[2]; @NAMES = (); $I=0;
while (<FILE>) {
@Fld = split( /s+/, $_, 100);
if ($Fld[0] eq ")") {last;}
$NAMES[$I] = $Fld[1];
if ($I++ > 0) {print FOUT ",";}
if ($Fld[2] eq "INTEGER") {print FOUT "$Fld[1] ($Fld[2])n";}
elsif ($Fld[2] eq "SMALLINT") {print FOUT "$Fld[1] ($Fld[2])n";}
elsif ($Fld[2] eq "BYTEINT") {print FOUT "$Fld[1] ($Fld[2])n";}
elsif ($Fld[2] eq "DATE") {print FOUT "$Fld[1] ($Fld[2] FORMAT
'YYYYMMDD')n";}
elsif (substr($Fld[2],0,4) eq "CHAR") {print FOUT "$Fld[1] ($Fld[2])n";}
# and then other datatypes
}
print FOUT "file=insert.input;n";
print FOUT "begin loading $TNAME errorfiles $TNAME.err1, $TNAME.err2;n";
print FOUT "INSERT INTO $TNAME (n";
for ($j=0; $j<$I; $j++) {
.................................................................................................. if ($j > 0) {print FOUT ",";}
print FOUT ":$NAMES[$j]n";
}
print FOUT ");nend loading;nlogoff;nnnnn";
}
}
close FILE; close FOUT;

Below is a figure that shows the text file input of the create table statement on the left and the
resulting FASTLOAD script on the right.

Input
CREATE TABLE accts (

Output
sessions 2;

CUST_ID INTEGER NOT NULL FORMAT '(10)9',

errlimit 25;

ACCT_TYPE CHAR(2) NOT NULL,

set record unformatted;

ACCT_NBR CHAR(16) NOT NULL,
ACCT_STRT_DATE DATE NOT NULL FORMAT
'YYYYMMDD',
ACCT_END_DATE DATE FORMAT 'YYYYMMDD'
)

define
CUST_ID (INTEGER)
,ACCT_TYPE (CHAR(2))
,ACCT_NBR (CHAR(16))
,ACCT_STRT_DATE (DATE FORMAT 'YYYYMMDD')
,ACCT_END_DATE (DATE FORMAT 'YYYYMMDD')

PRIMARY INDEX XIE1accts (
CUST_ID,
ACCT_TYPE
)
;

logontdpid/username,password;

file=insert.input;
begin loading accts errorfiles accts.err1, accts.err2;
INSERT INTO accts (
:CUST_ID
,:ACCT_TYPE
,:ACCT_NBR
,:ACCT_START_DATE
,:ACCT_END_DATE
);
end loading;
logoff;

1.5.2 Historic Data Migration Synchronization
There are two kinds of synchronization that you will encounter.
Batch load synchronization
This type of synchronization assumes that the load process adds rows to a table on
each cycle and that the rows of data are stamped with a date or id, the migration will
move the tables from a point in time.
Synchronization of updated tables
These are tables that are not loaded by adding rows, but rather by updating current
rows or by doing a full refresh. With this type of table, the synchronization can
approached in two different ways. The best approach is to move the tables at the very
end of the migration process, after the last run on the legacy system and before the
first run of the new system. In cases where this is not possible it might be necessary
to build a custom solution to achieve synchronization. A custom solution might be to
move the tables over and then on the legacy system perform a delta process to send a
synchronization file.
Synchronization fits into the general project timeline right before you are ready to begin the
production version of the incremental loads.
Basically your synchronization strategy is attempting to take any kind of gap there may be
between the DB2 system and the Teradata system and fill it. For example you may have
loaded all historical data as of the end of last month (for all tables). Three weeks have passed
and you need to load the three weeks of data that have been added/updated in the DB2
System to the Teradata system.
The basic steps for synchronization are as follows:
1.5.2.1 Step 1
Record the period id(s) for each table that you are loading. This is anything that can identify
a block of data that you are loading (e.g. timestamp, batch id, etc.)
1.5.2.2 Step 2
Migrate the data. This is the initial history migration. You may choose to do this in batches
of periods (e.g. 1 month of data at a time), to make each load smaller and more manageable.
1.5.2.3 Step 3
From recorded period(s) on, load additional periods into temporary tables. It may be
important to do this on a regular basis if the time from the initial history load to the time of
production turnover is protracted for some reason.
1.5.2.4 Step 4
Validate the data migrated to the temporary tables.
1.5.2.5 Step 5
Insert data from temporary tables into base production tables and delete temporaries. This
should be completed right before cutover to production.
It should be noted that for tables that require updates or delta loads you will have to develop
your own custom solution to ensure that updates are applied properly in the synchronization
process.
For tables that are totally refreshed each time they are loaded, you should plan to just do
those at the end of the migration process using a flat file as input.
1.5.3 Process Control and Tracking
For large migrations with hundreds or thousands of tables keeping track of which ones have
been loaded and with which periods is a complex task. As part of the DB2 To Teradata
Migration Tools package a tracking database and a set of macros for both updating the
database and reporting on the progress have been developed. The reporting macros can be
executed using a tool such as an excel spreadsheet. The maintenance macros can be executed
either manually or within a script depending on process that you are using for loading and
verifying data. This automated approach to tracking is detailed later in the section of this
document that is dedicated to conversion tools.
1.5.4 Testing
Testing will differ from site to site based on requirements however here are a few guidelines
for testing a history migration that can be considered as components of your overall test plan.
A field by field compare of all of the history may be cumbersome but could be performed
on random samples of rows from each table to ensure that the data appears to have loaded
successfully
Perform a check of the catalog to ensure that all physical structures have been created
Accessibility check – ensure that the roles, profiles, permissions, rights, etc allow for the
appropriate update/access of the data.
A field by field compare of incremental loads may be appropriate depending on the size
of the loads. Again if they are large you could reduce the compare to a sample of the
rows.
1.5.5 User Tables
User tables typically fall outside of normal maintenance activities. In most cases you will
have to do some digging to find out when they are loaded, who loads, them how they get
loaded, etc. These tables are typically small in size but can be great in number so you will
need an approach for migrating them. There are two basic ways to deal with them.
Use the self service approach. In other words give the users some sort of GUI tool
(preferably freeware or trial version software of some kind that allows them to migrate
their tables on their own schedule (of course with an appropriate mandatory cut over
date).An advantage of this option is that users may decide that they don‟t need certain
tables thereby reducing the total number of tables migrated. Note that this option is
probably only good for smaller tables
Move all the tables using one of the methods described earlier for migrating tables. Note
that this will require some level of coordination with the general user population as to
when it will be performed and that after that given date that changes made on the DB2
system will not be moved forward. This most likely will have to be a general rather than a
specific communication as it in most cases will be difficult to discover who owns which
tables.
1.5.6 Archived Data Migration
One area that is often overlooked in migrations and in scoping migrations is the effort
required to migrate archived data from the DB2 environment. Archived data is typically
required for accounting, regulatory, legal, etc. reasons. These reasons mandate that the data
may need to be restored at some point. You will need to understand the both the extent of the
archives that need to be migrated as well as the intended service level that is expected for a
restore in order to make a decision as to how to handle this type of data. It should be noted
that DB2 Archives are stored in a proprietary format so they cannot be directly restored to
Teradata.
There are three general options from which to choose when dealing with archived data. They
are as follows:
Restore  Extract  Load Archive on Teradata  Clean up
The steps for this option are to restore the archives on DB2, Extract the data From
DB2, load the data to Teradata, archive the data on Teradata, and, finally delete the
data from Teradata. This is the most involved in terms of effort to get the data moved
to Teradata and then archive the data in that format. This approach will allow for the
quickest turnaround time for a restore process.
Restore  Extract
The steps for this option are to restore the archives on DB2, then unload them and the
associated DDL to text files. When a restore is needed the files would need to be
loaded table by table (most likely using FASTLOAD) The number of tables needed to
restore will directly affect the amount of time needed.
Do nothing now
If for some reason there will continue to be a DB2 system on site (e.g. for
applications that are not being migrated), you may chose to not migrate the history.
The restore system could simply be the remaining DB2 system. This of course would
have to be acceptable to the client and you would have to carefully explore the steps
required to restore and retrieve the data. At a minimum these would need to be
documented as part of your engagement. This would also have to meet any SLAs for
restore that the customer had in place.

1.6 DB2 Application Migration Considerations
1.6.1 Data Type Considerations
1.6.1.1 Decimal 19 and Above
Prior to Teradata 12, Decimal 19 and above is not supported by Teradata but is by DB2. If
dealing with a Teradata system that has not been upgraded to Teradata 12, the following may
apply. When considering conversion options one should determine the purpose for which the
field is used. In this example we will use currency. Decimal (19,4) will store a value that
represents over 999 trillion dollars. In most scenarios, there should be no need to store a
value greater than 99 Trillion dollars which can be accomplished in Teradata using a Decimal
(18,4). You should determine the max value on each column within the DB2 system and see
if it will fit into a Decimal 18 on Teradata. It would also be a good idea to check with subject
matter experts at the client site to determine if there is a reason to believe that a value of
greater than 100 million would ever need to be stored in one of these columns. If not than the
preferred method would be to convert this columns to a smaller decimal field. If the evidence
shows that values greater than 100 million may be required than you must use either a Float
or Char (as required) data type to store the values.
1.6.1.2 VARCHAR
VARCHARs are used heavily in DB2, especially in DB2 EEE. This is probably a result of
DB2 not having column level compression until release 8. In Teradata, there is a 2 byte
overhead that is used for VARCHARs. Depending on the situation it may be more
advantageous to use CHAR data types with Multi-Value Compression (which cannot be used
on a VARCHAR in Teradata) as this may save more space than straight conversion from to
VARCHARs.
1.6.1.3 SHORT FLOAT
This is a data type that was used in DB2 because it was believed that it would save space.
These should be converted to the FLOAT data type within Teradata
1.6.1.4 BOOLEAN
This data type is not that common but could be used by procedural logic in downstream
processing. When converting this data type, use the BYTEINT data type in Teradata.
1.6.1.5 The Dumb Tilde (character delimiter)
DB2 exports and imports will use this in their processing. The default is a quote character. In
most cases you will need to strip those out when you find them. There are a few methods that
are can be used.
Use the SED command from Unix
Use the Substring functionality in SQL
Use an ETL Tool to strip them out
1.6.1.6 BIGINT
For Releases of Teradata that do not support 64 bit integer data types (those prior to TD12),
use the DECIMAL data type to convert DB2 BIGINT data types. In some cases you may find
that 3rd party tools have not yet incorporated support for Teradata‟s 64 bit integer data type.
In that case you would also use a Decimal field in its place.
1.6.2 Industry Standard Interfaces / Protocols
ODBC, OLEDB, ADO, RDO, and JDBC are all supported by Teradata. You may run into
something called SQLJ (which is for Java pre-processing), this is not supported by Teradata
unless the C++ pre-processor can be used.
It is interesting to note that the Native DB2 CLI can be mapped for the most part directly to
ODBC 1 for 1 as the two are very similar. This makes ODBC a good choice when converting
DB2 CLI rather than converting to native Teradata CLI
1.6.3 GUI Tools
GUI tools can be broken down into few different categories. The first category is tools that
are procedural in nature. This category includes tools like PowerBuilder, MS Access, Visual
Basic, Visual C, Excel, etc. These tools typically use ODBC JDBC interfaces. When doing
your initial assessment these should be considered as being moderately complex.
SQL that is generated by these tools shouldn‟t require conversion as it should be generating
ANSI compliant SQL. Note that an SQL that part of one of these applications and is not
directly generated by the tool may contain DB2 specific extensions and would require
conversion in order work with Teradata.
The second category of applications is SQL Submitters. An example of this type of tool in
the Teradata world is SQL Assistant (Queryman). In the DB2 world you may run into QMF
or some home grown query tool (e.g. John‟s Query Tool).
If you want or need to run QMF against Teradata there is TSAPI which can be used to run
QMF against Teradata as DB2 but because the SQL may contain DB2 specific, syntax you
will probably end up converting the SQL anyway. Therefore it is probably best to convert all
SQL from any DB2 SQL Submitter and migrate it to SQL Assistant. Then you should
perform testing and parallel testing to assure the same results are being generated.
The third category of GUI tools is GUI Application Writers. This category includes tools
such as Cognos, Microstrategy, Hummingbird, Crystal Reports, etc. These tools build some
kind of metadata layer or pseudo dictionary within the tool. They use these to understand the
data types which were in DB2, how to access tables, and what join criteria to use. These tools
typically have both ODBC support and CLI support for different database vendors. Because
most implementations make use of the ODBC interface these are amount the easiest
applications to convert. Typically you rebuild the metadata layer to point to Teradata
structures and all of the queries should work. The proof is in the testing and environments
with lots of standard query‟s / reports and lots of users will require diligent management of
the testing process.
1.6.4 GUI Three/N Tier Architectures
GUI Three/N Tier Architectures include tools like Cognos build cubes on a middle tier server
and then records are accessed from these middle tier servers. These servers can be either
small windows servers or mid range to large Unix servers. The migration of these cubes to
Teradata is relatively easy except if they are using some sort of “pass through” functionality.
With this type of functionality these tools send requests to the database to drill into the
underlying data that was used to build the cube. This SQL may need to be converted for two
reasons. The first is if it uses any DB2 specific extensions (as previously mentioned in this
document). The second is to ensure that it is optimized for Teradata. In some, DB2
implementations the SQL may be optimized to take advantage of DB2 indexing. Because the
indexing scheme will be different on Teradata, this SQL may need to be re-optimized for
Teradata.
1.6.5 Web Enabled Applications
Web enabled applications are those that allow for access to the application through a
browser. The idea behind these applications was to relieve the desktop of the need to have
client software resident on it. These applications require multiple tiers and web services to
talk with the DBMS and usually use Java technology and JDBC. One thing to watch out for
when migrating these applications is performance. In some cases large buffer pools may be
used in conjunction with a succession of small result sets. This may not work well in a
Teradata environment and may need to be re-engineered to make better use of the database
engine. Other than that these types of applications make use of standard interfaces and are
relatively easy to migrate. Again rigorous testing is the key to success.
1.6.6 Custom Applications
Custom Applications are by far the most difficult to migrate to Teradata. In the DB2 world
you will find many applications that are written in Cobol, C, PL/1, etc. Converting these
programs to work with Teradata may require and entire re-write. Often this is difficult
because these applications were written long ago and the documentation is poor or does not
exist. In some cases you will have to convert these applications which will involve re-writing
(in total or in part) them and/or at least qualifying and recompiling them. Before re-writing
1000s of lines of code consider the following alternatives.
Is the functionality required in the new warehouse environment or can the application be
retired?
Can the functionality be replicated using a standard piece of software or some standard
functionality within a tool? For example you may encounter a complex scheduling
application that can be replaced with an off the shelf solution.
Can the program functionality be separated from the database access by creating a
separate database interface (still requires changes to the program but may avoid a total
rewrite which is helpful in poor documentation scenarios)?
.
1.6.7 Downstream Data Feeds and Exports
It is very common for databases to feed downstream, applications, data marts, ODSs, etc.
These are performed in a couple of different ways. First they may be built in a Cobol, C,
C++, etc. Interface. Second they may be exported from DB2 using a command line
processor. Most of these should be converted to FAST EXPORT. And most will be
relatively easy to convert.
Some things to consider when converting command line exports are as follows:
When using UNIX you will need a disk landing area or pipe as you cannot go directly to
tape.
You will need to pay attention to character, column, and file delimiters used in DB2. As
of TTU 8.1 there is no delimiter function so if delimiters are required in the output you
will have add them in yourself. This can be done with an OUTMOD if you so desire.
You will need to understand whether the character set is ASCII or EBCIDIC

Más contenido relacionado

La actualidad más candente

The Database Environment Chapter 1
The Database Environment Chapter 1The Database Environment Chapter 1
The Database Environment Chapter 1Jeanie Arnoco
 
Building Data Warehouse in SQL Server
Building Data Warehouse in SQL ServerBuilding Data Warehouse in SQL Server
Building Data Warehouse in SQL ServerAntonios Chatzipavlis
 
The Database Environment Chapter 9
The Database Environment Chapter 9The Database Environment Chapter 9
The Database Environment Chapter 9Jeanie Arnoco
 
Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionAditya Trivedi
 
Data ware house design
Data ware house designData ware house design
Data ware house designSayed Ahmed
 
SQL - Parallel Data Warehouse (PDW)
SQL - Parallel Data Warehouse (PDW)SQL - Parallel Data Warehouse (PDW)
SQL - Parallel Data Warehouse (PDW) Karan Gulati
 
Datastage to ODI
Datastage to ODIDatastage to ODI
Datastage to ODINagendra K
 
TheETLBottleneckinBigDataAnalytics(1)
TheETLBottleneckinBigDataAnalytics(1)TheETLBottleneckinBigDataAnalytics(1)
TheETLBottleneckinBigDataAnalytics(1)ruchabhandiwad
 
The Database Environment Chapter 6
The Database Environment Chapter 6The Database Environment Chapter 6
The Database Environment Chapter 6Jeanie Arnoco
 
Tuning data warehouse
Tuning data warehouseTuning data warehouse
Tuning data warehouseSrinivasan R
 
Magic quadrant for data warehouse database management systems
Magic quadrant for data warehouse database management systems Magic quadrant for data warehouse database management systems
Magic quadrant for data warehouse database management systems divjeev
 
Capacity Management of an ETL System
Capacity Management of an ETL SystemCapacity Management of an ETL System
Capacity Management of an ETL SystemASHOK BHATLA
 
OLAP & Data Warehouse
OLAP & Data WarehouseOLAP & Data Warehouse
OLAP & Data WarehouseZalpa Rathod
 
Scalable data pipeline
Scalable data pipelineScalable data pipeline
Scalable data pipelineGreenM
 
SQL Server 2016 new features
SQL Server 2016 new featuresSQL Server 2016 new features
SQL Server 2016 new featuresSpanishPASSVC
 
Building a data warehouse of call data records
Building a data warehouse of call data recordsBuilding a data warehouse of call data records
Building a data warehouse of call data recordsDavid Walker
 

La actualidad más candente (20)

The Database Environment Chapter 1
The Database Environment Chapter 1The Database Environment Chapter 1
The Database Environment Chapter 1
 
Building Data Warehouse in SQL Server
Building Data Warehouse in SQL ServerBuilding Data Warehouse in SQL Server
Building Data Warehouse in SQL Server
 
The Database Environment Chapter 9
The Database Environment Chapter 9The Database Environment Chapter 9
The Database Environment Chapter 9
 
Oracle 11g data warehouse introdution
Oracle 11g data warehouse introdutionOracle 11g data warehouse introdution
Oracle 11g data warehouse introdution
 
Data ware house design
Data ware house designData ware house design
Data ware house design
 
SQL - Parallel Data Warehouse (PDW)
SQL - Parallel Data Warehouse (PDW)SQL - Parallel Data Warehouse (PDW)
SQL - Parallel Data Warehouse (PDW)
 
Datastage to ODI
Datastage to ODIDatastage to ODI
Datastage to ODI
 
TheETLBottleneckinBigDataAnalytics(1)
TheETLBottleneckinBigDataAnalytics(1)TheETLBottleneckinBigDataAnalytics(1)
TheETLBottleneckinBigDataAnalytics(1)
 
The Database Environment Chapter 6
The Database Environment Chapter 6The Database Environment Chapter 6
The Database Environment Chapter 6
 
Oracle: DW Design
Oracle: DW DesignOracle: DW Design
Oracle: DW Design
 
Teradata
TeradataTeradata
Teradata
 
Tuning data warehouse
Tuning data warehouseTuning data warehouse
Tuning data warehouse
 
Magic quadrant for data warehouse database management systems
Magic quadrant for data warehouse database management systems Magic quadrant for data warehouse database management systems
Magic quadrant for data warehouse database management systems
 
Capacity Management of an ETL System
Capacity Management of an ETL SystemCapacity Management of an ETL System
Capacity Management of an ETL System
 
OLAP & Data Warehouse
OLAP & Data WarehouseOLAP & Data Warehouse
OLAP & Data Warehouse
 
Teradata sql-tuning-top-10
Teradata sql-tuning-top-10Teradata sql-tuning-top-10
Teradata sql-tuning-top-10
 
Scalable data pipeline
Scalable data pipelineScalable data pipeline
Scalable data pipeline
 
SQL Server 2016 new features
SQL Server 2016 new featuresSQL Server 2016 new features
SQL Server 2016 new features
 
Building a data warehouse of call data records
Building a data warehouse of call data recordsBuilding a data warehouse of call data records
Building a data warehouse of call data records
 
Printing without printers
Printing without printersPrinting without printers
Printing without printers
 

Similar a Db2 migration -_tips,_tricks,_and_pitfalls

Handling Database Deployments
Handling Database DeploymentsHandling Database Deployments
Handling Database DeploymentsMike Willbanks
 
Oracle applications r12.2, ebr, online patching means lot of work for devel...
Oracle applications r12.2, ebr, online patching   means lot of work for devel...Oracle applications r12.2, ebr, online patching   means lot of work for devel...
Oracle applications r12.2, ebr, online patching means lot of work for devel...Ajith Narayanan
 
Mukhtar resume etl_developer
Mukhtar resume etl_developerMukhtar resume etl_developer
Mukhtar resume etl_developerMukhtar Mohammed
 
Mukhtar_Resume_ETL_Developer
Mukhtar_Resume_ETL_DeveloperMukhtar_Resume_ETL_Developer
Mukhtar_Resume_ETL_DeveloperMukhtar Mohammed
 
Dynamics of Leading Legacy Databases
Dynamics of Leading Legacy DatabasesDynamics of Leading Legacy Databases
Dynamics of Leading Legacy DatabasesCognizant
 
ELT Publishing Tool Overview V3_Jeff
ELT Publishing Tool Overview V3_JeffELT Publishing Tool Overview V3_Jeff
ELT Publishing Tool Overview V3_JeffJeff McQuigg
 
Obevo Javasig.pptx
Obevo Javasig.pptxObevo Javasig.pptx
Obevo Javasig.pptxLadduAnanu
 
Evolutionary Database Design
Evolutionary Database DesignEvolutionary Database Design
Evolutionary Database DesignAndrei Solntsev
 
FME World Tour 2015 - FME & Data Migration Simon McCabe
FME World Tour 2015 -  FME & Data Migration Simon McCabeFME World Tour 2015 -  FME & Data Migration Simon McCabe
FME World Tour 2015 - FME & Data Migration Simon McCabeIMGS
 
Building the DW - ETL
Building the DW - ETLBuilding the DW - ETL
Building the DW - ETLganblues
 
Test labs 2016. Тестирование data warehouse
Test labs 2016. Тестирование data warehouse Test labs 2016. Тестирование data warehouse
Test labs 2016. Тестирование data warehouse Sasha Soleev
 
SOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBSOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBUniFabric
 
Tauhid: SQL Server Database Change Automation
Tauhid: SQL Server Database Change AutomationTauhid: SQL Server Database Change Automation
Tauhid: SQL Server Database Change AutomationSQABD
 
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdf
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdfTips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdf
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdfAkhashRamnath
 
MDF and LDF in SQL Server
MDF and LDF in SQL ServerMDF and LDF in SQL Server
MDF and LDF in SQL ServerMasum Reza
 

Similar a Db2 migration -_tips,_tricks,_and_pitfalls (20)

Handling Database Deployments
Handling Database DeploymentsHandling Database Deployments
Handling Database Deployments
 
Oracle applications r12.2, ebr, online patching means lot of work for devel...
Oracle applications r12.2, ebr, online patching   means lot of work for devel...Oracle applications r12.2, ebr, online patching   means lot of work for devel...
Oracle applications r12.2, ebr, online patching means lot of work for devel...
 
Mukhtar resume etl_developer
Mukhtar resume etl_developerMukhtar resume etl_developer
Mukhtar resume etl_developer
 
Mukhtar_Resume_ETL_Developer
Mukhtar_Resume_ETL_DeveloperMukhtar_Resume_ETL_Developer
Mukhtar_Resume_ETL_Developer
 
Dynamics of Leading Legacy Databases
Dynamics of Leading Legacy DatabasesDynamics of Leading Legacy Databases
Dynamics of Leading Legacy Databases
 
ELT Publishing Tool Overview V3_Jeff
ELT Publishing Tool Overview V3_JeffELT Publishing Tool Overview V3_Jeff
ELT Publishing Tool Overview V3_Jeff
 
Obevo Javasig.pptx
Obevo Javasig.pptxObevo Javasig.pptx
Obevo Javasig.pptx
 
Evolutionary Database Design
Evolutionary Database DesignEvolutionary Database Design
Evolutionary Database Design
 
Linq to sql
Linq to sqlLinq to sql
Linq to sql
 
FME World Tour 2015 - FME & Data Migration Simon McCabe
FME World Tour 2015 -  FME & Data Migration Simon McCabeFME World Tour 2015 -  FME & Data Migration Simon McCabe
FME World Tour 2015 - FME & Data Migration Simon McCabe
 
Building the DW - ETL
Building the DW - ETLBuilding the DW - ETL
Building the DW - ETL
 
2007 SAPTech Ed
2007 SAPTech Ed2007 SAPTech Ed
2007 SAPTech Ed
 
Test labs 2016. Тестирование data warehouse
Test labs 2016. Тестирование data warehouse Test labs 2016. Тестирование data warehouse
Test labs 2016. Тестирование data warehouse
 
SOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DBSOUG_Deployment__Automation_DB
SOUG_Deployment__Automation_DB
 
12363 database certification
12363 database certification12363 database certification
12363 database certification
 
Tauhid: SQL Server Database Change Automation
Tauhid: SQL Server Database Change AutomationTauhid: SQL Server Database Change Automation
Tauhid: SQL Server Database Change Automation
 
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdf
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdfTips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdf
Tips_Tricks_for_Oracle_E-Business Suite_Database_Upgrade.pdf
 
Sql server
Sql serverSql server
Sql server
 
Sqllite
SqlliteSqllite
Sqllite
 
MDF and LDF in SQL Server
MDF and LDF in SQL ServerMDF and LDF in SQL Server
MDF and LDF in SQL Server
 

Último

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brandgvaughan
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr LapshynFwdays
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 

Último (20)

"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
WordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your BrandWordPress Websites for Engineers: Elevate Your Brand
WordPress Websites for Engineers: Elevate Your Brand
 
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 

Db2 migration -_tips,_tricks,_and_pitfalls

  • 1. 1 DB2 Migration – Tips, Tricks, and Pitfalls 1.1 Overview The following sections include tips and tricks for migrating from DB2 environments to Teradata. The concepts put forth here were derived (and in some cases updated) from two web casts that are available on Teradata University. They are: Migrating Your DB2 Database to Your Teradata Database Webcast (about 74 minutes long) Migrating your DB2 Applications to your Teradata Data Warehouse Webcast (about 61 minutes long) If you wish to view these web casts in their entirety, log on to Teradata University and search for “DB2”. These courses will appear in the search results. You can then choose to enroll and take the courses. 1.2 Key Information The following is a list of information that is important to gather in advance of planning your DB2 to Teradata Migration. Production Job Schedule – It is important to understand what each production job you need to convert is. You should understand the types of jobs (autoloader, optiload, cube build, etc), from what platform they run, and what tools are employed if any. Concurrency/Dependency of Load Jobs (How many at once and to what tables in what tablespaces). Load / Batch Windows, SLAs, and procedures if SLAs are missed. What Scheduler and Version Control software is being used and on what platform? What kind of connectivity is available for Teradata to the Source (e.g. Channel Connect – ESCON, etc.) Is there any use of native CLI? Reverse engineered Erwin Models from all warehouses Are there any application / Non-DBMS (or coexisting processes on same node) node analysis, End User Access What processes are being performed on these nodes, ETL, reporting? Representative sample of applications and queries from user community. What user tools / applications are running against which systems? What access methods (ODBC, JDBC, SQLJ, SPUFI, QMF, QMF Windows, mainframe, etc…) are used? What is the level of ad-hoc usage vs. canned queries, What are these queries/reports written in? (COBOL, Java, Power Builder, Visual Basic, Cognos, etc.) Is there and Data Replication (DPROP/DPROPR) Which tables, when, what sort of process?
  • 2. ETL Phase and Methodology o Processed on the mainframe/what flavor of Tool/combination? o Are SQL Processes used in ETL? o Is there INSERT, UPDATE, DELETE, UPSERT logic? o Are Temp Tables? o Does old data roll off, when or how often, how much? Unix/Windows/ETL What is the Refresh Methodology currently o Monthly? o Daily? o Continual? o Triggers, MQSeries? o RI?  Across all of history or just insert new data? Relative Table sizing in rows and gigabytes Partition key analysis for DB2 – What tables belong to what tablespaces which existing on which node groups? 1.3 Migration Overview A DB2 Migration consists of three distinct functions: Establishing the Teradata Data Warehouse Infrastructure Migrating the existing DB2 Environment to the new Teradata Environment Establishing a production environment 1.3.1 Establishing the Teradata Data Warehouse Infrastructure This phase establishes the hardware and software environment that will support the new warehouse environment. This includes Teradata and non-Teradata environments for development, test, QA, and production. Major activities include: Prepare data center Establish Teradata environment Establish BAR environment Establish network connectivity Establish ETL server(s) and cloning of old environment Establish BI server(s) and cloning of old environment 1.3.2 Migrating the existing DB2 Environment to the new Teradata Environment This involves migrating all the data, processes and applications associated with the DB2 Environment to Teradata. Major areas of focus include: Physical Database – Design and Build of the physical database with modifications made to take into consideration the differences between DB2 and Teradata
  • 3. Database Security – Design and Build of the security views. Possibly implementation of LDAP ETL – Migration of the ETL that will maintain the data. Additionally, all supporting process and scheduling will migrate Post Load Processes – Migration of processes that are performed after the load is complete. For example: post load reports Historical Data Migration – The migration and validation of the historical data that is in the current DB2 system Reinstatement of Archive data – It is normal process to archive data to tape. This activity sets up the process to allow retrieval of the DB2 archive data on to the new EDW. User Table Migration – Migrate the tables and possibly the data created by users BI Migration – Migrate the BI applications to Teradata Regression / Benchmark Test Suite – Establish a set of scripts that will be utilized to determine the impact of upgrades or changes in the new environment Continuity of Business (COB) – Design and build the COB solution for the new warehouse Backup, Archive and Recovery (BAR) – Design and build the BAR solution for the new warehouse Testing – Define the test and execute the different levels of testing: o Unit Test o Integration Test o System Test o User Validation Test o User Parallel Testing Production Install – Establishing the new EDW into the production environment Decommission – Decommission the existing DB2 system and infrastructure 1.3.3 Establishing a production environment This involves establishing the processes and procedures that will be used to maintain the new Teradata environment and assisting the end users in the migration. Major areas of focus include: Establish operational processes and procedures User Change Management – Defines and Builds the infrastructure that will be used to communicate to the end user of the EDW. This activity may also include establishing a help desk
  • 4. 1.4 DB2 Database Migration Considerations 1.4.1 Physical Data Base Conversion 1.4.1.1 Table Conversion The first thing to do when converting the physical database is to collect an inventory of DB2 DDL and Indexes. There are three basic methods of converting DB2 DDL to Teradata DDL. 1.4.1.2 Using a Script to Convert DB2 DDL This is the best approach if there are a large number of tables to convert. There is a set of Perl scripts that have been developed as part of the DB2 to Teradata Migration tool set that are discussed later is this offer guide. These scripts will read the DB2 catalog and create DDL. Please refer to that section for more detail on this option. Note that in some cases these scripts may need to be modified slightly to work for your particular client. If this is the case, and if you do not have anyone on your team who is comfortable modifying Perl scripts you may want to choose one of the other options listed below for migrating tables. 1.4.1.3 Table Conversion Using Erwin To use this method the steps are as follows: Reverse Engineer the DB2 database(s) into an Erwin Model. Then in Erwin: Make the target database Teradata. This will change the data types for you Go to the Tasks menu and forward engineer to generate the schema. Do a Preview or Report, outline all, and Ctl-C (copy) to copy the text Open up a text editor, Paste Remove objects that are not to be ported Removechange primary index names If there are many tables, consider doing the above using scripting language Do object renames as necessary Insert a logon statement before CREATE TABLE statement Using the text file and Bteq, create the objects When using this method you may want to use a scripting language for certain modification tasks, especially when dealing with a large number of tables. Below is an example of a Perl script that removes index names (assuming that is what you wanted to do) from the DDL. #!/usr/bin/perl $FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ; $FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ; open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!n"; return; }; open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!n"; return; }; while (<FILE>) { a
  • 5. @Fld = split( /s+/, $_, 100); if (($Fld[1] eq "PRIMARY" ) && ($Fld[2] eq "INDEX" )) { print FOUT "$Fld[1] $Fld[2] Fld[4]n"; } elsif (($Fld[1] eq "UNIQUE" ) && ($Fld[2] eq "PRIMARY" )) {print FOUT "$Fld[1] $Fld[2] $Fld[3] $Fld[5]n";} elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "INDEX" )) {print FOUT "$Fld[0] $Fld[1] $Fld[3]n";} elsif (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "UNIQUE" )) {print FOUT "$Fld[0] $Fld[1] $Fld[2] $Fld[4]n";} elsif ($Fld[1] eq "INDEX" ) {print FOUT "$Fld[1] $Fld[3]n";} else {print FOUT $_; } } close FILE; close FOUT; Here is the before and after picture of the DDL that was modified by this scripts Before CREATE TABLE accts ( CUST_ID FORMAT '-(10)9', After CREATE TABLE accts ( INTEGER NOT NULL CUST_ID FORMAT '-(10)9', INTEGER NOT NULL ACCT_TYPE CHAR(2) NOT NULL, ACCT_TYPE CHAR(2) NOT NULL, ACCT_NBR CHAR(16) NOT NULL, ACCT_NBR CHAR(16) NOT NULL, ACCT_START_DATE FORMAT 'YYYYMMDD', ACCT_END_DATE 'YYYYMMDD' DATE NOT NULL ACCT_START_DATE FORMAT 'YYYYMMDD', DATE FORMAT ) ACCT_END_DATE 'YYYYMMDD' DATE FORMAT ) PRIMARY INDEX XIE1accts ( PRIMARY INDEX ( CUST_ID, CUST_ID, ACCT_TYPE ACCT_TYPE ) ; DATE NOT NULL ) ; 1.4.1.4 DB2LOOK Command The third method that can be employed is using the DB2LOOK command. This command is similar to the SHOWTABLE command in Teradata. IBM uses this command in order to move objects from a production database to a test database but it can be used to extract DDL as well. One caveat is that it is not always installed in all environments so you should check with the DB2 administrator to determine if it has been on your system. If not it is a relatively simple process to get it installed (similar to running a dip script). Once available you can run
  • 6. this command and loop through getting DDL for each table and placing it in a text file where it can be modified. Of course you can either do this manually or you can use the scripting language of your choice to make the modifications, as mentioned above. 1.4.1.5 Index Conversion For the indexes, you should focus on the primary index selection first. Secondary indexes in DB2 are not necessarily comparable to Teradata Secondary Indexes so it is usually best to select a primary index and the select secondary indexes based solely on Teradata criteria. 1.4.1.6 View Conversion Erwin can be used to reverse engineer the views out of db2 as long as they are fairly simple and straight forward. It does not, however, excel at converting more complex views. It does not do well with the following: Converting views that contain multiple joins Converting all of the DB2 functions to proper Teradata functions Converting view that contain derived tables For these more complex views it would be best to build a conversion tool of your own using your favorite scripting language or to do them manually. Even when using a scripting language you may need to make some manual adjustments to some views in order to get them to perform adequately. 1.4.1.7 UDFs and UDTs These are being encountered more and more in conversions. They will require special attention because while Teradata supports UDFs the languages and features are different. Converting may not always be the answer. Each of these should be taken on a case by case basis. Consider the following alternatives. Port UDFs to Macros or Stored Procedures Port UDTs to data containers in Stored Procedures Consider if UDF/UDT logic can be replaced by a Teradata function thereby eliminating the need for it. 1.4.1.8 Stored Procedures Stored procedures have been out on IBM for a few years now. On IBM platforms they can be written in Java, C, or any other platform supported language, as well as in SPL (Stored Procedure Language). Terdata internal stored procedures must be written in ANSI Standard SPL. External Stored Procedures may be written in C. This is one of those conversion categories that you must grind out manually. One tip is to look for alternative ways of accomplishing the same task. For example many Stored Procedures make use of sequential looping. There may be a way to accomplish the same thing in a different way using a parallel process in Teradata.
  • 7. 1.5 Historic Data Migration 1.5.1 Historic Data Migration Approaches There are three basic approaches to migrating history. They are outlined below. 1.5.1.1 Unload DB2 / Load Teradata via Script There are a set of Perl scripts that are a part of the DB2 to Teradata Migration toolkit that will read the DB2 catalog and produce DB2 Unloads and corresponding FASTLOAD or MULTILOAD scripts. These scripts run on UNIX and at this time will operate only against DB2 EEE or other UNIX flavor of DB2. There is also a tool that generates MULTILOAD scripts for the OS390 version of DB2. For more information on this option, refer to the section on tools with in this guide. 1.5.1.2 Use an Existing ETL Tool. If there is an ETL tool such as Informatica, Ab Initio, etc. available, this may provide a viable method for moving history from DB2 to Teradata. 1.5.1.3 Manual Unload / Load This is the least desirable option as it is the most labor intensive. It may, however be necessary because the options above are not suitable or because the data must be physically moved from one site to another. If this is the method that is chosen, the first step is to create the exports. There are a number of methods to do this. You should use the one that is most familiar to you and keep the following in mind: Sorting by any column is not desirable Export numeric data types as displayable, sign preceding Export dates as “YYYYMMDD” Try to export fixed length rows, adding blanks to fill out char strings (OS390) If VARCHAR exports are necessary, you need a 2 byte binary length preceding the VARCHAR All data must be in the host format of the EXPORT requestor, if OS390 then EBCDIC (except for 2 byte lengths, which must be gig endian, little endian) When creating ASCII flat files from a DB2 EEE environment the records can be fixed or delimited either can be handled by Teradata. Here are some guidelines. If Delimited Choose a delimiter – Standard Pip “|” or other Do not use a cancellation delimiter as they are different in DB2 than in Teradata. A delimiter followed by a delimiter (e.g. ||) indicates a null value There should be no delimiter at the end of a line If Fixed Determine the null indicator that you will use
  • 8. For Both Date - format „YYYY-mm-dd‟ Time w/timezone – format HH:MI:SS.S(F)Z Timestamp w/timezone – format YYYY-MM-DDBHH:MI:SS.S(F)Z Once you have created the extracts you must create FASTLOAD scripts. The following Perl script will take text version of the DDL from Erwin and create a Fastload script. #!/usr/bin/perl $FINPUT = "./before.txt" ; $FOUTPUT = "./after.txt" ; $FINPUT = $ARGV[0] if ($ARGV[0]) ; $FOUTPUT = $ARGV[1] if ($ARGV[1]) ; open(FILE, $FINPUT) || do { warn "Can't open $FINPUT: $!n"; return; }; open(FOUT, ">$FOUTPUT") || do { warn "Can't create $FOUT: $!n"; return; }; while (<FILE>) { @Fld = split( /s+/, $_, 100); if (($Fld[0] eq "CREATE" ) && ($Fld[1] eq "TABLE" )) { print FOUT "sessions 2;nerrlimit 25;nlogontdpid/username,password;n"; print FOUT "set record unformatted;ndefinen"; $TNAME = $Fld[2]; @NAMES = (); $I=0; while (<FILE>) { @Fld = split( /s+/, $_, 100); if ($Fld[0] eq ")") {last;} $NAMES[$I] = $Fld[1]; if ($I++ > 0) {print FOUT ",";} if ($Fld[2] eq "INTEGER") {print FOUT "$Fld[1] ($Fld[2])n";} elsif ($Fld[2] eq "SMALLINT") {print FOUT "$Fld[1] ($Fld[2])n";} elsif ($Fld[2] eq "BYTEINT") {print FOUT "$Fld[1] ($Fld[2])n";} elsif ($Fld[2] eq "DATE") {print FOUT "$Fld[1] ($Fld[2] FORMAT 'YYYYMMDD')n";} elsif (substr($Fld[2],0,4) eq "CHAR") {print FOUT "$Fld[1] ($Fld[2])n";} # and then other datatypes } print FOUT "file=insert.input;n"; print FOUT "begin loading $TNAME errorfiles $TNAME.err1, $TNAME.err2;n"; print FOUT "INSERT INTO $TNAME (n"; for ($j=0; $j<$I; $j++) { .................................................................................................. if ($j > 0) {print FOUT ",";} print FOUT ":$NAMES[$j]n"; }
  • 9. print FOUT ");nend loading;nlogoff;nnnnn"; } } close FILE; close FOUT; Below is a figure that shows the text file input of the create table statement on the left and the resulting FASTLOAD script on the right. Input CREATE TABLE accts ( Output sessions 2; CUST_ID INTEGER NOT NULL FORMAT '(10)9', errlimit 25; ACCT_TYPE CHAR(2) NOT NULL, set record unformatted; ACCT_NBR CHAR(16) NOT NULL, ACCT_STRT_DATE DATE NOT NULL FORMAT 'YYYYMMDD', ACCT_END_DATE DATE FORMAT 'YYYYMMDD' ) define CUST_ID (INTEGER) ,ACCT_TYPE (CHAR(2)) ,ACCT_NBR (CHAR(16)) ,ACCT_STRT_DATE (DATE FORMAT 'YYYYMMDD') ,ACCT_END_DATE (DATE FORMAT 'YYYYMMDD') PRIMARY INDEX XIE1accts ( CUST_ID, ACCT_TYPE ) ; logontdpid/username,password; file=insert.input; begin loading accts errorfiles accts.err1, accts.err2; INSERT INTO accts ( :CUST_ID ,:ACCT_TYPE ,:ACCT_NBR ,:ACCT_START_DATE ,:ACCT_END_DATE ); end loading; logoff; 1.5.2 Historic Data Migration Synchronization There are two kinds of synchronization that you will encounter. Batch load synchronization This type of synchronization assumes that the load process adds rows to a table on each cycle and that the rows of data are stamped with a date or id, the migration will move the tables from a point in time. Synchronization of updated tables These are tables that are not loaded by adding rows, but rather by updating current rows or by doing a full refresh. With this type of table, the synchronization can approached in two different ways. The best approach is to move the tables at the very end of the migration process, after the last run on the legacy system and before the first run of the new system. In cases where this is not possible it might be necessary to build a custom solution to achieve synchronization. A custom solution might be to move the tables over and then on the legacy system perform a delta process to send a synchronization file.
  • 10. Synchronization fits into the general project timeline right before you are ready to begin the production version of the incremental loads. Basically your synchronization strategy is attempting to take any kind of gap there may be between the DB2 system and the Teradata system and fill it. For example you may have loaded all historical data as of the end of last month (for all tables). Three weeks have passed and you need to load the three weeks of data that have been added/updated in the DB2 System to the Teradata system. The basic steps for synchronization are as follows: 1.5.2.1 Step 1 Record the period id(s) for each table that you are loading. This is anything that can identify a block of data that you are loading (e.g. timestamp, batch id, etc.) 1.5.2.2 Step 2 Migrate the data. This is the initial history migration. You may choose to do this in batches of periods (e.g. 1 month of data at a time), to make each load smaller and more manageable. 1.5.2.3 Step 3 From recorded period(s) on, load additional periods into temporary tables. It may be important to do this on a regular basis if the time from the initial history load to the time of production turnover is protracted for some reason. 1.5.2.4 Step 4 Validate the data migrated to the temporary tables. 1.5.2.5 Step 5 Insert data from temporary tables into base production tables and delete temporaries. This should be completed right before cutover to production. It should be noted that for tables that require updates or delta loads you will have to develop your own custom solution to ensure that updates are applied properly in the synchronization process. For tables that are totally refreshed each time they are loaded, you should plan to just do those at the end of the migration process using a flat file as input. 1.5.3 Process Control and Tracking For large migrations with hundreds or thousands of tables keeping track of which ones have been loaded and with which periods is a complex task. As part of the DB2 To Teradata Migration Tools package a tracking database and a set of macros for both updating the database and reporting on the progress have been developed. The reporting macros can be
  • 11. executed using a tool such as an excel spreadsheet. The maintenance macros can be executed either manually or within a script depending on process that you are using for loading and verifying data. This automated approach to tracking is detailed later in the section of this document that is dedicated to conversion tools. 1.5.4 Testing Testing will differ from site to site based on requirements however here are a few guidelines for testing a history migration that can be considered as components of your overall test plan. A field by field compare of all of the history may be cumbersome but could be performed on random samples of rows from each table to ensure that the data appears to have loaded successfully Perform a check of the catalog to ensure that all physical structures have been created Accessibility check – ensure that the roles, profiles, permissions, rights, etc allow for the appropriate update/access of the data. A field by field compare of incremental loads may be appropriate depending on the size of the loads. Again if they are large you could reduce the compare to a sample of the rows. 1.5.5 User Tables User tables typically fall outside of normal maintenance activities. In most cases you will have to do some digging to find out when they are loaded, who loads, them how they get loaded, etc. These tables are typically small in size but can be great in number so you will need an approach for migrating them. There are two basic ways to deal with them. Use the self service approach. In other words give the users some sort of GUI tool (preferably freeware or trial version software of some kind that allows them to migrate their tables on their own schedule (of course with an appropriate mandatory cut over date).An advantage of this option is that users may decide that they don‟t need certain tables thereby reducing the total number of tables migrated. Note that this option is probably only good for smaller tables Move all the tables using one of the methods described earlier for migrating tables. Note that this will require some level of coordination with the general user population as to when it will be performed and that after that given date that changes made on the DB2 system will not be moved forward. This most likely will have to be a general rather than a specific communication as it in most cases will be difficult to discover who owns which tables. 1.5.6 Archived Data Migration One area that is often overlooked in migrations and in scoping migrations is the effort required to migrate archived data from the DB2 environment. Archived data is typically required for accounting, regulatory, legal, etc. reasons. These reasons mandate that the data may need to be restored at some point. You will need to understand the both the extent of the archives that need to be migrated as well as the intended service level that is expected for a restore in order to make a decision as to how to handle this type of data. It should be noted
  • 12. that DB2 Archives are stored in a proprietary format so they cannot be directly restored to Teradata. There are three general options from which to choose when dealing with archived data. They are as follows: Restore  Extract  Load Archive on Teradata  Clean up The steps for this option are to restore the archives on DB2, Extract the data From DB2, load the data to Teradata, archive the data on Teradata, and, finally delete the data from Teradata. This is the most involved in terms of effort to get the data moved to Teradata and then archive the data in that format. This approach will allow for the quickest turnaround time for a restore process. Restore  Extract The steps for this option are to restore the archives on DB2, then unload them and the associated DDL to text files. When a restore is needed the files would need to be loaded table by table (most likely using FASTLOAD) The number of tables needed to restore will directly affect the amount of time needed. Do nothing now If for some reason there will continue to be a DB2 system on site (e.g. for applications that are not being migrated), you may chose to not migrate the history. The restore system could simply be the remaining DB2 system. This of course would have to be acceptable to the client and you would have to carefully explore the steps required to restore and retrieve the data. At a minimum these would need to be documented as part of your engagement. This would also have to meet any SLAs for restore that the customer had in place. 1.6 DB2 Application Migration Considerations 1.6.1 Data Type Considerations 1.6.1.1 Decimal 19 and Above Prior to Teradata 12, Decimal 19 and above is not supported by Teradata but is by DB2. If dealing with a Teradata system that has not been upgraded to Teradata 12, the following may apply. When considering conversion options one should determine the purpose for which the field is used. In this example we will use currency. Decimal (19,4) will store a value that represents over 999 trillion dollars. In most scenarios, there should be no need to store a value greater than 99 Trillion dollars which can be accomplished in Teradata using a Decimal (18,4). You should determine the max value on each column within the DB2 system and see if it will fit into a Decimal 18 on Teradata. It would also be a good idea to check with subject matter experts at the client site to determine if there is a reason to believe that a value of greater than 100 million would ever need to be stored in one of these columns. If not than the preferred method would be to convert this columns to a smaller decimal field. If the evidence shows that values greater than 100 million may be required than you must use either a Float or Char (as required) data type to store the values.
  • 13. 1.6.1.2 VARCHAR VARCHARs are used heavily in DB2, especially in DB2 EEE. This is probably a result of DB2 not having column level compression until release 8. In Teradata, there is a 2 byte overhead that is used for VARCHARs. Depending on the situation it may be more advantageous to use CHAR data types with Multi-Value Compression (which cannot be used on a VARCHAR in Teradata) as this may save more space than straight conversion from to VARCHARs. 1.6.1.3 SHORT FLOAT This is a data type that was used in DB2 because it was believed that it would save space. These should be converted to the FLOAT data type within Teradata 1.6.1.4 BOOLEAN This data type is not that common but could be used by procedural logic in downstream processing. When converting this data type, use the BYTEINT data type in Teradata. 1.6.1.5 The Dumb Tilde (character delimiter) DB2 exports and imports will use this in their processing. The default is a quote character. In most cases you will need to strip those out when you find them. There are a few methods that are can be used. Use the SED command from Unix Use the Substring functionality in SQL Use an ETL Tool to strip them out 1.6.1.6 BIGINT For Releases of Teradata that do not support 64 bit integer data types (those prior to TD12), use the DECIMAL data type to convert DB2 BIGINT data types. In some cases you may find that 3rd party tools have not yet incorporated support for Teradata‟s 64 bit integer data type. In that case you would also use a Decimal field in its place. 1.6.2 Industry Standard Interfaces / Protocols ODBC, OLEDB, ADO, RDO, and JDBC are all supported by Teradata. You may run into something called SQLJ (which is for Java pre-processing), this is not supported by Teradata unless the C++ pre-processor can be used. It is interesting to note that the Native DB2 CLI can be mapped for the most part directly to ODBC 1 for 1 as the two are very similar. This makes ODBC a good choice when converting DB2 CLI rather than converting to native Teradata CLI 1.6.3 GUI Tools GUI tools can be broken down into few different categories. The first category is tools that are procedural in nature. This category includes tools like PowerBuilder, MS Access, Visual Basic, Visual C, Excel, etc. These tools typically use ODBC JDBC interfaces. When doing your initial assessment these should be considered as being moderately complex.
  • 14. SQL that is generated by these tools shouldn‟t require conversion as it should be generating ANSI compliant SQL. Note that an SQL that part of one of these applications and is not directly generated by the tool may contain DB2 specific extensions and would require conversion in order work with Teradata. The second category of applications is SQL Submitters. An example of this type of tool in the Teradata world is SQL Assistant (Queryman). In the DB2 world you may run into QMF or some home grown query tool (e.g. John‟s Query Tool). If you want or need to run QMF against Teradata there is TSAPI which can be used to run QMF against Teradata as DB2 but because the SQL may contain DB2 specific, syntax you will probably end up converting the SQL anyway. Therefore it is probably best to convert all SQL from any DB2 SQL Submitter and migrate it to SQL Assistant. Then you should perform testing and parallel testing to assure the same results are being generated. The third category of GUI tools is GUI Application Writers. This category includes tools such as Cognos, Microstrategy, Hummingbird, Crystal Reports, etc. These tools build some kind of metadata layer or pseudo dictionary within the tool. They use these to understand the data types which were in DB2, how to access tables, and what join criteria to use. These tools typically have both ODBC support and CLI support for different database vendors. Because most implementations make use of the ODBC interface these are amount the easiest applications to convert. Typically you rebuild the metadata layer to point to Teradata structures and all of the queries should work. The proof is in the testing and environments with lots of standard query‟s / reports and lots of users will require diligent management of the testing process. 1.6.4 GUI Three/N Tier Architectures GUI Three/N Tier Architectures include tools like Cognos build cubes on a middle tier server and then records are accessed from these middle tier servers. These servers can be either small windows servers or mid range to large Unix servers. The migration of these cubes to Teradata is relatively easy except if they are using some sort of “pass through” functionality. With this type of functionality these tools send requests to the database to drill into the underlying data that was used to build the cube. This SQL may need to be converted for two reasons. The first is if it uses any DB2 specific extensions (as previously mentioned in this document). The second is to ensure that it is optimized for Teradata. In some, DB2 implementations the SQL may be optimized to take advantage of DB2 indexing. Because the indexing scheme will be different on Teradata, this SQL may need to be re-optimized for Teradata. 1.6.5 Web Enabled Applications Web enabled applications are those that allow for access to the application through a browser. The idea behind these applications was to relieve the desktop of the need to have client software resident on it. These applications require multiple tiers and web services to talk with the DBMS and usually use Java technology and JDBC. One thing to watch out for when migrating these applications is performance. In some cases large buffer pools may be used in conjunction with a succession of small result sets. This may not work well in a Teradata environment and may need to be re-engineered to make better use of the database engine. Other than that these types of applications make use of standard interfaces and are relatively easy to migrate. Again rigorous testing is the key to success.
  • 15. 1.6.6 Custom Applications Custom Applications are by far the most difficult to migrate to Teradata. In the DB2 world you will find many applications that are written in Cobol, C, PL/1, etc. Converting these programs to work with Teradata may require and entire re-write. Often this is difficult because these applications were written long ago and the documentation is poor or does not exist. In some cases you will have to convert these applications which will involve re-writing (in total or in part) them and/or at least qualifying and recompiling them. Before re-writing 1000s of lines of code consider the following alternatives. Is the functionality required in the new warehouse environment or can the application be retired? Can the functionality be replicated using a standard piece of software or some standard functionality within a tool? For example you may encounter a complex scheduling application that can be replaced with an off the shelf solution. Can the program functionality be separated from the database access by creating a separate database interface (still requires changes to the program but may avoid a total rewrite which is helpful in poor documentation scenarios)? . 1.6.7 Downstream Data Feeds and Exports It is very common for databases to feed downstream, applications, data marts, ODSs, etc. These are performed in a couple of different ways. First they may be built in a Cobol, C, C++, etc. Interface. Second they may be exported from DB2 using a command line processor. Most of these should be converted to FAST EXPORT. And most will be relatively easy to convert. Some things to consider when converting command line exports are as follows: When using UNIX you will need a disk landing area or pipe as you cannot go directly to tape. You will need to pay attention to character, column, and file delimiters used in DB2. As of TTU 8.1 there is no delimiter function so if delimiters are required in the output you will have add them in yourself. This can be done with an OUTMOD if you so desire. You will need to understand whether the character set is ASCII or EBCIDIC