2. XRec – Extended Reconciliation using Excel VBA
ABSTRACT
This paper describes the use of an Excel Application
which was developed during the Testing phase of the project
[Mainframe Information Delivery Management System – It’s a
Reverse engineering project where in we have to migrate the
existing FEEDS from Mainframes to Open Systems (ETL)]. We
have developed two applications; each will have its own
Advantages and uniqueness in terms of generating output. This
document describes one of the applications. The Name of the
Application is ComparX-R. Prior to the development of this
application we had to compare the Mainframe Feed data with
ETL Generated Feed data manually. This was time consuming
in terms of judging the accuracy of the output data. With the use
of this application we were able to present the comparison,
mismatched, statistics reports to our client successfully. This
application is used to automate the comparison process [of Data
from Legacy systems and Data generated by Open systems -
ETL]. It’s created with the help of Excel and VBA codes and
was very much handy at the time of Testing, Implementation. It
can be used for MIS related presentations too.
TARGET AUDIENCE
End Users, Testers, ETL/Report Developers, Data Migration
Developers, and Managers
1. INTRODUCTION
ComparX – R [Compare eXcel Row wise] is a
Comparison Tool which is primarily helpful for you when
you have to comparing huge data. Normally in Day to day
activities Excel is very much a part of our both Official work
and personal work.
Here we can put the data in source sheets and execute the
options available in Menu to perform any operation on the
data. This tool will generate 10 Reports which can be for
used for analysis further.
Process:
It takes the first row from ETL source data and
compares it with first row of Mainframes source data. Both
Mainframes and ETL data should be sorted in the same order
based on same number of columns. If Mainframe has one
record in row number 2 then ETL also should have the same
row in the same row number. [You may or may not have
mismatched columns in that rows] But Key [Columns used
for sorting] should be synchronized between Mainframe
system and ETL.
2. CHALLENGES
Display the data in readable format so that the same is handy
at the time of presentation to the users
Understand the requirement and choose the optimum logic
Generate the various reports in quick succession
Make it as user friendly
Optimize the Report Running time
3. REQUIRED COMPONENTS
The following table lists these identified Components.
Table 1: Components
S. No Component Name Type
1. MS EXCEL Application
2. Windows XP or
higher
OS
4. CONTENTS
Comprises of Sheets like Navigation, Steps, Sample
Data, Colors List, Mainframe, ETL, and Compare Data. Each
sheet has its own options and descriptions. Compare Data is a
Main menu option where we can generate reports, Format the
reports, and Export the reports in various formats like TXT,
HTML and CSV.
3. 5. FEATURES
Statistics - This will be useful for analysis and
presentation.
Gives Report Generation Time, Reports Count
We can Load External Files.
Formatting the reports for presentation
Converting the Excel Data into HTML, CSV, Delimited
TXT file formats
Row and Column count - To decide whether to go for
the result set or not.
Colors list generation to see what colors can be used for
the report formatting.
Loading external delimited data files, loading the
sample data for testing.
Save Sources and Reports alone in a new file.
Exported reports will be saved in the same folder of
application.
Deletion of Reports, Navigation to all reports
6. REPORTS
Compared Report:
Gives the comparison of two sheets with format.
Layout attached in Appendix A.
Compared > 2000 Rows Report:
Gives the comparison of two sheets without format.
Layout attached in Appendix A.
[This is for Greater than 2000 Rows].
Matched Report:
Gives the comparison of two sheets, but only
the MATCHED Records with format.
Layout attached in Appendix A.
Matched > 2000 Rows Report:
Gives the comparison of two sheets, but only
the MATCHED Records without format.
Layout attached in Appendix A.
[This is for Greater than 2000 Rows]
MisMatched Report:
Gives the comparison of two sheets, but only
the MISMATCHED Records with format.
Layout attached in Appendix A.
MisMatched > 2000 Rows Report:
Gives the comparison of two sheets, but only the
MISMATCHED Records without format.
Layout attached in Appendix A.
[This is for Greater than 2000 Rows]
Mismatched_Cols Report:
Gives the MISMATCHES between two sheets,
but only the MISMATCHED Columns with format.
Layout attached in Appendix A.
Mismatched_Cols > 2000:
Gives the MISMATCHES between two sheets,
but only the Rows Report MISMATCHED
Columns without format.
Layout attached in Appendix A.
[This is for Greater than 2000 Rows]
Stats Report:
Gives the STATISTICS of two sheets like
Total Rows and Columns, Matched/Mismatched
Rows and columns. MisMatched Column names
and Column Number etc. You can find the layout
in Appendix A.
Stats > 2000 Rows Report:
Gives the STATISTICS of two sheets like
Total Rows and Columns, Matched/ Mismatched
Rows and columns. MisMatched Column names
and Column Number etc. You can find the layout
in Appendix A. [This is for Greater than 2000 Rows]
7. REPORT LAYOUTS
Compared Report and Compared > 2000 Rows Report:
[MF] <Mainframe data> = <ETL data> [ETL] if both data
are equal - This will be displayed in Single cell.
[MF] <Mainframe data> <> <ETL data> [ETL] {Row #: |
Column :} if both data are NOT equal - This will be
displayed in single cell.
Status Column will have value as MATCHED if both data
are matched, MISMATCHED if both data are not matched.
Status Rows:
1st Row:
No. of Rows in mainframe = No. of Rows in ETL.
Row count matched or not matched.
Completely matched Records.
2nd Row: No. of Mismatched Records
4. Report Generation time.
3rd Row:
Link to Top Row of the Report, to Compare
Data Menu, to Steps Sheet.
Compared Report generates with format
Compared > 2000 Rows Report generates
without format
Refer Appendix A
Matched Report and Matched > 2000 Rows Report:
[MF] <Mainframe data> = <ETL data> [ETL] if both data
are equal - This will be displayed in single cell.
Status Column will only have values MATCHED if both
data are matched.
Status Rows:
1st Row:
No. of Rows in mainframe = No. of Rows in ETL.
Completely matched Records.
2nd Row: No. of Matched Records
Report Generation time.
3rd Row:
Link to Top Row of the Report, to Compare
Data Menu, to Steps Sheet.
Matched Report generates with format
Matched > 2000 Rows Report generates
without format
Refer Appendix A
MisMatched Report and MisMatched > 2000 Rows
Report:
[MF] <Mainframe data> <> <ETL data> [ETL] if both data
are equal - This will be displayed in single cell.
[MF] <Mainframe data> = <ETL data> [ETL] if both data
are equal - This will be displayed in Single cell.
Status Column will only have value as MISMATCHED if
both data are NOT matched.
Status Rows:
1st Row:
No. of Rows in mainframe = No. of Rows in ETL.
MISMATCHED Records Count.
2nd Row: No. of MisMatched Records
Report Generation time.
3rd Row:
Link to Top Row of the Report, to Compare
Data Menu, to Steps Sheet.
MisMatched Report generates with format
MisMatched > 2000 Rows Report generates
without format
Refer Appendix A
Mismatched_Cols Report and Mismatched_Cols > 2000
Rows Report:
[MF] <Mainframe data> <> <ETL data> [ETL] if both data
are equal - This will be displayed in single cell.
[MF] <Mainframe data> = <ETL data> [ETL] if both data
are equal - This will be displayed in single cell.
If Entire Column has matching value that column will be
deleted.
Status Column will only have values MISMATCHED if both
data are NOT matched.
Status Rows:
1st Row:
No. of Rows in mainframe = No. of Rows in ETL.
MISMATCHED Records Count.
2nd Row: No. of MisMatched Records
Report Generation time.
3rd Row:
Link to Top Row of the Report, to Compare
Data Menu, to Steps Sheet.
MisMatched Report generates with format
MisMatched > 2000 Rows Report
generates without format
Refer Appendix A
Stats Report and Stats > 2000 Rows Report:
Table 1
Mismatched Column's Number
Mismatched Column's Name
Mismatched Records for Each column
Total Mismatched Values <Sum>
Table 2
<Count of Records for each category >
<Percentage of Records for each category>
5. Total Rows
Total Columns
Mismatched Rows
Mismatched columns of Mismatched Rows
Matched columns of Mismatched Rows
Matched Rows
Matched columns
Link to Compare Data Menu, Link to MisMatched Report,
Link to Compared Report
Refer Appendix A
8. LIMITAIONS
Can process 255 columns as excel column limit per
sheet is 255 plus one status column.
Can process 65533 rows as excel sheet limit is 65536
rows and status will take 3 rows.
Row Count should be equal in both sources to get the
desired result.
Source files should be sorted properly.
This is really helpful for Sequential Comparison and
gives accurate result.
Not as fast as than UNIX or other scripts, as it has to
compare each cell of one sheet against each cell in other
sheet in a sequential manner
But Comparison wise it gives Accurate results with a
good look and feel.
Rejected records will not be saved in separate file.
Even if there are columns with no values when comparing
columns, it displays the status as Mismatched only.
9. SUMMARY
It can be used as a Standard and Standalone Reporting
Tool. Various complex Reporting features like Statistics, Matched
and Mismatched, Export Options, etc. can be clubbed using the
integrated solution which ComparX Offers.
The solution is ideal for users who:
Want a single Reporting solution that incorporates all of their
Comparison and Presentation Requirements.
Need to compare Mainframe and ETL data
Want to see Comparison report
Need to reconcile data across systems to gain a complete
view of customers.
Want to Export generated reports in HTML formats
Want to supplement Comparison reporting with a full breadth
of other options like Matched vs. Mismatched, Statistics reports
etc.
Wants to load external text files into excel for comparison.
Wants to format the reports.