Key Considerations and Approach for Performance Testing of Cards Application

1. T r u s t t h e E x p e r t s
WHITE PAPER
Performance Testing of Card Applications
Key Considerations
“If it is fast and ugly, they will use it and curse you; if it is
slow, they will not use it”
David Cheriton
Performance is critical to cards-applications from both the
acquiring and issuing perspectives. Organizations, especially
financial institutions, recognize repercussions of poor
performance on customers and end users. Amazon found that
every 100 milliseconds of latency cost them 1% in sales.’1
The cards landscape spans a highly complex architecture as it
involves a multitude of channels, applications and interfaces
across diverse platforms that are integrated with core
processing systems. As acquirers and issuers add new layers of
security along with advanced fraud detection and risk analysis
components, the burden on performance is ever increasing.
Although the average response time between card schemes
and banks is between 140 and 500 milliseconds, card-schemes
mandate further reduction in response times2,3,4
. To improve
cardholder experience at the point of sale, the issuer and
acquirer response times are limited to 10 seconds after which
transactions are timed out.
The overall objective of performance-improving measures is to
reduce response times by 25% to 45%. As a consequence, the
typical response times within each system should not exceed a
second if one were to consider the following factors:
• The network response
• Processing of authorization requests in the core systems
• Processing of decisions in risk or fraud interface systems
This white paper examines the key considerations, approaches,
activities and triggers that drive for performance testing and the
challenges and risks involved in improving the performance of
card systems and applications.
Introduction

2. T r u s t t h e E x p e r t s
2
WHITE PAPER
White Paper – Performance testing of card applications
Figure 1: Components in Performance Testing of a Cards Application
The figure depicts the typical components involved in performance testing in cards implementation. The figure
shows the transaction and data flows from the Front-End and Back-End.
• Front-End constitutes online authorizations and service requests through both self service and operator assisted
channels.
• Back-End consists of clearing, settlement and enterprise systems for GL accounting, Data warehousing and
interfaces from and to third parties.

3. T r u s t t h e E x p e r t s
White Paper – Performance testing of card applications 3
WHITE PAPER
Figure 2: Key Considerations for Performance Testing of a Cards Application
Phased and Modular Approach: The
performance testing of cards systems involves
multiple modules, applications and interfaces or
channels. Before testing the performance across
the implementation architecture, it is important to
assess the performance of each component. Any
risks due to bottlenecks in a particular component
which leads to degradation of the performance
across the entire implementation architecture can
adversely affect the testing schedule.
The key considerations for performance testing are summarized in Figure 2.
Approach to Planning of Performance testing
Acquiring / Issuing
Considerations
● Identify the nature of
business - acquiring and/or
issuing
● Ascertain the nature of On
Us/ not On Us transactions
for performance
Credit / Debit Considerations
● Identify the nature of portfolio
- credit, debit or both
● Ascertain how debit
authorizations are handled -
by cards application or
banking system or
frameworks like Base 24?
Online/ Batch Performance
Considerations
● Review authorizations & Web
access and service requests
from online perspective
● Analyze overall batch
process vs critical path, all
modules vs key modules
Card Technology
Considerations
● Assess usage of Magnetic
Stripe vs Chip cards for
authorizations involving
crytogram functions
● Assess PIN based
transactions involving PIN
translations vs signature
based transactions5
Interfaces / Surround systems
● Consider inclusion / non -
inclusion of interfaces and
surround systems - fraud,
rewards, behavioral risk etc
● Considerations for specific
fraud rules & triggers/reward
programs/ strategies and
case managements
Message Formats
● Consider specific local/
domestic / native message
formats when no off the shelf
tools are available
● Consider three leg
transactions - e.g.
authorizations, approval and
confirmation (200 210 202)

4. It is advisable to follow a phased and modular approach in testing the performance of cards application implementation as
depicted in Figure 3.
4White Paper – Performance testing of card applications
WHITE PAPER
I. Discovery sessions help determine the performance
testing objectives and should consider: -
• Results of prior performance testing (if any) on any of
the components
• List of performance tools used or available
• Acceptability on the usage of open source tools in the
environment
• Performance issues or bottlenecks experienced till then.
II. Tool Compatibility Study: Performance testing for a
card management system includes multiple protocols
and applications. Thus, the usage of suite of tools (open
source or commercial) would be effective for this
domain. It is advisable to ascertain the compatibility of
selected tools for the planned performance testing.
III. Volumetric Analysis and Data Profiling helps in
establishing the expected volume of transactions,
transaction mix and business scenarios for performance
Component-level performance testing should precede the End-to-End and Joint Performance Test phases.
However, component-level testing need not be repeated for cases where the results of a prior test prove that the
component already meets the performance objectives. In such circumstances, the components can be considered
only during End-to-End and Joint Performance Test phases.
Volumetric Analysis and Data Profiling is the cornerstone for conducting performance testing on cards
applications and help in arriving at the throughput requirements for the testing.
• Volumetric analysis helps in establishing the expected volume of transactions, transaction mix and business
scenarios for performance testing.
• Data profiling helps in determining the composition of data required to simulate a production like load for
performance testing.
For testing Online performance, the considerations should include expected volume of transactions by
Product-Scheme-Transaction, Source-Transaction Type, transaction arrival pattern and customer service
requests by channel, product, service request type, and frequency of usage in a day, month.
For testing Batch performance, the considerations should include portfolio mix, percentage of accounts cycling
in a batch, number of transactions or input files processed and output files generated along with business day
processing patterns.
Figure 3: A Phased Approach for Performance Testing of a Cards Application
Key Activities in Performance Testing of Cards Applications
Component
E.g., Specific Module
for authorizations
All modules part of
implementation
architecture E.g., Card
management system,
Fraud interface, Base
24 etc
All modules part of
implementation
architecture & client
systems: E.g., Card
management system,
Fraud interface, Base 24
& Bank host
Helps measure performance & bottlenecks of
each individual components/system
Helps measure degradation of performance
when other components are also active
Helps measure degradation of performance
when client components are also active
EndtoEnd
Joint

5. 5White Paper – Performance testing of card applications
WHITE PAPER
tests. It helps in determining the composition of data
required to simulate a production like load for
performance testing. This should be done to arrive at
the Work Load Matrix (WLM).
WLM is critical to ensuring that performance tests
conducted are as close as possible to the production
scenarios and anticipated growth. A structured
questionnaire to collect the details will help in preparing
the WLM for the performance testing.
IV. Strategize: Strategizing performance testing is the next
key activity. The performance test strategy would need
to define the performance objectives, scope, approach,
requisite environment and tools (established through a
tool compatibility study exercise conducted prior to
strategy), identify the risks and agree upon mitigation
plans. The roles and responsibilities for all stakeholders
will need to be clearly defined and agreed during
strategy phase.
V. Baseline/Benchmark Test must be carried out before
the data volumes are created. This will ensure that
bottlenecks (if any) are rectified before the start of the
actual Load test runs. The stability of the system and the
response times under minimal load is established
during this bench marking phase.
VI. Database volume creation is the next key activity after
the baseline or benchmark testing. The approach for
creating the database volume (creating new data or
migrating scrambled production data in to the
performance test environment) is typically finalized
during the strategy phase. It is important to ensure that
data volumes are created as per the portfolio and
transaction history requirements specified in the
strategy.
VII. Load/Volume/Stress/Endurance Tests are then
performed to assess the stability of system under
varying loads. The objectives of the each of these are
shown in the table.
Test Type Definition
Load Test Assessment of system performance and stability for various workload levels
Volume Test Assessment of system performance and stability for various account/ card/ transaction
database volume levels, vis-à-vis current database volume and future database
volumes based on growth expectations
Stress Test To find out the system break point by increasing the workload levels beyond the peak load
Endurance Test Assessment of reliability and stability of the SUT under various volumes & loads
sustained for a given duration of time
VIII. Reporting: Metrics would need to be provided at multiple levels (Web layer, Application layer, Database layer).
Recommendations to fine tune the system can be provided based on the system behavior during performance
testing.
Triggers for Performance Testing
A performance test may be required when:
• A new application module or component is introduced
• A new institution or product is introduced, leading to a
significant change in the existing portfolio, architecture,
or environment
• A change is introduced in the flow of information in the
existing systems
• An increase in Load/ Volume occurs where performance
testing was not carried out earlier.
• A significant change occurs in the
code base
• A tuning change is made to
address performance issues or
bottleneck identified previously
Challenges
Most of the challenges that arise in the performance testing
of cards applications originate from the complexity of the
implementations and the involvement of multiple
stakeholders. Critical challenges include:
• Availability of production-like test environment for
performance testing
• Simulation of real time business scenarios
• Simulation of high database volumes
• Pre-requisite activities for each of the parent and child
batch threads / programs
• Coordination with multiple stakeholders
- Application Development Team
- IT Infrastructure
- Project Teams
- Client Team(s)
- Third Party Team(s)

6. 5White Paper – Performance testing card applications
WHITE PAPER
• Clear PIN/PIN block creation for transaction
authorization requests
• Parameter configuration for performance testing
Risks
Risks involved in performance testing are:
• Incompatibility of the tools with the respective
components
• Tool limitations
• Non availability of test data as per the data
pre-requisites and volumes agreed for each run
• Data mismatch between different systems and host for
Joint performance testing
• Quality of migrated data
• Non availability of production like environment for
performance testing
• Data security
Disclaimer: All the documentation and other material contained herein is the property of Thinksoft Global Services and all intellectual
property rights in and to the same are owned by Thinksoft Global Services. You shall not, unless previously authorized by Thinksoft
Global Services in writing, copy, reproduce, market, license, lease or in any other way, dispose of, or utilize for profit, or exercise any
ownership rights over the same. In no event, unless required by applicable law or agreed to in writing, shall Thinksoft Global Services,
or any person be liable for any loss, expense or damage, of any type or nature arising out of the use of, or inability to use any material
contained herein. Any such material is provided “as is”, without warranty of any type or nature, either express or implied. All names,
logos are used for identification purposes only and are trademarks or registered trademarks of their respective companies.
For more details visit, www.thinksoftglobal.com
T r u s t t h e E x p e r t s
• Downtime during test execution due to problems in the
Test environment
• Non availability of interfaces
• Critical defects in the functional area, observed during
performance testing
• Conducting a joint performance testing without / before
completion of E2E performance test
These risks should be addressed across the project.
Conclusion
Performance testing of cards applications needs
comprehensive planning. It is also equally important that
the performance testing plan identifies the key challenges
and risks and includes strategies to mitigate them.
Bibliography
1
http://highscalability.com/latency-everywhere-and-it-costs-you-sales-how-crush-it
2
http://www.mastercard.com/us/company/en/newsroom/pr_mastercard_telefonica.html
3
http://www.cartes-bancaires.com/spip.php?article58
4
http://www.mastercard.com/us/company/en/newsroom/masterCard_upgrades_global_processing.html
5
http://www.andyorrock.com/2009/09/analysis-by-transaction-class-at-an-acquirer.html#tp