This session provides an overview of how the Oracle Quality Assurance team tests Oracle E-Business Suite. It covers the main areas that you should consider during functional testing, approaches for new feature and regression testing, how to reduce the test script generation and execution time, experiences on capturing and presenting metrics to showcase the ROI of the testing investment, leveraging automation for testing Oracle E-Business Suite applications, and more.

2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Oracle E-Business Suite Best
Practices
Keeping Up with Change…
Confidential – Oracle Internal/Restricted/Highly Restricted
Presented with
Mazen Arar
Senior Director, Oracle
Sarvesh Tiwari
Vice President, Genpact
September 21, 2016

3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Confidential – Oracle Internal/Restricted/Highly Restricted 3

4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
4Confidential – Oracle Internal/Restricted/Highly Restricted

5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
EBS Evolution
Change is Constant
• Change happens every day
• In today’s business world, it is the only thing that is
consistent
• Long-term plans may become outdated and possibly
obsolete within months
• Business plans must be reviewed, updated, and
revised constantly
5Confidential – Oracle Internal/Restricted/Highly Restricted

6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
EBS Evolution
Oracle E-Business Suite in the Digital World
Cloud Social
Mobile
IoTBig Data
Talent
Mgmt
Revenue
Mgmt
Recruiting
Innovation
MgtCommerce
Financial
Reporting
Global
Trade Mgmt
Enterprise
Planning
Service
Transportation
Mgmt
Sales
Marketing
6Confidential – Oracle Internal/Restricted/Highly Restricted

7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
7Confidential – Oracle Internal/Restricted/Highly Restricted

8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness
Challenging Facts
• Continuous development calls for continuous testing
• With ongoing EBS innovation, all teams need to be aligned
• Different types of activities require different strategy for
testing
• QA teams have to be pro-active not re-active
• Focusing on the present and planning for the future
8Confidential – Oracle Internal/Restricted/Highly Restricted

9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness
Ongoing EBS Innovation
• New features / Products
– EBS 12.1
– EBS 12.2
– Modern / Simplified UI
• Mobile
• In Memory Applications
• Big Data - Information Discovery
• Cloud
• IoT
9Confidential – Oracle Internal/Restricted/Highly Restricted

10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Testing Readiness
Strategy
10
Smartphone Applications
In Memory Applications
Information Discovery
Cloud
IoT
Confidential – Oracle Internal/Restricted/Highly Restricted

11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
11Confidential – Oracle Internal/Restricted/Highly Restricted

12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Where Do You Start?
12
Upgrades
Confidential – Oracle Internal/Restricted/Highly Restricted

13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
13Confidential – Oracle Internal/Restricted/Highly Restricted

14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
• What is the task on hand
• Available resources
– Release Content Document
• Information about new or changed functionality
• List of bugs included
– Functional Impact Assessment document
• Functional changes
• Functional impacts
• Testing recommendations
14
Plan
Confidential – Oracle Internal/Restricted/Highly Restricted

15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
15
Evaluate
• Task on hand
Review
• Documentation
Scope
• Key business flows
• Impacted Flows
• Usage patterns
Plan
Confidential – Oracle Internal/Restricted/Highly Restricted

16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
16Confidential – Oracle Internal/Restricted/Highly Restricted

17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Basics …
17
Execute
Entry / Exit Criteria
Follow A Testing Methodology Based on Task
Theme Based Exploratory Role based
End to end flow
based
Mark Test Flows With A Priority
Basic Sanity check Detailed
Confidential – Oracle Internal/Restricted/Highly Restricted

18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 18
Shorter Release
Cycle
Automation
coverage
Rapid Addition of
Features
Faster
validation
Security
Testing data
security and
data leaks
Integration
Test
integrations
for Cloud &
On Premise
Multi Tenancy
Simulated
customers
and roles
Testing
Recommendations & Tools
Cloud Testing …
Execute
Accelerate
testing with
automation
tools
Test in short
sprints
Authentication,
Authorization,
Input
Validation,
PWD, etc ..
End to end
flows between
the cloud apps
and integrating
apps including
on premise ERP
Ensure
isolation of
individual
customer
applications
and data

19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 19
Complex
Architecture
Component
specific &
integration
testing
Complex Data
Controlled
statistical data
with pre-
defined
patterns
Security
Data security,
leaks, and
authentication
Validation
Insights &
predictions VS.
Pre-validated
data
Performance and
Scalability
Performance
and stress
testing
Recommendations & Tools
IoT Testing …
Execute
Test different
services
separately and
run end to end
application
flows
Feed data into
the IOT apps to
see if system
can decode the
time series
data
Authentication,
Authorization,
Input
Validation,
PWD, etc ..
Feed same
data to IoT
apps & data
mining tool to
compare
results
Test with
volume data &
multiple users

20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Good Practice
• Be creative / Innovative
– Review results
– Learn from results
• Hot spots
• Safe spots
– Plan next cycle with a new flavor
• Shift focus to hot spots
• Certify safe spots
– Improve scoping
20
Execute
Confidential – Oracle Internal/Restricted/Highly Restricted

21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
Automate
Execute
Plan
21Confidential – Oracle Internal/Restricted/Highly Restricted

22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
• Manual Vs. Automation
• Reduce testing time & effort
• Repeatable!
22
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Recommendations & Tools
23
Anyone
Critical + Repeatable Flows
It is 2016
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

24. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
• Load and Performance Style Testing
• Web and Oracle Application Testing Accelerators
• Smart Integration with EM Diagnostics
• Test Planning, Requirements and Test Cases
• Defects Tracking
• Reporting
• Functional and Regression Testing
• Web and Oracle Application Testing Accelerators
• Oracle Flow Builder and Starter Pack
Oracle Application Testing Suite
Overview
Oracle Functional Testing Suite for
Oracle Applications
Oracle Load Testing Suite
for Oracle Applications
Oracle Test Manager
25
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Functional Testing Suite for Oracle Applications
Oracle Flow Builder and EBS Pre-built Test Content
• Use Flow Builder to Generate Test Scripts
• Start with EBS Pre-Built Test Content
• Create and Maintain Your Test Scripts
– Add Your Flows & Data
– Extend with More Components
– Support Customizations
– Non technical resources
Record
Manually
Generate
Test
Automation
Scripts
OpenScript
Record & Playback
+ Advanced Scripting
Flow Builder
Framework to combine
flows and data to generate
test scripts
EBS Pre-Built
Test Content
Your
Data
Your
Flows
OracleFunctionalTestingSuite
forOracleApplications
EBS Quality Assurance Team
Reduced Automation Time by 60%
26
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content
Starter Pack
• Commonly used base functionality
• For creating high level certification flows
• Broad coverage to jump start the testing
For Both 12.1.3 and 12.2
2000+ Components and 200+ Flows across EBS
ProcurementFinancials
ManufacturingLogistics Service
Master Data
Management
Order
Management
27
Asset
Lifecycle
Management
Projects HCM
3600+ components and 230+ flows across EBS
Logistics ManufacturingFinancials
ProjectsProcurement
Product
Information
Management Service
Human Capital
Management
Order
Management
Asset
Lifecycle
Management
Automate

27. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Pre-Built Test Content
Advanced Pack
• Niche products which stand alone and deep
functionality
• For creating detailed and complex test flows
• Significant number of components above Starter Pack
(95% coverage)
For Both 12.1.3 and 12.2
28
10 product families / products across EBS
Incentive
Compensation
Discrete
Manufacturing
Channel
Revenue
Management
Project
ManagementProcurement
Process
Manufacturing
Property
Manager Service
Lease /Finance
Management
Asset
Lifecycle
Management
Automate

28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Web Services & APIs Support
• What are Web Services & APIs?
– Stored procedures that enable you to insert and update data into Oracle Applications
• Automate Web Services & APIs using OFB
– No need for an in-depth knowledge of PL/SQL
• Advantage
– Automated mass upload into the application
– Scripts created are more stable and robust than UI
29
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Automate
Oracle Flow Builder: EBS Pre-Built Test Content
Advanced Pack includes content for API/Web Services
• Drastic reduction in test execution time
• Hybrid flows, where you can execute a flow with API
and UI scenarios
• Verification of data in UI, Database, or output data
after interface execution
Enhanced Content for 12.1.3
30
875 components and 48 flows across EBS
Logistics ManufacturingFinancials
ProjectsProcurement
Product
Information
Management Service
Trade
Management
Order
Management
Asset
Lifecycle
Management
API
Content for
EBS 12.1.3

30. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Flow Builder: EBS Web Services & APIs Support
• UI VS. API
– Value added through execution
• In Oracle Project Management:
– Creating a Project takes 1 - 1.5 hrs using UI mode VS. 26 seconds using OFB API
– Create Task Assignments takes 15 min using UI mode VS. 8 seconds using OFB API
31
Automate
Confidential – Oracle Internal/Restricted/Highly Restricted

31. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
DemoOracle Flow Builder - APIs
Oracle Confidential – Internal/Restricted/Highly Restricted 32

32. 33Qualcomm Proprietary and Confidential
Testimonial from Qualcomm
• Created ~115 custom components in Oracle Flow Builder which saved huge effort during design phase.
• Created ~160 automated scripts which cover ~1470 business process scenarios in past ~2years
• Metrics against manual testing that is qualified for automation:
◦ Time Savings with Automation ~50%
◦ Automation Script Coverage ~50%
• Automated scripts used in below activities which reduced a significant manual effort during major projects:
◦ Regression cycle
◦ Quick Build test
◦ Month-end activity close
◦ Pre-production checkout
◦ User Acceptance test
◦ Go-live checkout
◦ Emergency test in PRD Support

33. 34Qualcomm Proprietary and Confidential
Testimonial from Qualcomm
• Custom Solutions:
◦ We use QC for test management. Created a plug-in in order to interface OATS scripts directly to update results in QC.
◦ For accelerated test execution created customized framework in order to execute same OATS script in parallel for
different business scenarios

34. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
EBS Evolution
Testing Readiness
Recommendations & Tools
Customer Story
1
2
3
4
35Confidential – Oracle Internal/Restricted/Highly Restricted

35. Testing Oracle E-Business Suite
Best Practices & Lessons Learned
Sarvesh Tiwari
Lead Solutions Architect
Sarvesh.Tiwari@genpact.com @ideating

36. 37© 2015 Copyright Genpact. All Rights Reserved.
68,000 +
employees
800+
Clients
(One Third of Fortune
500)
30
languages
17
delivery
countries
70
delivery
centers
50 +
Oracle Go Lives /
year
BPMAnalyticsDigital
Technolog
y
ReengineerConsulting
Outcome driven, Process-
led ERP
Analytic Solutions & Insights to support
executive decisions
Using Lean Six Sigma for process
improvements & Transformations
Robotic and Digital technologies While running some of the most
complex business processes for
our clients
Deep domain consulting for business
blueprints
AT-A-GLANCE
Founded by GE in 1997
NYSE (“G”)
$2.5 Billion (FY2015)

37. 38© 2015 Copyright Genpact. All Rights Reserved.
FUTURE IS HERE, NOW
Computing Everywhere … EBS on wearables?
Internet of Things … sensor to boardroom
Advanced Pervasive Invisible Analytics … In-memory
Context Rich Applications UI … tried Endeca?
Cloud-Client Architecture … mobile native
Infinitely Elastic Infra … cloud compute…

38. 39© 2015 Copyright Genpact. All Rights Reserved.
IT Practices
Business
Technology
17% CIOs say they are fast enough for the pace of their business
“IT Speed: The Crisis and the Savior of the Enterprise” Forrester Research
CRISIS

39. 40© 2015 Copyright Genpact. All Rights Reserved.
ORGANIZATIONS ARE LEARNING FROM SILICON VALLEY
LEAN START UP | AGILE DEV + OPS
TEST DRIVEN DEVELOPMENT | CONTINUOUS TESTING

40. 41© 2015 Copyright Genpact. All Rights Reserved.
HOWEVER, MOST FACE SPEED BUMPS IN TESTING
45% of effort is spent in creation of prerequisites and
documenting, rather than testing the outcomes
53% ERP Programs execute EBS testing as “part
time” function of the business end users
62% report having no automation in EBS Functional
Testing
43% CRITICAL PATH OF A TYPICAL EBS PROJECT IS IN TESTING

41. 42© 2015 Copyright Genpact. All Rights Reserved.
LEARNING FROM A JOURNEY
US headquartered global leader in data center solutions, cloud
management, performance management, services management,
workload automation and virtualization lifecycle management products
& services
• Complex global EBS footprint (Fin, SCM, CRM)
• 58 SOBs, 40 countries, 13 localizations
• 20+ modules, 30+ integrated systems
• Heavily customized (3,500+ CEMLI)
• 5 custom-built bespoke bolt-on modules
• Products shifting to SaaS
• Failed attempts to upgrade to R12.2
• Heavily manual-dependent testing effort
• Painful release cycle time: want to establish monthly
releases
CONTEXT CHALLENGE
CLIENT

42. 43© 2015 Copyright Genpact. All Rights Reserved.
5 TOP CHALLENGES THEY FACED IN AUTOMATING EBS TESTING
• Domain Knowledge and Testing knowledge mutually exclusive impacting speed, quality
and effectiveness of testing
• Too much cycle time it takes to develop automated test scripts
• Always Obsolete: Scripting based test cases /data become easily obsolete therefore
require huge ongoing effort in keeping scripts up to date.
• Difficult to discover / reuse the test data, scripts, scenarios from release to release, project
to project
• Difficult to scope and tailor (Test some / Test whole) for individual releases

43. 44© 2015 Copyright Genpact. All Rights Reserved.
JOURNEY
Oct
2014
Dec
2014
Mar
2015
June
2015
Sep
2015
Nov
2015
Jan
2016
R12 Upgrade Kick
Off
OATS Pilot
1st OATS MVP
Ready – Used in
Unit Test
Progressive
Additions in
OATS
R122.2 Gone
Live
Monthly
Regression
Pack
Test Iterations Ongoing Optimization

44. 45© 2015 Copyright Genpact. All Rights Reserved.
RESULTS
•50% faster test iterations
•40% Less business effort in test cycles: Focus on value-added
test
•60+% reduction in regression cycle times
•Production releases: 8 /year (from earlier 4 /year)
•Adoption of Agile and Test-driven development methods
•100% Sanity, 80% Regression, 60% “projects” EBS testing
automated

45. 46© 2015 Copyright Genpact. All Rights Reserved.
LESSONS LEARNED: BALANCE OF MANUAL & AUTOMATED
Low Testing
Effort
Repeatable / Rule Based
High Testing
Effort
High Intuitive
1 2
• Automate the pre-requisites, support SMEs to focus
on “real” validation
• “Auto-gathering” of test evidence / progress: End to
End scenarios
• Role Based Testing
3
Limited value: diminishing returns of automation
Low Hanging Fruits
Automate to sustainable extent
X

46. 47© 2015 Copyright Genpact. All Rights Reserved.
BUSINESS-DRIVEN DOMAIN KNOWLEDGE LED
Script-free;
Drag & Drop;
Keywords;
Tags

47. 48© 2015 Copyright Genpact. All Rights Reserved.
PRE-BUILT TEST CONTENT: AGILE JUMP START
Business Scenarios
• End to end test scenarios covering
detailed critical business
processes, use cases and industry
nuances
• Context specific functional
scenarios
• COA change
• R12.2 upgrade

48. 49© 2015 Copyright Genpact. All Rights Reserved.
DESIGN FOR MAINTAINABILITY: MINIMIZE OBSOLESCENCE
•EBS Configuration Half Life:
•Data / Set Ups: 3 - 6 months
•Functionality/Logic: 6 - 12 months
•UI / major patch: 9 – 18 months
•Continuously updated library of test content
•Not “Screen Placement” but Object-Action driven
•“Keywords > Component > Flow” | upward bubbling of changes
•Changes in scripts for EBS customizations

49. 50© 2015 Copyright Genpact. All Rights Reserved.
HOT SPOTS VS SAFE SPOTS
•Test Clustering: L1 … L5 end to end scripts
•Iterative Learning: Progressive adaptability – Hot Spots vs Safe Spots
•Impact Traceability to Test Scenarios
•Test Driven Enhancement Management
•Continuous Optimization: “Regression Packs”, “Sanity Packs”
•Test Data
•Role based Focal Data Points (PO data vs Accounting data)
•Data Randomization, Variability
•Purpose-driven Data Generator scripts

50. Thank You
Sarvesh Tiwari
Sarvesh.Tiwari@genpact.com | @ideating

51. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Questions
52Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted