The document discusses performance testing and how harnessing the cloud can help address challenges. It defines performance testing and outlines the typical process. Conducting performance tests on-premises can be challenging due to lab setup and maintenance, scaling users, and testing from different locations. These challenges can be overcome by using cloud-based performance testing services which provide flexible, scalable resources and pay-as-you-go models. Visual Studio Online is highlighted as one such cloud-based solution.

1. TenXLabs
Performance Testing: Harnessing
the Cloud
-Shuchita Ahuja Kaushik

2. Agenda
www.tenxlabs.com 2
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

3. Agenda
www.tenxlabs.com 3
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

4. What is Performance Testing
www.tenxlabs.com 4
In software engineering, performance testing is the testing performed to determine
how a system performs in terms of responsiveness and stability under a particular
workload. Performance testing is an Integral component of Quality Assurance
Volume Testing
Scalability Testing
Endurance
Testing
Longevity
Testing
Spike Testing
• Performance is a common term – Used in
different flavours
• Check performance from different aspects –
different organizations can have different
performance indicators that they might want to
performance test the application against and
attain.

5. 5
What is Performance Testing
Other tests falling under
bigger umbrella of
performance -
 Capacity Testing
 Endurance Testing
 Spike Testing
Image Source - http://www.qualitykiosk.com/
www.tenxlabs.com

6. Typical Performance Test Procedure
www.tenxlabs.com 6
Typically all performance testing activities follow a particular
sequence of steps at the execution level.
Image Source - http://www.sabritech.com//
There is a very
close analogy
between a
performance
testing process
and a functional
testing process –
the difference
being the clarity
of the expected
behavior.

7. Performane Testing Tools
www.tenxlabs.com 7
Both Commercial and open source tools are available
•JMeter
•OpenSTA
•Grinder
•Silk Performer
•Load Runner
•Rational Performance Tester
•Microsoft VSTS
•Fiddler
•WebLoad

8. Agenda
www.tenxlabs.com 8
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

9. Cloud – Concept and Capability
• Cloud computing is defined as a type of computing that relies on sharing
computing resources rather than having local servers or personal devices to
handle applications. Cloud computing is comparable to grid computing, a
type of computing where unused processing cycles of all computers in a
network are harnesses to solve problems too intensive for any stand-alone
machine
• A concept of systems, infrastructures and software being dynamically
available to users in a real time while being present on Virtual machines
located somewhere on the globe.
• Choosing to use the cloud frees the user from setting up, installing and
maintaining the resources.
• Using Cloud is like using mobile phone in a tariff plan. You pay for as much
as you use, you pay for as long as you use.
www.tenxlabs.com 9
Definition and Usage Models

10. Cloud – Concept and Capability
www.tenxlabs.com 10
Cloud Computing – All Offerings are services
The service is
managed by a
Cloud portal
where the
user can
choose,
manage and
pay for the
units he
consumes
and also that
he plans to
consume.

11. Commercial Services Offerings
www.tenxlabs.com 11

12. Types of Cloud
www.tenxlabs.com 12

13. Agenda
www.tenxlabs.com 13
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

14. Performance Testing – On Premises: Challenges
www.tenxlabs.com 14
Some challenges for Realistic tests executed on premises-
• Lab Set-up – Hardware, software and tools installation and up keeping -
Infrastructure is one of the biggest challenges with performance and load
testing.
• Testing large numbers of virtual users - scalability
• Testing outside the firewall
• Testing from multiple geographical locations
• There might be a need to execute performance test for many builds.
Some issues commonly heard –
• It just takes too much time to configure the tool
• Site works fine in US, but does not work from EU
• It’s too expensive to setup, maintain and update a test lab
• It takes too much time and effort to setup a test lab
• Test phases last for only 2 months: Underutilized Test Boxes
• I don’t have a snap shot of test environment in case it crashes

15. Agenda
www.tenxlabs.com 15
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

16. Performance Testing - A Cloud Service Offering
www.tenxlabs.com 16
Move the performance testing lab to cloud
• Organizations have changed the way
they offer performance testing
solutions!
• Organizations have changed the way
they buy performance testing solutions!
• Performance labs are available on the
cloud
• Customers can use the labs on demand
and pay for the usage and scalability.

17. Performance Testing - A Cloud Service Offering
www.tenxlabs.com 17
To overcome the issues discussed, the
availability of cloud is being harnessed to
run routine tests
Lab up-keeping and maintenance is no more
a challenge with the organization.
Scalable virtual systems are available to
host the performance lab – with latest tool,
system perquisites, and hardware
configuration
Scripts reusability is offered by most
performance testing cloud services.
Uniformity can be attained on tool behavior
and application response across the globe.
Load can be generated from different
geographical locations across the globe. On demand high scalability with respect to
load generation can be achieved.
Advantages
Pre-launch performance testing rounds can
be performed

18. Performance Testing - A Cloud Service Offering
www.tenxlabs.com 18
Advantages - Summary
• Infinite computing power
• Affordable
• Pay as you scale and go model
• Saving the cost of software licenses
• Quick and easy set up
• Ready to use environment
• Ability to simulate multiple geographies

19. Performance Testing with Cloud – Tools available
www.tenxlabs.com 19
• Visual Studio Online – by Microsoft
• SandStorm from Impetus
• LoadStorm
• Grinder In The Cloud
• CloudTest by SOASTA
• VMLogix and Skytap
• BrowserMob
• KeyNote
• Pylot - Web Load Testing from Amazon Elastic Compute Cloud
• HP LoadRunner
• Xceptance LoadTest
• Load-Intelligence
• TrafficSpike from Intechnica

20. Performance Testing with Cloud – Cloud Model
www.tenxlabs.com 20

21. Performance Testing with Cloud – Hybrid Model
www.tenxlabs.com 21

22. Cloud Based performance Testing - Challenges
www.tenxlabs.com 22
• Data Security
• Intranet Applications
• Access
• Network Latency
• Monitoring and diagnostic tools
• Test Data Management
• Test Script creation and validation

23. Recommended -
www.tenxlabs.com 23
• Performance goals are uncertain
• Actual user base is globally distributed
• Performance-centric development is required
• Source builds are located on cloud
• Costs of set ups are unconvincing to budgets!
Choosing to go the cloud way still recommended when-

24. Agenda
www.tenxlabs.com 24
What is Performance Testing1
Cloud- Concept and Capability2
Performane Testing – On Premises: Challenges3
Performance Testing - A Cloud Service Offering4
Cloud Load Testing – Demo with Visual Studio Online5

25. Visual Studio Online
www.tenxlabs.com 25
• Quick: Just one click gets a performance lab in the cloud, no
setup needed. Scale up or down based on your needs.
• Free load testing: Gives 20,000 virtual user minutes every
month to load test at no extra charge.
• Real-time charts and graphs: Application's performance can be
watched while the tests run. Results can be shared when your
tests are done.
• Test where the users are: Test from different locations to reduce
latency and simulate local conditions.
• Capacity on demand: Load testing service is elastic and
dynamic. E.g. weather for a load requirement of 200 concurrent
virtual users or 20,000 – one will be able to queue a test as per
his requirements.
• Same tests work on premises and on the cloud: The tests
authored on-premises just run on the service
Cloud Performance Testing Solution with many advantages

26. Visual Studio Online
• A quick demo to execute tests on cloud
www.tenxlabs.com 26

27. Q
&
Awww.tenxlabs.com 27

28. TenXLabs
Block B6, Vindhya C6
IIIT Campus, Gachibowli
Hyderabad, 500032
India
www.tenxlabs.com