1.
How
to
Run
a
Test
for
1M
Users
Backend
Tes6ng
(Mobile,
Web
&
API)
Alon Girmonsky
Founder & CEO
785 Castro Street, Mountain View, CA 94041 | 1.855.445.2285 | info@blazemeter.com

2.
Are
You
Sure?
• Question: Do you really need to test a
concurrency level of 1M users?
Answer:
a. I only need 1M named users
b. I really expect to have 1M concurrent open
sessions
c. I don’t know
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

3.
Things
to
Consider
• The Script
• The Performance Lab
• Running the Test
• Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

4.
Things
to
Consider
• The Script
• The Performance Lab
• Running the Test
• Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

5.
How
Do
I
Write
the
Script?
Comprehensive ScripGng at Your FingerGps
HTTP
/
HTTPS
Login
/
Signup
Form
POSTing
/
Credit
Card
Transac6ons
Rest
/
Web
Services
/
JSON
Parameters
Extrac6on
(
JSON
/
RegEx
/
Xpath
)
Mobile
Mongo,
MSSQL,
Oracle
SQL,
MySQL
Mul6ple
User
Profiles
JMeter is the most commonly used open source web, mobile and API
backend tesGng soluGon. BlazeMeter is 100% compaGble with JMeter
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

6.
Apache
JMeter
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

7.
Things
to
Consider
ü The Script
• The Performance Lab
• Running the Test
• Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

8.
The
Performance
Lab
Challenges
(1/3)
• Resources. Are they:
– Sufficient?
– Strong?
– Dedicated?
• Bandwidth
– Can I saturate my system?
– Is the load network stable?
• Throughput
– Can I generate enough hits per second?
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

9.
The
Performance
Lab
Challenges
(2/3)
• Load Distribution
– Will the load be evenly distributed across
resources?
• CDNs
– Am I testing a CDN (e.g. Akamai)?
– Is the test realistic?
– Will I avoid traffic shaping?
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

10.
The
Performance
Lab
Challenges
(3/3)
• DDOS
– Can I avoid triggering DDOS mechanisms?
• Load Balancers
– Can I handle the load balancers properly (e.g.
close TCP connections on time)?
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

11.
The
JMeter
Super
Computer
4,400
CPU
Cores
&
16.5
TB
of
Memory
=
1M
Users
Load
Engine
Type:
m3.xlarge
CPU:
4
Cores
Memory:
15
GB
BW:
100Mbps
Max
TPS:
1K
1,000
X
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

12.
The
JMeter
Super
Computer
4,400
CPU
Cores
&
16.5
TB
of
Memory
=
1M
Users
Load
Cluster
=
Console
+
10
Engines
1+10
X
Load
Engine
Type:
m3.xlarge
CPU:
4
Cores
Memory:
15
GB
BW:
100Mbps
Max
TPS:
1K
CPU:
44
Cores
Mem:
165
GB
BW:
1
Gbps
M/TPS:
10K
1,000
X
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

13.
The
JMeter
Super
Computer
4,400
CPU
Cores
&
16.5
TB
of
Memory
Super
Cluster
=
100
X
Load
Clusters
=
1M
Users
Load
Cluster
=
Console
+
10
Engines
1+10
X
Load
Engine
Type:
m3.xlarge
CPU:
4
Cores
Memory:
15
GB
BW:
100Mbps
Max
TPS:
1K
CPU:
44
Cores
Mem:
165
GB
BW:
1
Gbps
M/TPS:
10K
100
X
CPU:
4,400
Cores
Mem:
16.5
TB
BW:
100
Gbps
M/TPS:
1M
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

14.
Performance
Lab
in
a
Box
Up
to
1M
Users,
Mul6
GEO,
100%
Apache
JMeter
Compa6ble
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

15.
Things
to
Consider
ü The Script
ü The Performance Lab
• Running the Test
• Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

16.
Running
the
Test
Challenges
• Time-to-test should be minutes
• Synchronizing all the servers to start at once
• Crunching the data
– 4GB of data per hour
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

17.
Running
a
Test
With
1M
Users
Time-­‐to-­‐Test
<
2
Minutes
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

18.
Things
to
Consider
ü The Script
ü The Performance Lab
ü Running the Test
• Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

19.
What
To
Look
For
"We'd like to squeeze in a load test for XXXX's biggest event of the year, the
Fantasy Football season. Our best opportunity to execute this is tonight at
3-7am EST. We'd like to generate a constant 40,000 hits/sec for about 1
hour. .."
“This would have been a disaster for our
most important day of the year. BlazeMeter
really saved our bacon here. …. and kudos on
the success!"
Tradi6onally
this
would
have
taken
weeks
and
cost
$$$
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

20.
Running
a
Test
With
1M
Users
Time-­‐to-­‐Test
<
2
Minutes
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

21.
What
To
Look
For
The
Load
Sensi6vity
Point
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

22.
NewRelic
Integra6on
Capture Monitoring Data
Capture Load Testing Data
Ask the Test Data any Question
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

23.
Things
to
Consider
ü The Script
ü The Performance Lab
ü Running the Test
ü Understanding the Results
• Test Iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

24.
Run
Load
Test
Analyze
Client
Performance
Analyze
Server
Performance
Performance
Op6miza6on
Run
as
Many
Itera6ons
as
Needed
–
and
More!
Update
Back
End
Code
/
Infrastructure
• Load + Results
• Look outside & within
– App KPIs
– Load KPIs
• Run hundreds of
iterations
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

25.
Tes6ng
Can
Save
Your
Day!
Made
for
Developers,
Time-­‐to-­‐Test
<
2
min
• Run dozens to hundreds of test iterations
• Run once, then automate
• Test continuously
Ø A Superior Product
Ø A Faster Release Cycle
785 Castro Street, Mountain View, CA 94041|1.855.445.2285|info@blazemeter.com @blazemeter

26.
THANK
YOU
FOR
YOUR
TIME!
Q
&
A
For more information:
BlazeMeter.com/blog/
info@blazemeter.com
@BlazeMeter