BDDP: Behaviour Driven Development for Performance (NZ Tech Day - Presentation 1)

www.planittesting.com
BDDP
A Discussion on Behaviour Driven
Development for Performance
Joel Deutscher
Performance Practice Lead
jd@planittesting.com
© Planit Software Testing 1

www.planittesting.com © Planit Software Testing 2
Continuous Performance
Testing

Meaningful Continuous
Performance Testing
And can BDD get us there?

My Background

It’s my opinion
Which is very likely to be incorrect

Why can’t I have a big go button?

There are lots of excuses!

A typical release based process

So what could we do?
We need to start somewhere

Automated Execution and Reporting

We actually created a go button!

Automated Reporting – SLA Based

Automated Reporting - Baseline

Automated Pass / Fail Notifications

Test Preparation

How do we get to continuous
performance testing?
… and make it meaningful

So what is left?

What are the cool kids doing?

TDD
Did we build the system right?
BDD
Did we build the right system?

“TDD took extra time upfront the resulting quality
was higher than teams that adopted a non-TDD
approach by an order of at least two times.”
Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies
Thirumalesh Bhat and Nachiappan Nagappan

Extra Time Upfront?

Why isn’t performance
considered earlier?
Requirements

• Functional Requirements are usually terrible
• Non-Functional Requirements are worse!
• Garbage in, Garbage Out
Requirements

• Requirements gathering is the hardest part of most
performance tests
• Manual and laborious process
• Why do we need to manually evaluate the risk if its
properly defined in requirements?
Why does it matter?

Why are they so bad?

• Requirements are written by people with no idea of what they want
• Performance is not well understood and is either:
• Assumed or considered the "Of course" stuff
• Considered but not well defined “I want it to be fast”
• Considered but of no actual benefit “Must not be slower than the
existing platform”
• Considered but pointless “I want it as fast as the industry average”
• Not considered at all
Its Complicated

OK Google, What are some
good NFRs?
This should be good…

“System Y shall process a minimum of 8 transactions per
“second”
and
“The system must handle 1,000 transactions per second”
This is what I found…

What's wrong with that?
Where do I begin.

The system must handle 1,000 transactions per second
• What should the response times should there be when it hits
1,000 transactions per second
• What type of transactions?
• Is it OK if the CPU maxes out to 100% while this is happening?
• How long should it be able to handle 1,000 per second for?
• Where should this be measured?
• What is an acceptable error rate if any?
Its not serving its purpose

“System Y shall process a minimum of 8 transactions per second”
and
Where did I find these examples?
“System Y shall process a minimum of 8 transactions per second”
and

How do you automate
performance testing without
good requirements?

Either through Non-Functional Requirements or Server Logs we
tend to get the information like:
We want to run a test that has:
• 1000 logins per hour
• 800 Transaction History Views per Hour
• 300 Fund Transfers
• 50 BPay Transactions
• 5 Credit Card Applications
Volumetrics are not requirements

Oh, and everything should be
as fast as the current release
… which is?

Could clear requirements
defined early be the answer?

The internet banking platform must:
• support 1000 valid user logins per hour
• with a 95 percentile response time of 1 second
• with a maximum response time of 3 seconds
• with response times measured at the web server
• with a maximum error rate of 1%
• with peak platform usage occurring on the system
• with system resource utilisation not breaching 60% on any
component
Login Page NFR

User Stories Can help

As a <type of user>,
I want <some goal>
so that <some reason>
User Stories

As an Account Holder,
I want to login to internet banking
So that I can do banking on the internet
User Stories

User Stories for NFRs

As an Account Holder,
I want to login to internet banking within 3 seconds
So that I can transact on the internet
User Stories

Title (one line describing the story)
Narrative:
As a [role]
I want [feature]
So that [benefit]
Acceptance Criteria: (presented as Scenarios)
Scenario 1: Title
Given [context]
And [some more context]...
When [event]
Then [outcome]
And [another outcome]...
User Stories and Acceptance Criteria

Its in the acceptance criteria?
But how detailed should we get for NFRs?

Scenario: Submit the contact form with correct data (declarative)
Given I am on the company contact page
When I submit the form with correct data
Then I should see the correct message
A declarative (informative) example

Scenario: Submit the contact form with correct data (imperative)
Given I am on the company contact page
When I set "What is your name?" to “Joel"
And I set “What is your query to?" to “I wish to discuss performance testing"
And I set "What is your email address to “jd@planittesting.com"
And I click "Submit"
Then I should see the message "Thanks! The Planit team will be in contact shortly."
An imperative (communicative) example

Imperative

Scenario: Submit the contact form with correct data (imperative)
Given I am on the company contact page with 1000 concurrent users on the site
When I set "What is your name?" to “Joel"
And I set “What is your query to?" to “I wish to discuss performance testing"
And I set "What is your email address to “jd@planittesting.com"
And I click "Submit"
Then I should see the message "Thanks! The Planit team will be in contact shortly.“
And the response should be returned within 2 seconds 95 percent of the time
And the response should not take more than 5 seconds to return
And the error rate should be less than 1%
And the server utilisation should be less than 60% for all components
Scenario with Performance Requirements

Problem Solved?
Now we just need to automate it….

PROS
• Performance Considered Early
• Developers can Design for
Performance
• Seems easier to specify
CONS
• Its not exactly elegant
• Potentially gets in the way of
delivering functionality
• Very easy to get wrong
Its not exactly simple

Moon Shot?

Scenario: Weather Service
Given the webservice named Weather
When I call the method GetCurrentTemperature with a valid postcode 1000 times in an hour
Then I expect it to return an containing the current temperature
And the response should be returned within 2 seconds 95 percent of the time
And the response should not take more than 5 seconds to return
And the error rate should be less than 1%
And the server utilisation should be less than 60% for all components
Weather Micro Service

Meaningful Continuous
Performance Testing
Can BDD help get us there?

• Repeatable Environment
• Managing Test Data
• SLA vs Baseline Comparison
• Trend Analysis
There are some other considerations

• BDD for Performance is still a work in progress
• User Story / Acceptance Test Format is not ideal in every
situation
• Works well for simple web services, less so for socialised
performance tests
• System Resource Utilisation Metrics are important, though
shouldn’t be in every acceptance scenario
The Story has Just Begun

What has been your
experience?
Please come and speak to me throughout the day

BDDP: Behaviour Driven Development for Performance (NZ Tech Day - Presentation 1)

Recomendados

Recomendados

Más contenido relacionado

Último

Último (20)

Destacado

Destacado (20)

BDDP: Behaviour Driven Development for Performance (NZ Tech Day - Presentation 1)

Notas del editor