Testing Sap: Modern Methodology

Testing SAP:
Modern Testing Methodologies
Ethan Jewett

But first!

What’s the problem?

“Enterprise” project trade-offs

Value

Budget Risk

Duration

Software development trade-offs

Features

Budget Quality

Time

Client requests

Build Faster
And produce more Value
Cheaper
with less Risk

Remember…

Value

Budget Risk

Duration

Are there hidden assumptions?

Yes.

Methodology
People and Culture
Technology

Any other assumptions?
Project Preparation

Business Blueprint

Project Realization

Final Preparation

Go-live and Support

Technical Technical
Test Fix Transport Go Live
Design Build

Integration Live
Write and
& Regression
Write test scripts Training
run unit test Support
Test

The topic
Over the next hour and a half, we’ll talk about
how testing methodologies, techniques, and
technologies can help us

Change these underlying assumptions.
Enable modern project methodologies.
Control the basic project trade-off.

The focus
Project Preparation

Business Blueprint

Project Realization

Final Preparation

Go-live and Support

Technical Technical
Test Fix Transport Go Live
Design Build

Integration Live
Write and
& Regression
Write test scripts Training
run unit test Support
Test

The goal

More value
Faster
With less risk

(and more fun)

V-Model methodology

http://en.wikipedia.org/wiki/File:V-model.JPG

More parallel, less repetition


Faster


More iterative


Modern methodology
• Modern quality assurance and testing
– Leading practice in custom development, open
source, and agile projects
• Test driven development (TDD), behavior driven
development (BDD), readable tests, executable
requirements, continuous integration
– Overview of the SAP test automation technologies
– Road we're going to take through testing SAP

Leading Practice
• Context
• Questions and problems
• Approaches
• Tools

Context
The context of modern software development is
above all open, agile and decentralized.

Open source projects are usually the furthest
along this path, and as such these projects use
(or are forced to invent) tools that fit their
processes.

Openness
Requirements, specifications, issues reports, test
cases, source code, and project tools and
metrics are readily open and available to all
participants in the project.

(Not necessarily the same as “open source”.)

Open Collaboration within Corporations Using Software Forges -
http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges

Agility
The values of the Agile Manifesto:
– Individuals and interactions over processes and
tools
– Working software over comprehensive
documentation
– Customer collaboration over contract negotiation
– Responding to change over following a plan

http://agilemanifesto.org/

Decentralization
Open-source projects are decentralized by
nature, but most organizations today have
some element of decentralization

Challenges:
– Quality control
– Communication
– Maintaining commitment

Questions and problems
How do we track 100s
How do we or 1000s of bugs
enforce disciplined (including duplicates)?
testing?
How do we make
Do we use unit,
tests relevant?
functional, integration,
or acceptance testing? How does testing
Or all of them? integrate with
issue tracking?
How do we ensure that How do we
the build complies with determine the cause
Manual or
the tests? of test failure?
automatic
testing?

Approaches
Test Coverage
Test Automation
Test Driven Development
Behavior Driven Development
“Spec as Test” (the test/spec convergence)
Exploratory Testing
Continuous Integration

Test coverage
The percentage of code or development that is
tested.

In code, this might be measured by determining
if every branch of code can result in a failing
test condition (“Branch coverage”)

Wikipedia - http://en.wikipedia.org/wiki/Code_coverage

Test automation
• Accepted as gospel modern dev communities
– Regardless of how good they are at testing, accept
that they should automate as much as possible
– Can lead to ignoring non-automatable testing
• In the SAP world we haven’t even accepted
that full test coverage is desirable, much less
automation
Test automation pushback -
http://railspikes.com/2008/7/11/testing-is-overrated
http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-
theo.html
Automated testing story on SDN (never completed) -
https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103

A note on the following slides
I’m using bowling as an example because I’m
copying from the internet. I don’t know much
about bowling, so the examples are probably
wrong. My assumption is that a game is
usually 10 frames of two balls per frame. If
the last frame is a spare or a strike, you get at
most two additional frames.

Or something like that.

Test Driven Development (TDD)
Build

Design Test

Design Build
• Technical • Run tests • Run tests
specification • Write tests • Write code
• Failure!
based on to address
specs test failures
Design Test Test

TDD Example

require ‘test/unit’
require ‘test/unit/assertions’
require ‘bowling’

class TC_Bowling < Test::Unit::TestCase

def setup
bowling = Bowling.new
end

def gutter_game
20.times { bowling.hit(0) }
assert bowling.score == 0
end
end

Behavior Driven Development (BDD)
• Focus on the external behavior of the code
• Abstract away implementation details
• Additionally, BDD libraries tend to codify the
best practices of unit testing and TDD

BDD example

require ‘spec’

describe Bowling do

it quot;should score 0 for gutter gamequot; do

bowling = Bowling.new
20.times { bowling.hit(0) }
bowling.score.should == 0

end
end

“Spec as Test” – or writing features
• Recently, the Ruby community has begun
developing testing frameworks that are even
closer to natural language.
– Cucumber
• These frameworks wrap TDD and BDD
frameworks and allow for “business-writable”,
“business-readable”, executable test scripts.

Feature example
Feature (visible to the user) Implementation (not visible to user)

require ‘spec/expectations’
Scenario: Gutter game
Given I have bowled 20 balls
and I have knocked over 0 pins per ball Given /I have knocked over (d+) pins per ball/ do |pins|
When I check the score @pins_per_ball = pins
end
Then I should have 0 points
Given /I have bowled (d+) balls/ do |balls|
@bowling = Bowling.new
balls.times { @bowling.hit( @pins_per_ball ) }
end

Then /I should have (d+) points/ do |points|
@bowling.score.should == points
end

Feature example cont.
Note that we can now implement many more
tests with no more code/implementation:
Scenario: Perfect game Scenario: Lots of spares
Given I have bowled 12 balls Given I have bowled 24 balls
and I have knocked over 10 pins per ball and I have knocked over 5 pins per ball
When I check the score When I check the score
Then I should have 300 points Then I should have ??? Points

Scenario: Bad game Or maybe I need something a little
Given I have bowled 20 balls different....
and I have knocked over 2 pins per ball
When I check the score Scenario: Too long a game
Then I should have 40 points Given I have bowled 30 balls
and I have knocked over 0 pins per ball
Then I should receive an error

Side-by-side
TDD BDD Spec as Test
require ‘test/unit’ require ‘spec/expectations’ Scenario: Gutter game
require ‘test/unit/assertions’ require ‘bowling’
require ‘bowling’ Given I have bowled 20 balls
describe Bowling do and I have knocked over 0 pins per ball
class TC_Bowling < Test::Unit::TestCase
it quot;should score 0 for gutter gamequot; do When I view the score
def setup
bowling = Bowling.new bowling = Bowling.new Then I should have 0 points
end 20.times { bowling.hit(0) }
bowling.score.should == 0
def gutter_game
20.times { bowling.hit(0) } end
assert bowling.score == 0 end
end
end

TDD, BDD, and Test-as-Spec References
• Test Driven Design/Development
– http://en.wikipedia.org/wiki/Test-
driven_development
– http://www.agiledata.org/essays/tdd.html
• Behavior Driven Development
– http://behaviour-driven.org/
• Spec as test
– Cucumber - http://cukes.info/
– Rspec - http://rspec.info/
– http://www.pragprog.com/titles/achbd/the-rspec-
book

Exploratory Testing
• The practice of trying to break things
– Career security for klutzes
• Exploratory testing appears informal, but can
be structured and is a very important aspect
of software testing.
– Probably the most neglected form of testing in
open source projects

http://www.kohl.ca/blog/archives/000185.html
http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf

Continuous Integration
• The practice of automating not only your tests
but your full commit-build-test cycle
1. A commit new or changed code
2. Triggers a full system build
3. And an execution of the entire test suite

Cruisecontrol.rb -
http://rubyforge.org/projects/cruisecontrolrb
http://cruisecontrolrb.thoughtworks.com/
Hudson -
https://hudson.dev.java.net/
http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-how-
to-build-an-army-of-killer-robots-with-hudson-and-cucumber

Tools
Source version control (svn, git)
Testing libraries (rspec, cucumber, junit,
ABAPunit, jspec)
Continuous Integration tools (cruisecontrol.rb,
Hudson)
Issue tracking (Sourceforge, trac, Lighthouse,
Google Code)
Agile methodologies (Scrum, XP, etc.)

Modern methodology
• Modern quality assurance and testing
– Leading practice in custom development, open
source, and agile projects
• Test driven development (TDD), behavior driven
development (BDD), readable tests, executable
requirements, continuous integration
– Overview of the SAP test automation
technologies
– Road we're going to take through testing SAP

The SAP World
• Manual
• Unit testing
– ABAP Unit
– Java
• Functional testing
– eCATT

ABAP Unit
• Modeled on Java Unit
• Excellent (if a bit unwieldy) for traditional unit
test automation or classes
– Works well for TDD
• ABAP Objects (object-oriented ABAP)

• TDD Demo – ZBOWLING & ZBOWLING_TEST

http://help.sap.com/saphelp_nw70ehp1/helpdata/en/a2/8a1b602e85
8645b8aac1559b638ea4/frameset.htm

Non-SAP
• We can use just about anything via RFCs
• For Web Dynpros, BSPs, or Portal applications,
there are a lot of options using web-drivers
that can automatically drive browsers

• Address in depth in future sessions

The Road to the Future
• Developing techniques to support agile, open,
decentralized development in an SAP
landscape
• Using SAP tools and 3rd party tools
• Shorter, more focused and hands-on sessions

Start with this


Get more parallel, less repetitive

Spec = Test


Get faster

Test
Automation


Get iterative
Continuous

Integration

And end up here
(or somewhere similar)

http://en.wikipedia.org/wiki/File:Scrum_process.svg

General references for inclusion
• Open Collaboration within Corporations Using Software Forges - http://www.riehle.org/publications/2009/open-collaboration-within-corporations-using-software-forges/
• Continuous integration tools
– http://rubyforge.org/projects/cruisecontrolrb - http://cruisecontrolrb.thoughtworks.com/
– https://hudson.dev.java.net/
– http://www.softwarebloat.com/2008/11/19/continuous-integration-blueprints-how-to-build-an-army-of-killer-robots-with-hudson-and-cucumber
• Exploratory testing
– http://www.kohl.ca/blog/archives/000185.html
– http://www.kohl.ca/articles/ExploratoryTesting_MusicofInvestigation.pdf
• The ongoing revolution in software testing
– http://www.kaner.com/pdfs/TheOngoingRevolution.pdf
• ABAP Unit
– Spotlight on ABAP Unit Part 1 - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/1088
• Load testing
– http://dannorth.net/2007/02/monkey-business-value
• Joel Spolsky – 12 steps to better code - http://www.joelonsoftware.com/articles/fog0000000043.html
• Automated testing story on SDN - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4103
• JUnit - http://www.junit.org/
• Open Source
– http://www.paulgraham.com/opensource.html (What business can learn from open source)
• Watir
– http://wtr.rubyforge.org/
• Spec as test
– Cucumber - http://cukes.info/
– Rspec - http://rspec.info/
– http://www.pragprog.com/titles/achbd/the-rspec-book
• SAP testing
– http://www.beteoblog.com/beteo-alm-miniguides/software-quality/
– http://raa.ruby-lang.org/project/saprfc/
– Integration Tests in ABAP Development und Tool Support - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/7131
– XSLT - https://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/12173
• Test automation as panacea (not)
– http://railspikes.com/2008/7/11/testing-is-overrated
– http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html
• BDD - http://behaviour-driven.org/
• Blue Ruby - https://sap.na.pgiconnect.com/p16473929/ - http://www.slideshare.net/schmerdy/blue-ruby-sdn-webinar-1260181 -
https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/408a9a3b-03f9-2b10-b29c-f0a3374b19d8

Testing Sap: Modern Methodology

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Testing Sap: Modern Methodology

Similar to Testing Sap: Modern Methodology (20)

Recently uploaded

Recently uploaded (20)

Testing Sap: Modern Methodology