API Testing: The heart of functional testing" with Bj Rollison

TEST Huddle
TEST HuddleOnline Marketing Executive (Content Creator)
Welcome to the 2013
www.eurostarconferences.com
This webinar is presented by Bj Rollison and he will discuss:
‘API Testing: The heart of functional testing’
Join us on Twitter! @esconfs #esconfs
Overview
• What is API Testing?
• Why should we do API testing?
• What’s the difference between Unit and API Tests?
• How to we approach API testing?
• What types of bugs will we find?
www.eurostarconferences.com
@esconfs
#esconfs
What Is API Testing?
• API – Application Programming Interface
• Implementation hidden from calling program
• Component & integration levels
• Individual API
• End-to-End scenarios
• API Testing – testing below the UI
• Programmatically simulate data or
control flow scenarios
• Focus on functionality;
not on behavior or customer experience
www.eurostarconferences.com
@esconfs
#esconfs
http://www.faqs.org/faqs/
software-eng/testing-
faq/section-14.html
What Is API Testing?
• Functional testing where it lives!
• Separation of business logic and “look and feel”
www.eurostarconferences.com
@esconfs
#esconfs
Cloud Services
Core business logic
functionality is here!
Customer experience
and behavior is here!
Why Should We Do API Testing
• Agile software development
• Team approach – testing engaged early
• Drive testing upstream
• Testing in parallel with development, not after the fact
• Automated test cost reduction
• Stable API interfaces == automation stability
• Reduced business costs
• Reduced build breaks (pre-check-in suite)
• Refocus testing objectives at system level or through GUI
• Software behavior and customer experience
www.eurostarconferences.com
@esconfs
#esconfs
When Is API Testing A Good Investment
• Business logic is encapsulated in classes
rather than within forms
• No graphical user interface –
headless testing
• External APIs used by 3rd party developers
• Internal APIs used by 1st party application
developers
www.eurostarconferences.com
@esconfs
#esconfs
Unit Testing vs. API Testing
Unit Testing
• Owned by development
• Limited in scope
• Mocked dependencies
• “Does it work by itself”
• Usually ran before check-in
API Testing
• Owned by test
• Broader in scope
• Mocked and real dependencies
• “Does it play well with others”
• Pre-check-in suites
• Ran after build is created
www.eurostarconferences.com
@esconfs
#esconfs
Approaches To API Testing
It’s not either black box or white box;
it’s both black and white!
www.eurostarconferences.com
@esconfs
#esconfs
Testing an API as a “Black Box”
www.eurostarconferences.com
@esconfs
#esconfs
INPUT PARAMETERS
• lpLocaleName [in, optional]
Pointer to a locale name, or
one of the predefined
variants
• LCType [in]
The locale information to
retrieve.
• lpLCData [out, optional]
Pointer to a buffer in which
this function retrieves the
requested locale
information.
• cchData [in]
Size, in characters, of the
data buffer indicated by
lpLCData.
OUTPUT VALUE
• Returns the number of
characters retrieved in the
locale data buffer if successful
and cchData is a nonzero
value.
• The function returns 0 if it
does not succeed.
• Error codes
ERROR_INSUFFICIENT_BUFFER
ERROR_INVALID_FLAGS
ERROR_INVALID_PARAMETER
int GetLocaleInfoEx(
_In_opt_ LPCWSTR lpLocaleName,
_In_ LCTYPE LCType,
_Out_opt_ LPWSTR lpLCData,
_In_ int cchData
);
API testing starts as soon as the API
interface is spec’d out!
Testing an API as a “Black Box”
• Input argument values
• Boundary values, equivalent set samples
• Parameter combinations
• Combinatorial (n-wise) testing
• Error Validation
• Exception handling, and output data w/error codes
• Properties
• Getters and setters
• Scenario testing
• Sequence of API calls, E-2-E scenario testing
www.eurostarconferences.com
@esconfs
#esconfs
Testing an API as a “White Box”
BOOL KillProcessByName(char *szProcessToKill)
{
HANDLE hProcessSnap;
HANDLE hProcess;
PROCESSENTRY32 pe32;
DWORD dwPriorityClass;
hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0);
if(hProcessSnap == INVALID_HANDLE_VALUE){
return( FALSE );
}
pe32.dwSize = sizeof(PROCESSENTRY32);
if (!Process32First(hProcessSnap, &pe32)) {
CloseHandle(hProcessSnap);
return( FALSE );
}
do{
if(!strcmp(pe32.szExeFile,szProcessToKill)){
hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID);
TerminateProcess(hProcess,0);
CloseHandle(hProcess);
}
}while(Process32Next(hProcessSnap,&pe32));
…..
www.eurostarconferences.com
@esconfs
#esconfs
Testing an API as a “White Box”
• Code Review
• Early and effective bug detection
• Code coverage analysis
• Structural testing, identify holes
• Fault injection
• Break points, resource constraints, system failures
• Traces
• Trace statements useful for debugging
www.eurostarconferences.com
@esconfs
#esconfs
• Robust automated test cases 5 stages of API test design
• Setup
• Initialize data, create objects, start services, etc.
• Execution
• Steps to exercise API or scenario, also logging
• Verification
• Oracles to evaluate execution outcome
• Reporting
• Pass, Fail, or Blocked
• Clean up
• Pre-test state
API Test Design Principles
www.eurostarconferences.com
@esconfs
#esconfs
What Types Of Bugs Will API Testing Find
• Fails to handle error conditions gracefully
• Missing or duplicate functionality
• Unused flags
• Not implemented errors
• Stress
• Reliability
• Security
• Multi-threading issues
• Inconsistent error handling
• Performance
• Improper errors/warnings to caller
• Incorrect handling of valid argument values
www.eurostarconferences.com
@esconfs
#esconfs
Key Points
• API testing benefits both the development and testing organization.
• API testing is both a black box and white box activity
• Finds some types of bugs more
efficiently; but…can’t find all bugs!
• Doesn’t evaluate “look and feel”
• Can change focus of GUI automation
• Requires testers with additional skills
• Not for everyone!
www.eurostarconferences.com
@esconfs
#esconfs
Questions or Comments…
• For more thoughts on API testing see…
• www.testingmentor.com/imtesty
• If you have questions or comments…
• Email – Bj.Rollison@TestingMentor.com
• Twitter - @TestingMentor
www.eurostarconferences.com
@esconfs
#esconfs
Thanks for listening!
@esconfs
#esconfs
www.eurostarconferences.com
Go to www.eurostarconferences.com for more information
1 de 17

Recomendados

B4USolution_API-Testing por
B4USolution_API-TestingB4USolution_API-Testing
B4USolution_API-Testingb4usolution .
1.3K vistas31 diapositivas
An Introduction To Automated API Testing por
An Introduction To Automated API TestingAn Introduction To Automated API Testing
An Introduction To Automated API TestingSauce Labs
4.1K vistas23 diapositivas
API Testing por
API TestingAPI Testing
API TestingBikash Sharma
7.9K vistas9 diapositivas
Api testing por
Api testingApi testing
Api testingHamzaMajid13
841 vistas19 diapositivas
Api Testing por
Api TestingApi Testing
Api TestingVishwanath KC
1.4K vistas20 diapositivas
API Test Automation por
API Test Automation API Test Automation
API Test Automation SQALab
1.9K vistas26 diapositivas

Más contenido relacionado

La actualidad más candente

Test Design and Automation for REST API por
Test Design and Automation for REST APITest Design and Automation for REST API
Test Design and Automation for REST APIIvan Katunou
883 vistas104 diapositivas
API Testing. Streamline your testing process. por
API Testing. Streamline your testing process.API Testing. Streamline your testing process.
API Testing. Streamline your testing process.Andrey Oleynik
795 vistas84 diapositivas
Api testing por
Api testingApi testing
Api testingKeshav Kashyap
30.2K vistas21 diapositivas
API TESTING por
API TESTINGAPI TESTING
API TESTINGSijan Bhandari
595 vistas23 diapositivas
Postman. From simple API test to end to end scenario por
Postman. From simple API test to end to end scenarioPostman. From simple API test to end to end scenario
Postman. From simple API test to end to end scenarioHYS Enterprise
2.5K vistas20 diapositivas
API Testing for everyone.pptx por
API Testing for everyone.pptxAPI Testing for everyone.pptx
API Testing for everyone.pptxPricilla Bilavendran
909 vistas24 diapositivas

La actualidad más candente(20)

Test Design and Automation for REST API por Ivan Katunou
Test Design and Automation for REST APITest Design and Automation for REST API
Test Design and Automation for REST API
Ivan Katunou883 vistas
API Testing. Streamline your testing process. por Andrey Oleynik
API Testing. Streamline your testing process.API Testing. Streamline your testing process.
API Testing. Streamline your testing process.
Andrey Oleynik795 vistas
Postman. From simple API test to end to end scenario por HYS Enterprise
Postman. From simple API test to end to end scenarioPostman. From simple API test to end to end scenario
Postman. From simple API test to end to end scenario
HYS Enterprise2.5K vistas
Introduction to APIs & how to automate APIs testing with selenium web driver? por BugRaptors
Introduction to APIs & how to automate APIs testing with selenium web driver?Introduction to APIs & how to automate APIs testing with selenium web driver?
Introduction to APIs & how to automate APIs testing with selenium web driver?
BugRaptors406 vistas
API Testing Presentations.pptx por ManmitSalunke
API Testing Presentations.pptxAPI Testing Presentations.pptx
API Testing Presentations.pptx
ManmitSalunke119 vistas
How to Automate API Testing por Bruno Pedro
How to Automate API TestingHow to Automate API Testing
How to Automate API Testing
Bruno Pedro7.9K vistas
4 Major Advantages of API Testing por QASource
4 Major Advantages of API Testing4 Major Advantages of API Testing
4 Major Advantages of API Testing
QASource8K vistas
Reasons To Automate API Testing Process por QASource
Reasons To Automate API Testing ProcessReasons To Automate API Testing Process
Reasons To Automate API Testing Process
QASource979 vistas
RESTful API Testing using Postman, Newman, and Jenkins por QASymphony
RESTful API Testing using Postman, Newman, and JenkinsRESTful API Testing using Postman, Newman, and Jenkins
RESTful API Testing using Postman, Newman, and Jenkins
QASymphony 5.5K vistas
Postman.ppt por ParrotBAD
Postman.pptPostman.ppt
Postman.ppt
ParrotBAD2.9K vistas
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P... por Postman
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
POST/CON 2019 Workshop: Testing, Automated Testing, and Reporting APIs with P...
Postman2.3K vistas
API Testing With Katalon Studio por Knoldus Inc.
API Testing With Katalon StudioAPI Testing With Katalon Studio
API Testing With Katalon Studio
Knoldus Inc.842 vistas
API Testing Using REST Assured with TestNG por Siddharth Sharma
API Testing Using REST Assured with TestNGAPI Testing Using REST Assured with TestNG
API Testing Using REST Assured with TestNG
Siddharth Sharma1K vistas

Similar a API Testing: The heart of functional testing" with Bj Rollison

Practices and Tools for Building Better APIs por
Practices and Tools for Building Better APIsPractices and Tools for Building Better APIs
Practices and Tools for Building Better APIsPeter Hendriks
943 vistas28 diapositivas
An introduction to the API for OnTime for IBM por
An introduction to the API for OnTime for IBMAn introduction to the API for OnTime for IBM
An introduction to the API for OnTime for IBMontimesuite
1.1K vistas91 diapositivas
Developing Brilliant and Powerful APIs in Ruby & Python por
Developing Brilliant and Powerful APIs in Ruby & PythonDeveloping Brilliant and Powerful APIs in Ruby & Python
Developing Brilliant and Powerful APIs in Ruby & PythonSmartBear
1.7K vistas32 diapositivas
Practices and tools for building better API (JFall 2013) por
Practices and tools for building better API (JFall 2013)Practices and tools for building better API (JFall 2013)
Practices and tools for building better API (JFall 2013)Peter Hendriks
4.5K vistas29 diapositivas
Practices and tools for building better APIs por
Practices and tools for building better APIsPractices and tools for building better APIs
Practices and tools for building better APIsNLJUG
569 vistas29 diapositivas
Always up to date, testable and maintainable documentation with OpenAPI por
Always up to date, testable and maintainable documentation with OpenAPIAlways up to date, testable and maintainable documentation with OpenAPI
Always up to date, testable and maintainable documentation with OpenAPIGOG.com dev team
669 vistas47 diapositivas

Similar a API Testing: The heart of functional testing" with Bj Rollison(20)

Practices and Tools for Building Better APIs por Peter Hendriks
Practices and Tools for Building Better APIsPractices and Tools for Building Better APIs
Practices and Tools for Building Better APIs
Peter Hendriks943 vistas
An introduction to the API for OnTime for IBM por ontimesuite
An introduction to the API for OnTime for IBMAn introduction to the API for OnTime for IBM
An introduction to the API for OnTime for IBM
ontimesuite1.1K vistas
Developing Brilliant and Powerful APIs in Ruby & Python por SmartBear
Developing Brilliant and Powerful APIs in Ruby & PythonDeveloping Brilliant and Powerful APIs in Ruby & Python
Developing Brilliant and Powerful APIs in Ruby & Python
SmartBear1.7K vistas
Practices and tools for building better API (JFall 2013) por Peter Hendriks
Practices and tools for building better API (JFall 2013)Practices and tools for building better API (JFall 2013)
Practices and tools for building better API (JFall 2013)
Peter Hendriks4.5K vistas
Practices and tools for building better APIs por NLJUG
Practices and tools for building better APIsPractices and tools for building better APIs
Practices and tools for building better APIs
NLJUG569 vistas
Always up to date, testable and maintainable documentation with OpenAPI por GOG.com dev team
Always up to date, testable and maintainable documentation with OpenAPIAlways up to date, testable and maintainable documentation with OpenAPI
Always up to date, testable and maintainable documentation with OpenAPI
GOG.com dev team669 vistas
Building Better Web APIs with Rails por All Things Open
Building Better Web APIs with RailsBuilding Better Web APIs with Rails
Building Better Web APIs with Rails
All Things Open1.4K vistas
Api fundamentals por AgileDenver
Api fundamentalsApi fundamentals
Api fundamentals
AgileDenver1.5K vistas
Effective API Lifecycle Management por SmartBear
Effective API Lifecycle Management Effective API Lifecycle Management
Effective API Lifecycle Management
SmartBear1.2K vistas
10 Useful Testing Tools for Open Source Projects @ TuxCon 2015 por Peter Sabev
10 Useful Testing Tools for Open Source Projects @ TuxCon 201510 Useful Testing Tools for Open Source Projects @ TuxCon 2015
10 Useful Testing Tools for Open Source Projects @ TuxCon 2015
Peter Sabev5.9K vistas
GPCE16: Automatic Non-functional Testing of Code Generators Families por Mohamed BOUSSAA
GPCE16: Automatic Non-functional Testing of Code Generators FamiliesGPCE16: Automatic Non-functional Testing of Code Generators Families
GPCE16: Automatic Non-functional Testing of Code Generators Families
Mohamed BOUSSAA859 vistas
API workshop: Introduction to APIs (TC Camp) por Tom Johnson
API workshop: Introduction to APIs (TC Camp)API workshop: Introduction to APIs (TC Camp)
API workshop: Introduction to APIs (TC Camp)
Tom Johnson3.2K vistas
Infinum Android Talks #13 - Developing Android Apps Like Navy Seals by Ivan Kušt por Infinum
Infinum Android Talks #13 - Developing Android Apps Like Navy Seals by Ivan KuštInfinum Android Talks #13 - Developing Android Apps Like Navy Seals by Ivan Kušt
Infinum Android Talks #13 - Developing Android Apps Like Navy Seals by Ivan Kušt
Infinum668 vistas
Writing Well Abstracted Automation on Foundations of Jello por Dan Cuellar
Writing Well Abstracted Automation on Foundations of JelloWriting Well Abstracted Automation on Foundations of Jello
Writing Well Abstracted Automation on Foundations of Jello
Dan Cuellar380 vistas

Más de TEST Huddle

Why We Need Diversity in Testing- Accenture por
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- AccentureTEST Huddle
1.9K vistas17 diapositivas
Keys to continuous testing for faster delivery euro star webinar por
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar TEST Huddle
887 vistas30 diapositivas
Why you Shouldnt Automated But You Will Anyway por
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway TEST Huddle
1K vistas38 diapositivas
Being a Tester in Scrum por
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in ScrumTEST Huddle
1.5K vistas25 diapositivas
Leveraging Visual Testing with Your Functional Tests por
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional TestsTEST Huddle
698 vistas36 diapositivas
Using Test Trees to get an Overview of Test Work por
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test WorkTEST Huddle
884 vistas32 diapositivas

Más de TEST Huddle(20)

Why We Need Diversity in Testing- Accenture por TEST Huddle
Why We Need Diversity in Testing- AccentureWhy We Need Diversity in Testing- Accenture
Why We Need Diversity in Testing- Accenture
TEST Huddle1.9K vistas
Keys to continuous testing for faster delivery euro star webinar por TEST Huddle
Keys to continuous testing for faster delivery euro star webinar Keys to continuous testing for faster delivery euro star webinar
Keys to continuous testing for faster delivery euro star webinar
TEST Huddle887 vistas
Why you Shouldnt Automated But You Will Anyway por TEST Huddle
Why you Shouldnt Automated But You Will Anyway Why you Shouldnt Automated But You Will Anyway
Why you Shouldnt Automated But You Will Anyway
TEST Huddle1K vistas
Being a Tester in Scrum por TEST Huddle
Being a Tester in ScrumBeing a Tester in Scrum
Being a Tester in Scrum
TEST Huddle1.5K vistas
Leveraging Visual Testing with Your Functional Tests por TEST Huddle
Leveraging Visual Testing with Your Functional TestsLeveraging Visual Testing with Your Functional Tests
Leveraging Visual Testing with Your Functional Tests
TEST Huddle698 vistas
Using Test Trees to get an Overview of Test Work por TEST Huddle
Using Test Trees to get an Overview of Test WorkUsing Test Trees to get an Overview of Test Work
Using Test Trees to get an Overview of Test Work
TEST Huddle884 vistas
Big Data: The Magic to Attain New Heights por TEST Huddle
Big Data:  The Magic to Attain New HeightsBig Data:  The Magic to Attain New Heights
Big Data: The Magic to Attain New Heights
TEST Huddle522 vistas
Will Robots Replace Testers? por TEST Huddle
Will Robots Replace Testers?Will Robots Replace Testers?
Will Robots Replace Testers?
TEST Huddle1.1K vistas
TDD For The Rest Of Us por TEST Huddle
TDD For The Rest Of UsTDD For The Rest Of Us
TDD For The Rest Of Us
TEST Huddle648 vistas
Scaling Agile with LeSS (Large Scale Scrum) por TEST Huddle
Scaling Agile with LeSS (Large Scale Scrum)Scaling Agile with LeSS (Large Scale Scrum)
Scaling Agile with LeSS (Large Scale Scrum)
TEST Huddle761 vistas
Creating Agile Test Strategies for Larger Enterprises por TEST Huddle
Creating Agile Test Strategies for Larger EnterprisesCreating Agile Test Strategies for Larger Enterprises
Creating Agile Test Strategies for Larger Enterprises
TEST Huddle1.3K vistas
Is There A Risk? por TEST Huddle
Is There A Risk?Is There A Risk?
Is There A Risk?
TEST Huddle547 vistas
Are Your Tests Well-Travelled? Thoughts About Test Coverage por TEST Huddle
Are Your Tests Well-Travelled? Thoughts About Test CoverageAre Your Tests Well-Travelled? Thoughts About Test Coverage
Are Your Tests Well-Travelled? Thoughts About Test Coverage
TEST Huddle901 vistas
Growing a Company Test Community: Roles and Paths for Testers por TEST Huddle
Growing a Company Test Community: Roles and Paths for TestersGrowing a Company Test Community: Roles and Paths for Testers
Growing a Company Test Community: Roles and Paths for Testers
TEST Huddle1.2K vistas
Do we need testers on agile teams? por TEST Huddle
Do we need testers on agile teams?Do we need testers on agile teams?
Do we need testers on agile teams?
TEST Huddle1.7K vistas
How to use selenium successfully por TEST Huddle
How to use selenium successfullyHow to use selenium successfully
How to use selenium successfully
TEST Huddle1.9K vistas
Testers & Teams on the Agile Fluency™ Journey por TEST Huddle
Testers & Teams on the Agile Fluency™ Journey Testers & Teams on the Agile Fluency™ Journey
Testers & Teams on the Agile Fluency™ Journey
TEST Huddle693 vistas
Practical Test Strategy Using Heuristics por TEST Huddle
Practical Test Strategy Using HeuristicsPractical Test Strategy Using Heuristics
Practical Test Strategy Using Heuristics
TEST Huddle1.6K vistas
Thinking Through Your Role por TEST Huddle
Thinking Through Your RoleThinking Through Your Role
Thinking Through Your Role
TEST Huddle928 vistas
Using Selenium 3 0 por TEST Huddle
Using Selenium 3 0Using Selenium 3 0
Using Selenium 3 0
TEST Huddle9.5K vistas

Último

iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... por
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...Bernd Ruecker
50 vistas69 diapositivas
Future of AR - Facebook Presentation por
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook PresentationRob McCarty
62 vistas27 diapositivas
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda... por
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...ShapeBlue
120 vistas13 diapositivas
Microsoft Power Platform.pptx por
Microsoft Power Platform.pptxMicrosoft Power Platform.pptx
Microsoft Power Platform.pptxUni Systems S.M.S.A.
80 vistas38 diapositivas
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti... por
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...ShapeBlue
98 vistas29 diapositivas
Uni Systems for Power Platform.pptx por
Uni Systems for Power Platform.pptxUni Systems for Power Platform.pptx
Uni Systems for Power Platform.pptxUni Systems S.M.S.A.
61 vistas21 diapositivas

Último(20)

iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... por Bernd Ruecker
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
Bernd Ruecker50 vistas
Future of AR - Facebook Presentation por Rob McCarty
Future of AR - Facebook PresentationFuture of AR - Facebook Presentation
Future of AR - Facebook Presentation
Rob McCarty62 vistas
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda... por ShapeBlue
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
ShapeBlue120 vistas
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti... por ShapeBlue
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
ShapeBlue98 vistas
Extending KVM Host HA for Non-NFS Storage - Alex Ivanov - StorPool por ShapeBlue
Extending KVM Host HA for Non-NFS Storage -  Alex Ivanov - StorPoolExtending KVM Host HA for Non-NFS Storage -  Alex Ivanov - StorPool
Extending KVM Host HA for Non-NFS Storage - Alex Ivanov - StorPool
ShapeBlue84 vistas
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f... por TrustArc
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc Webinar - Managing Online Tracking Technology Vendors_ A Checklist f...
TrustArc160 vistas
KVM Security Groups Under the Hood - Wido den Hollander - Your.Online por ShapeBlue
KVM Security Groups Under the Hood - Wido den Hollander - Your.OnlineKVM Security Groups Under the Hood - Wido den Hollander - Your.Online
KVM Security Groups Under the Hood - Wido den Hollander - Your.Online
ShapeBlue181 vistas
Data Integrity for Banking and Financial Services por Precisely
Data Integrity for Banking and Financial ServicesData Integrity for Banking and Financial Services
Data Integrity for Banking and Financial Services
Precisely78 vistas
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ... por ShapeBlue
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
Live Demo Showcase: Unveiling Dell PowerFlex’s IaaS Capabilities with Apache ...
ShapeBlue85 vistas
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue por ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlueCloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
CloudStack Managed User Data and Demo - Harikrishna Patnala - ShapeBlue
ShapeBlue94 vistas
Confidence in CloudStack - Aron Wagner, Nathan Gleason - Americ por ShapeBlue
Confidence in CloudStack - Aron Wagner, Nathan Gleason - AmericConfidence in CloudStack - Aron Wagner, Nathan Gleason - Americ
Confidence in CloudStack - Aron Wagner, Nathan Gleason - Americ
ShapeBlue88 vistas
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive por Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... por ShapeBlue
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
ShapeBlue117 vistas
The Role of Patterns in the Era of Large Language Models por Yunyao Li
The Role of Patterns in the Era of Large Language ModelsThe Role of Patterns in the Era of Large Language Models
The Role of Patterns in the Era of Large Language Models
Yunyao Li80 vistas
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... por ShapeBlue
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
ShapeBlue132 vistas
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ... por ShapeBlue
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...
ShapeBlue123 vistas
State of the Union - Rohit Yadav - Apache CloudStack por ShapeBlue
State of the Union - Rohit Yadav - Apache CloudStackState of the Union - Rohit Yadav - Apache CloudStack
State of the Union - Rohit Yadav - Apache CloudStack
ShapeBlue253 vistas

API Testing: The heart of functional testing" with Bj Rollison

  • 1. Welcome to the 2013 www.eurostarconferences.com This webinar is presented by Bj Rollison and he will discuss: ‘API Testing: The heart of functional testing’ Join us on Twitter! @esconfs #esconfs
  • 2. Overview • What is API Testing? • Why should we do API testing? • What’s the difference between Unit and API Tests? • How to we approach API testing? • What types of bugs will we find? www.eurostarconferences.com @esconfs #esconfs
  • 3. What Is API Testing? • API – Application Programming Interface • Implementation hidden from calling program • Component & integration levels • Individual API • End-to-End scenarios • API Testing – testing below the UI • Programmatically simulate data or control flow scenarios • Focus on functionality; not on behavior or customer experience www.eurostarconferences.com @esconfs #esconfs http://www.faqs.org/faqs/ software-eng/testing- faq/section-14.html
  • 4. What Is API Testing? • Functional testing where it lives! • Separation of business logic and “look and feel” www.eurostarconferences.com @esconfs #esconfs Cloud Services Core business logic functionality is here! Customer experience and behavior is here!
  • 5. Why Should We Do API Testing • Agile software development • Team approach – testing engaged early • Drive testing upstream • Testing in parallel with development, not after the fact • Automated test cost reduction • Stable API interfaces == automation stability • Reduced business costs • Reduced build breaks (pre-check-in suite) • Refocus testing objectives at system level or through GUI • Software behavior and customer experience www.eurostarconferences.com @esconfs #esconfs
  • 6. When Is API Testing A Good Investment • Business logic is encapsulated in classes rather than within forms • No graphical user interface – headless testing • External APIs used by 3rd party developers • Internal APIs used by 1st party application developers www.eurostarconferences.com @esconfs #esconfs
  • 7. Unit Testing vs. API Testing Unit Testing • Owned by development • Limited in scope • Mocked dependencies • “Does it work by itself” • Usually ran before check-in API Testing • Owned by test • Broader in scope • Mocked and real dependencies • “Does it play well with others” • Pre-check-in suites • Ran after build is created www.eurostarconferences.com @esconfs #esconfs
  • 8. Approaches To API Testing It’s not either black box or white box; it’s both black and white! www.eurostarconferences.com @esconfs #esconfs
  • 9. Testing an API as a “Black Box” www.eurostarconferences.com @esconfs #esconfs INPUT PARAMETERS • lpLocaleName [in, optional] Pointer to a locale name, or one of the predefined variants • LCType [in] The locale information to retrieve. • lpLCData [out, optional] Pointer to a buffer in which this function retrieves the requested locale information. • cchData [in] Size, in characters, of the data buffer indicated by lpLCData. OUTPUT VALUE • Returns the number of characters retrieved in the locale data buffer if successful and cchData is a nonzero value. • The function returns 0 if it does not succeed. • Error codes ERROR_INSUFFICIENT_BUFFER ERROR_INVALID_FLAGS ERROR_INVALID_PARAMETER int GetLocaleInfoEx( _In_opt_ LPCWSTR lpLocaleName, _In_ LCTYPE LCType, _Out_opt_ LPWSTR lpLCData, _In_ int cchData ); API testing starts as soon as the API interface is spec’d out!
  • 10. Testing an API as a “Black Box” • Input argument values • Boundary values, equivalent set samples • Parameter combinations • Combinatorial (n-wise) testing • Error Validation • Exception handling, and output data w/error codes • Properties • Getters and setters • Scenario testing • Sequence of API calls, E-2-E scenario testing www.eurostarconferences.com @esconfs #esconfs
  • 11. Testing an API as a “White Box” BOOL KillProcessByName(char *szProcessToKill) { HANDLE hProcessSnap; HANDLE hProcess; PROCESSENTRY32 pe32; DWORD dwPriorityClass; hProcessSnap = CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS, 0); if(hProcessSnap == INVALID_HANDLE_VALUE){ return( FALSE ); } pe32.dwSize = sizeof(PROCESSENTRY32); if (!Process32First(hProcessSnap, &pe32)) { CloseHandle(hProcessSnap); return( FALSE ); } do{ if(!strcmp(pe32.szExeFile,szProcessToKill)){ hProcess = OpenProcess(PROCESS_TERMINATE,0, pe32.th32ProcessID); TerminateProcess(hProcess,0); CloseHandle(hProcess); } }while(Process32Next(hProcessSnap,&pe32)); ….. www.eurostarconferences.com @esconfs #esconfs
  • 12. Testing an API as a “White Box” • Code Review • Early and effective bug detection • Code coverage analysis • Structural testing, identify holes • Fault injection • Break points, resource constraints, system failures • Traces • Trace statements useful for debugging www.eurostarconferences.com @esconfs #esconfs
  • 13. • Robust automated test cases 5 stages of API test design • Setup • Initialize data, create objects, start services, etc. • Execution • Steps to exercise API or scenario, also logging • Verification • Oracles to evaluate execution outcome • Reporting • Pass, Fail, or Blocked • Clean up • Pre-test state API Test Design Principles www.eurostarconferences.com @esconfs #esconfs
  • 14. What Types Of Bugs Will API Testing Find • Fails to handle error conditions gracefully • Missing or duplicate functionality • Unused flags • Not implemented errors • Stress • Reliability • Security • Multi-threading issues • Inconsistent error handling • Performance • Improper errors/warnings to caller • Incorrect handling of valid argument values www.eurostarconferences.com @esconfs #esconfs
  • 15. Key Points • API testing benefits both the development and testing organization. • API testing is both a black box and white box activity • Finds some types of bugs more efficiently; but…can’t find all bugs! • Doesn’t evaluate “look and feel” • Can change focus of GUI automation • Requires testers with additional skills • Not for everyone! www.eurostarconferences.com @esconfs #esconfs
  • 16. Questions or Comments… • For more thoughts on API testing see… • www.testingmentor.com/imtesty • If you have questions or comments… • Email – Bj.Rollison@TestingMentor.com • Twitter - @TestingMentor www.eurostarconferences.com @esconfs #esconfs
  • 17. Thanks for listening! @esconfs #esconfs www.eurostarconferences.com Go to www.eurostarconferences.com for more information