Finding the Right Testing Tool for the Job

Finding the Right Testing
Tool for the Job
Sebastian Bergmann and
Ciaran McNulty

3 Dimensions of Testing
Goal - why we are writing the test
Scope - how much of the system is involved in the test
Form - how we express the test

3 4 Dimensions of Testing
Goal - why we are writing the test
Scope - how much of the system is involved in the test
Form - how we express the test
Time - when we write the test

What we will talk about
— Characterisation Tests
— Acceptance Tests
— Integration Tests
— Unit Tests

Characterisation Tests
Goals:
— Capture existing behaviour
— Find out when behaviour changes

Scopes:
— Often at UI level
— Sometimes at object/service level

Times:
— Always after implementation

Best practice:
— Treat these tests as a temporary measure
— Use a tool that makes it easy to create tests
— Expect pain and suffering

Form: Behat + MinkExtension builtin steps
Scenario: Product search returns results
Given I am on "/"
And I fill in "search" with "Blue jeans"
When I press "go"
Then I should see "100 Results Found"

Form: PHPUnit + phpunit-mink-trait
class SearchTest extends PHPUnit_Framework_TestCase
{
use phpunitminkTestCaseTrait;
public function testProductSearchReturnsResult()
{
$page = $this->visit('http://example.com/');
$page->fillField('search', 'Blue Jeans');
$page->pressButton('go');
$this->assertContains('100 Results Found', $page->getText());
}
}

Form: Selenium IDE

Form: Ghost Inspector

Form: PHPUnit + de-legacy-fy
See docs at on GitHub at
sebastianbergmann/de-legacy-fy

Acceptance Tests
Goals:
— Match system behaviour to business requirements
— Get feedback on proposed implementations
— Understand business better
— Document behaviour for the future

Acceptance Tests
Scopes:
— At a UI layer
— At an API layer
— At the service layer
— Lower level may be too disconnected

Acceptance Tests
Times:
— Before implementation
— Before commitment (as long as it's not expensive?)
— Hard to write in retrospect

Acceptance Tests
Best practices:
— Get feedback on tests early
— Make them readable, or have readable output, for the
intended audience
— Apply for the smallest scope first
— Apply to core domain model first
— Minimise end-to-end tests

Acceptance Tests
Form: Behat at service level
Scenario: Sales tax is applied to basket
Given "Blue Jeans" are priced as €100 in the catalogue
When I add "Blue Jeans" to my shopping basket
Then the basket total should be €120

class BasketContext implements Context
{
public function __construct()
{
$this->catalogue = new InMemoryCatalogue();
$this->basket = new Basket($catalogue);
}
/**
* @Given :productName is/are priced as :cost in the catalogue
*/
public function priceProduct(ProductName $productName, Cost $cost)
{
$this->catalogue->price($productName, $cost);
}
//...
}

class BasketContext implements Context
{
//...
/**
* @When I add :productName to my shopping basket
*/
public function addProductToBasket(ProductName $productName)
{
$this->basket->add($productName);
}
/**
* @Then the basket total should be :cost
*/
public function checkBasketTotal(Cost $cost)
{
assert($this->basket->total == $cost->asInt());
}
}

Acceptance Tests
Form: PHPUnit at service level
class BasketTest extends PHPUnit_Framework_TestCase
{
public function testSalesTaxIsApplied()
{
$catalogue = new InMemoryCatalogue();
$basket = new Basket($catalogue);
$productName = new ProductName('Blue Jeans');
$catalogue->price($productName, new Cost('100'));
$basket->add($productName);
$this->assertEquals(new Cost('120'), $basket->calculateTotal());
}
}

Acceptance Tests
Form: PHPUnit at service level

Acceptance Tests
Form: Behat at UI level
Scenario: Sales tax is applied to basket
Given "Blue Jeans" are priced as €100 in the catalogue
When I add "Blue Jeans" to my shopping basket
Then the basket total should be €120

class BasketUiContext extends MinkContext
{
public function __construct()
{
$this->catalogue = new Catalogue(/* ... */);
}
/**
* @Given :productName is/are priced as :cost in the catalogue
*/
public function priceProduct(ProductName $productName, Cost $cost)
{
$this->catalogue->price($productName, $cost);
}
//...
}

class BasketUiContext extends MinkContext
{
//...
/**
* @When I add :productName to my shopping basket
*/
public function addProductToBasket(ProductName $productName)
{
$this->visitPath('/products/'.urlencode($productName));
$this->getSession()->getPage()->pressButton('Add to Basket');
}
/**
* @Then the basket total should be :cost
*/
public function checkBasketTotal(Cost $cost)
{
$this->assertElementContains('#basket .total', '€120');
}
}

Acceptance Tests
Form: PHPUnit at UI level
class BasketUiTest extends PHPUnit_Framework_TestCase
{
use phpunitminkTestCaseTrait;
{
$catalogue = new Catalogue(/* ... */);
$catalogue->price(new ProductName('Blue Jeans'), new Cost(120));
$page = $this->visit('http://example.com/products/'.urlencode($productName));
$this->getSession()->getPage()->pressButton('Add to Basket');
$this->assertContains(
'€120', $page->find('css', '#basket .total')->getText()
);
}
}

Integration Tests
Goals:
— Test cross-boundary communication
— Test integration with concrete infrastructure

Integration Tests
Scopes:
— Large parts of the system
— Focus on the edges (not core domain)
— Areas where your code interacts with third-party
code

Integration Tests
Times:
— After the feature is implemented in core / contracts
are established
— During integration with real infrastructure
— When you want to get more confidence in
integration
— When cases are not covered by End-to-End
acceptance test

Integration Tests
Best Practices:
— Use tools with existing convenient integrations
— Focus on testing through your API
— Make sure your core domain has an interface

Integration Tests
Form: PHPUnit + DbUnit
class CatalogueTest extends PHPUnit_Extensions_Database_TestCase
{
public function getConnection()
{
$pdo = new PDO(/* ... */);
return $this->createDefaultDBConnection($pdo, 'myDatabase');
}
public function getDataSet()
{
return $this->createFlatXMLDataSet(dirname(__FILE__)
. '/_files/catalogue-seed.xml');
}
// ...
}

class CatalogueTest extends PHPUnit_Extensions_Database_TestCase
{
// ...
public function testProductCanBePriced()
{
$catalogue = new Catalogue(/* ... */);
$catalogue->price(
new ProductName('Blue Jeans'),
new Cost('100')
);
$this->assertEquals(
new Cost('100'),
$catalogue->lookUp(new ProductName('Blue Jeans')
);
}
}

Unit Tests
Goals:
— Test components individually
— Catch errors earlier
— Drive internal design quality
— Document units for other developers

Unit Tests:
Scopes:
— Single classes
— Single classes + value objects?
— Extremely small units of code

Unit Tests:
Times:
— Just before you implement

Unit Tests:
Times:
— Just before you implement
OK, maybe...
— Just after you implement
— If you want to learn more about a class
— But always before you share the code!

Unit Tests
Best Practices
— Write in a descriptive style
— Describe interactions using Test Doubles
— Don't get too hung up on isolation
— Don't touch infrastructure
— Don't test other people's code
— Don't double other people's code?

Unit Tests
Form: PHPUnit
class BasketTest extends PHPUnit_Framework_TestCase
{
{
$catalogue = $this->getMock(Catalogue::class);
$catalogue->method('lookUp')->with(new ProductName('Blue Jeans'))
->willReturn(new Cost('100'));
$basket = new Basket($catalogue);
$basket->add(new ProductName('Blue Jeans'));
$this->assertSame(new Cost('120'), $basket->calculateTotal());
}
}

Unit Tests
Form: PhpSpec
class BasketSpec extends ObjectBehavior
{
function it_applies_sales_tax(Catalogue $catalogue)
{
$catalogue->lookUp(new ProductName('Blue Jeans'))->willReturn(new Cost('100'));
$this->beConstructedWith($catalogue);
$this->add(new ProductName('Blue Jeans'));
$basket->calculateTotal()->shouldBeLike(new Cost('120'));
}
}

5th Dimension -Who?
— Choose the right approaches for your context
— What mix of languages can the team use?
— What styles of testing will add the most value?
— What formats make the most sense to the team?
— How will tests fit into the development process?
There is no right answer, there are many right
answers!

Photo Credits
— "tools" by velacreations (CC) - https://flic.kr/p/
8ZSb3r
— "Components" by Jeff Keyzer (CC) - https://flic.kr/p/
4ZNZp1
— Doctor Who stolen from BBC.co.uk
— Other images used under license

Thank You & Questions?
@s_bergmann
@ciaranmcnulty
https://joind.in/talk/80dbd

Finding the Right Testing Tool for the Job

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (7)

Similar a Finding the Right Testing Tool for the Job

Similar a Finding the Right Testing Tool for the Job (20)

Más de CiaranMcNulty

Más de CiaranMcNulty (18)

Último

Último (20)

Finding the Right Testing Tool for the Job