Annotated slides for phpCE workshop on November 3, 2017. Workshop repository: https://github.com/OndraM/selenium-workshop-phpce The workshop covered: - setting up local development environment (using Docker) - practical examples of functional tests implementation - exploring possibilities of Selenium WebDriver - parallel test execution using Steward - hands-on Page Object design pattern - dealing with asynchronous elements of web-pages (AJAX, JavaScript) - general tips & tricks how to keep a maintainable suite of functional tests in a long-term

1. Workshop:
Functional testing made easy
with PHPUnit & Selenium
3 November 2017
Poland
Ondřej Machulda
ondrej.machulda@gmail.com
@OndraM
Annotated slides

2. $ whoami
Ondřej Machulda
@OndraM
Symfony developer & QA automation engineer
+
Selenium, TDD & QA trainer
php-webdriver library co-maintainer

3. What are you going to learn today?
What is possible to do with Selenium
How to convert manual routine to automated functional tests
Write functional test in a few minutes
Why are functional tests irreplaceable
Execute tests on your machine in a few seconds
Write functional tests in a maintainable manner
Easily find root cause of failed test

4. Cost of fixing bugs
Source: Barry Boehm, Equity Keynote Address 2007
Time is money. So we want to find bugs as soon as possible after they
are created.
Because fixing them earlier in the development stage would be for us
much faster and thus cheaper.
And automated tests are a way how to accomplish this.

5. What do we want from tests? FIRST!
Fast
Isolated
Repeatable
Self-validating
Timely
Automated tests should follow the "FIRST" criteria!
Fast – tests must be executed fast, we need to get
fast feedback if there is a bug.
Isolated – tests should not depend on others, on
order of execution, on global state etc. This will
disallow parallelism.
Repeatable – we want to have same stable result
each time they are run on the same application.
Self-validating – test itself should reliably know
whether it passes or not. No human interaction
should be required to ensure test result.
Timely – tests should be written with the code or
maybe before (when applying Test Driven
Development)

6. Source: https://commons.wikimedia.org/wiki/File:Bicycle_diagram-unif.svg, author Fiestoforo, licence CC BY 3.0
Web app is a machine – like a bicycle. Lots of different parts in different layers, which needs to fit
together to accomplish the one ultimate purpose of the machine – so you can ride the bike.
We usually do test the machine starting from the smallest pieces, because it is easy to test them - you
can easily define their behavior and you can usually easily and fast validate it! Like inputs/outputs of
method and its behavior in edge cases. However, this is not always enough...
If you want to make sure the assembled machine works and everything fits together (eg. you can really
drive & steer the bike), you will test the main business critical scenarios from customer point of view.
Thats why these kind of tests is irreplaceable – its your output quality control. Next one in the QA chain
is usually only the customer, and thats too late :-).

7. Software Testing Ice-cream Cone
Business
Source: https://watirmelon.blog/2012/01/31/introducing-the-software-testing-ice-cream-cone/, author Alister Scott
However, you should not make manual tests or functional tests the testing layer in which you invest the most
- like we see in this ice-cream cone antipattern.

8. Test Pyramid
Timecost
FIRST
5 %
15 %
80 %
Test pyramid is a visual guide showing how much time you should invest in which layer of tests. Why this ratio? The
higher layer, the harder is to write and maintain the tests (they don't fulfil the FIRST criteria) and the slower feedback
you have from them. Unit-tests are fast to write and run, stable and helps with code design – so as a developer you
want to primary write them. But remember the bicycle – you could miss a lot without functional tests.

9. It is useless to run the functional tests only from developers laptop. They are necessary part of continuous
integration and should not be missing in your automated continuous deployment pipeline. As you know – the
faster you find your bugs, the faster and cheaper is to fix them!

10. Source: https://twitter.com/emilbronikowski/status/808669983639216128

11. Source: https://twitter.com/emilbronikowski/status/808669983639216128

12. Source: https://twitter.com/thepracticaldev/status/733908215021199360

13. Selenium
● Just a browser automation library
● Selenium aka Selenium WebDriver
● Platform and language independent
● Supported by all major browser
● Easy setup
● Not depending on any IDE
● Widely used, actively developed
● Many resources on the Internet
The tool for automated functional tests is Selenium - an
open-source library for browser automation. You tell it what
actions in browser should be done and it executes them. The
WebDriver protocol will also soon be W3C standard and it is
implemented in almost all browsers.

14. Selenium can...
● Navigate browser to URL (obviously)
● Find element on the page - via CSS / XPath / ...
● Interact with the element
○ Get it contents, location, styles, attributes...
○ Click on in
○ ...
● Take a screenshot
● Fill and submit form
● Change browser windows size
● Navigate through history (back / forward)
● ...

15. Steward - what it does for you
Test runner
Parallel execution, logs, handle test dependencies...
Convenience layer
Syntax sugar, PHPUnit + php-webdriver integration,
browser setup, screenshots...
https://github.com/lmc-eu/steward
We will also use Steward - an open-source tool built on top of PHPUnit and Symfony components. It is a
test-runner (controlled from CLI) and also extension for PHPUnit, integrating the php-webdriver library
right in your PHPUnit tests.

16. https://git.io/vFsUR
Here is wiki page with links, commands etc. for following examples:

17. Setting up local development environment
● Repository with examples for the workshop:
https://github.com/OndraM/selenium-workshop-phpce
cd [your projects directory]
git clone git@github.com:OndraM/selenium-workshop-phpce
cd selenium-workshop-phpce
git checkout step-0
composer install

18. Start Selenium Server inside Docker
docker run -p 4444:4444 -p 5900:5900
selenium/standalone-chrome-debug:3.6.0
localhost:4444
localhost:5900
vncviewer 127.0.0.1:5900 # on Linux
open vnc://127.0.0.1:5900 # on Mac

19. Executing tests using Steward
● Open new terminal window
○ cd selenium-workshop-phpce
● Show Steward help
○ vendor/bin/steward run --help
● run command
○ two required arguments: environment and browser
○ -vvv to set max output verbosity
vendor/bin/steward run staging chrome
vendor/bin/steward run staging chrome -vvv

20. Implementing first basic test
https://phpce-gz65nia-mxk4rjvb4la6e.eu.platform.sh/
Test scenario:
"Product detail loads basic product information"
1. Open product detail page
2. Check product header is as expected
3. Check product price is as expected
Solution ⇒ git checkout -f step-1 # force
checkout!

21. Source: https://martinfowler.com/bliki/PageObject.html
Page Object is a design pattern from
Martin Fowler, which suggest interacting
with the webpage UI through an
abstraction – ie. an object with methods
mapping the UI structure and UI
interactions. Because in the tests
scenario you want to interact with the UI,
not with its HTML implementation.
Page objects are also a way how to make
your functional tests maintainable in a
long-term.

22. Using Page Object in a test
Solution ⇒ git checkout -f step-1-fixed

23. ● Allows to do what user could do & see
● Copies hierarchy of user interface
● Separates test scenario from its HTML implementation
● Its method should return:
○ Primitive data (string, integer, array of strings...)
○ Or another Page Object
● Assertions should be in tests, not inside page object
Page Object - main principles

24. Extending Page Objects
Test scenario:
"Product could be added to a cart from product detail page"
1. Open product detail page
2. Add product to cart
3. Cart listing is opened
4. Cart contains product added in step 2.

25. Finding elements
Steward simplified syntax (syntax sugar)
$element = $this->findByCss('.foo');
$element = $this->findByXpath('//table/tr/td[2]//a');
$elements = $this->findMultipleByCss('.foo');
Other element finding strategies
findByClass, findById, findByName, findByLinkText,
findByPartialLinkText, findByTag
Traditional php-webdriver way
$element = $this->wd->findElement(WebDriverBy::cssSelector('.foo'));
$elements = $this->wd->findElements(WebDriverBy::cssSelector('.foo'));

26. Element selectors - ID, CSS, Xpath...
● ID - best option, if available
○ $this->findByCss('#login-button');
○ $this->findById('login-button');
● CSS selectors - simple to write
○ $this->findByCss('div.header > button.login');
● XPath - if there is no other way
○ $this->findByXpath(
'//table//a[contains(@href,"sticker-repellendus")]
/ancestor::tr/td/span[@class =
"sylius-quantity"]/input'
);
Beware of selectors stability - too specific vs. too generic

27. Extending Page Objects
Solution ⇒ git checkout -f step-2

28. Logs, screenshots
Test results overview:
● logs/results.xml
● vendor/bin/steward results [-vvv]
Screenshots, HTML snapshots:
● directory logs/
○ TestCaseName-TestName.png
○ TestCaseName-TestName.html

29. Extending Page Objects II.
git checkout -f step-3
Test scenario:
"Multiple products could be added to a cart"
1. Open product A detail page
2. Add product A to cart
3. Cart listing is opened
4. Open product B detail page (hint: just add slug)
5. Add product B to cart
6. Cart listing is opened (hint: addToCart() method returns CartPage)
7. Cart contain two products - A and B
(hint: finish getNamesOfProductsInCart() method of CartPage)

30. Run only one testcase class
vendor/bin/steward run staging chrome -vvv
--pattern ProductDetailTest.php
Run only one test method of one testcase class:
vendor/bin/steward run staging chrome -vvv
--pattern ProductDetailTest.php
--filter shouldAddMultipleProductsToCart

31. Extending Page Objects II.
git checkout -f step-3
Test scenario:
"Multiple products could be added to a cart"
1. Open product A detail page
2. Add product A to cart
3. Cart listing is opened
4. Open product B detail page (hint: just add slug)
5. Add product B to cart
6. Cart listing is opened (hint: addToCart() method returns CartPage)
7. Cart contain two products - A and B
(hint: finish getNamesOfProductsInCart() method of CartPage)
Solution ⇒ git checkout -f step-3-fixed

32. Forms
$element = $this->findByName('name');
$element->sendKeys('Some text')
$element->clear()
$element->submit()
<select> elements
$element = $this->findByName('country');
$select = new WebDriverSelect($element);
$select->selectByVisibleText('Poland');
$select->selectByValue('pl');
$select->selectByIndex(13);

33. Forms example
Test scenario:
"User submits order with products in cart"
1. Prerequisite: have a product in cart
2. Open cart listing
3. Go to Checkout
4. Address form is shown
5. Fill required fields
○ e-mail, first name, last name, street, country (select), city, postcode
6. Submit form
7. Shipping details form is shown
8. ...
git checkout -f step-4
Solution ⇒ git checkout -f step-4-fixed

34. Waiting ("explicit wait")
● Web is not a synchronous place to be...
● Selenium doesn't know when your actions are finished
(Except loading page via get())
○ Loading page after user clicks to a link
○ Submitting a form
○ Displaying modal window
○ AJAX
○ ...

35. Waiting - syntax sugar waitFor...() methods
$this->waitForCss('.#modal p');
// or
$this->wd->wait()->until(
WebDriverExpectedCondition::presenceOfElementLocated(
WebDriverBy::cssSelector('#modal p')
)
);
$this->waitForTitle('Search results');
// or
$this->wd->wait()->until(
WebDriverExpectedCondition::titleIs('Search results')
);

36. Advanced WebDriverExpectedConditions
$this->wd->wait()->until(
WebDriverExpectedCondition::titleIs('text')
);
● titleContains('text'); titleMatches('/.../')
● presenceOfElementLocated($by)
● visibilityOfElementLocated($by)
● elementTextContains($by, 'text'); elementTextIs($by,
'text')
● alertIsPresent()
● urlContains('?login=success'); urlMatches('/.../');
● numberOfWindowsToBe(2)
● not()
● custom callback
https://github.com/facebook/php-webdriver/wiki/HowTo-Wait

37. Explicit wait example
Test scenario:
"Registered but unlogged user could login during checkout"
1. Prerequisite: have a product in cart
2. Open cart listing
3. Go to Checkout
4. Address form is shown
5. Fill e-mail and password of existing user
○ E-mail: user@example.com / Password: sylius
6. E-mail input will no longer be shown
7. Name of logged user will be shown in the header
8. ...
Solution ⇒ git checkout -f step-5-fixed
git checkout -f step-5

38. Within one test-case class
/**
* @test
* @depends shouldRegisterUser
* @param string $username
*/
public function shouldLoginUser( $username)
{
...
}
Test dependencies
Amongst more than one test-case
● annotation @delayMinutes and @delayAfter on testcase class
/**
* @delayMinutes 2
* @delayAfter MyContactFormTest
*/
● Steward allows to pass data between testcases
https://github.com/lmc-eu/steward/wiki/Test-dependencies

39. Test dependencies are evil
☠

40. File upload
$input = $this->findByName('upload');
$input->setFileDetector(new LocalFileDetector())
->sendKeys('/path/to/image.jpg');
https://simple-u6rzw4q-mxk4rjvb4la6e.eu.platform.sh/upload-simple.html
git checkout -f step-6
Solution ⇒ git checkout -f step-6-fixed

41. JavaScript execution
Execute synchronous script (doesn't block test execution)
$this->wd->executeScript('
document.body.style.backgroundColor = "red";
');
Execute asynchronous script (wait for callback)
$this->driver->executeAsyncScript('
var callback = arguments[arguments.length - 1];
...
callback();
');
https://simple-u6rzw4q-mxk4rjvb4la6e.eu.platform.sh/hidden.html
git checkout -f step-7
Solution ⇒ git checkout -f step-7-fixed

42. Selenium can do much more
● Change window size ($this->wd->manage())
● Navigate through browsing history (navigate())
● Switch to different window / iframe / alert (switchTo())
○ github.com/facebook/php-webdriver/wiki/Alert,-tabs,-frames,-iframes
● Move mouse over element (action())
● Run your tests in different browsers
● Run Chrome / Firefox in headless mode (R.I.P. PhantomJS)

43. Other Steward features
● Annotation @noBrowser
○ Eg. seeding test data via API/database
○ Example: SeedDataTest.php
● Set default window size
● Sauce Labs / BrowserStack / TestingBot integration
○ Example: https://saucelabs.com/u/OndraM
● Pass capabilities (= configuration) to the browser
● Generate test execution timeline
● And more: https://github.com/lmc-eu/steward-example

44. Debugging tests via Xdebug & PHPStorm
$ vendor/bin/steward run dev chrome --xdebug
https://github.com/lmc-eu/steward/wiki/Debugging-Selenium-tests-With-Steward

45. Best practices
● Less functional tests is more
● Functional tests are not second-class citizen
● Have a strategy to maintain them
● Do not have long-term broken tests
○ markTestSkipped()
● Use Page Objects
● Unstable (flaky) tests - find root cause, no workarounds
● Tests must be fast
● Implicit wait instead of sleep
● Use self-describing names for tests, methods etc.

46. Other options how to use Selenium in PHP
There are multiple options how to run Selenium tests in PHP - choose the one that fits your needs!
Steward - integrates php-webdriver into classic PHPUnit-styled tests and also provides parallelization.
Codeception - complex test framework for all layers of tests, which adds BDD-like layer on top of PHPUnit.
Laravel Dusk - another semantics for writing selenium tests in a "Laravel" way, it also includes Laravel integration.
Behat + Mink - BDD way of writing test scenarios, however uses unmaintained library for Selenium integration
Phpunit-selenium - old and outdated extension for PHPUnit
● Steward
● Codeception
● Laravel Dusk
● Behat + Mink
● phpunit-selenium (PHPUnit_Extensions_Selenium2TestCase)

47. Ondřej Machulda
ondrej.machulda@gmail.com
www.ondrejmachulda.cz
@OndraM
A that's all for today!
✌