Did you know that WordPress has an automated test suite? It contains well over 1500 integration tests and growing. However one of the primary culprits of WordPress is in the quality of its plugins. Most plugins don't have an automated test suite you can run to verify all features are working as expected, and fail gracefully.
In this talk, Ptah will introduce you to automated testing in WordPress using PHPUnit. We will cover concepts like unit testing, integration testing and end-to-end testing with examples in WordPress. You will leave the talk equipped with practical knowledge and ready to start adding an automated test suite to your plugins.
2. Ptah (Pirate) Dunbar
●
Started with WordPress and PHP
in ‘05
●
Contributing developer to
WordPress, BuddyPress, bbPress
●
Full stack Web Developer
●
Architect at LiveNinja.com
●
WPMIA co-organizer and
SoFloPHP member
☠ Became Pirate Dunbar
#dc4d - Automated Testing in WordPress with @ptahdunbar
3. Ptah (Pirate) Dunbar
●
Started with WordPress and PHP
in ‘05
●
Contributing developer to
WordPress, BuddyPress, bbPress
●
Full stack Web Developer
●
Architect at LiveNinja.com
●
WPMIA co-organizer and
SoFloPHP member
☠ Became Pirate Dunbar
#dc4d - Automated Testing in WordPress with @ptahdunbar
4. Ptah (Pirate) Dunbar
●
Started with WordPress and PHP
in ‘05
●
Contributing developer to
WordPress, BuddyPress, bbPress
●
Full stack Web Developer
●
Architect at LiveNinja.com
●
WPMIA co-organizer and
SoFloPHP member
☠ Became Pirate Dunbar
#dc4d - Automated Testing in WordPress with @ptahdunbar
5. Agenda
In one hour
● Understand automated testing concepts,
ideas and best practices.
● Learn PHPUnit basics and the WordPress testsuite.
● Resources and homework
#dc4d - Automated Testing in WordPress with @ptahdunbar
8. “The result is that a lot of the
plugins are written in poor code
and turn out to be poorly
compatible with other plugins”
— Yoast
http://yoast.com/plugin-future/
#dc4d - Automated Testing in WordPress with @ptahdunbar
16. Automated Testing
A scripted process that
invokes your app to test
features and compares the
outcome with expected
results.
#dc4d - Automated Testing in WordPress with @ptahdunbar
26. PHPUnit
Terminology
● Test Case
A set of conditions that you set up in order
to assert expected outcome.
#dc4d - Automated Testing in WordPress with @ptahdunbar
27. PHPUnit
Terminology
● Test Case
A set of conditions that you set up in order
to assert expected outcome.
● Test Class
A collection of test cases, extends PHPUnit
#dc4d - Automated Testing in WordPress with @ptahdunbar
28. PHPUnit
Terminology
● Test Case
A set of conditions that you set up in order
to assert expected outcome.
● Test Class
A collection of test cases, extends PHPUnit
● Test Suite
A collection of test classes
#dc4d - Automated Testing in WordPress with @ptahdunbar
29. PHPUnit
TEST CLASS
<?php
// test class
class CalTest extends PHPUnit_Framework_TestCase
{
// test case
public function testAddReturnsSumOfTwoPositiveIntegers()
{
// assert stuff.
}
}
#dc4d - Automated Testing in WordPress with @ptahdunbar
30. PHPUnit
TEST CLASS
<?php
// test class
class CalTest extends PHPUnit_Framework_TestCase
{
// test case
public function testAddReturnsSumOfTwoPositiveIntegers()
{
// assert stuff.
}
}
#dc4d - Automated Testing in WordPress with @ptahdunbar
33. PHPUnit
phpunit.xml - configuration file for PHPUnit
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php">
<testsuites>
<testsuite name="tests">
<directory suffix="Test.php">tests/</directory>
</testsuite>
</testsuites>
</phpunit>
#dc4d - Automated Testing in WordPress with @ptahdunbar
34. PHPUnit
phpunit.xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php">
<testsuites>
<testsuite name="tests">
<directory suffix="Test.php">tests/</directory>
</testsuite>
</testsuites>
</phpunit>
Configure your test suite location
#dc4d - Automated Testing in WordPress with @ptahdunbar
35. PHPUnit
phpunit.xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php">
<testsuites>
<testsuite name="integration">
<directory suffix="Test.php">tests/integration</directory>
</testsuite>
<testsuite name="acceptance">
<directory suffix="Test.php">tests/acceptance</directory>
</testsuite>
</testsuites>
</phpunit>
Configure your test suite location
#dc4d - Automated Testing in WordPress with @ptahdunbar
36. PHPUnit
phpunit.xml
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="vendor/autoload.php">
<testsuites>
<testsuite name="tests">
<directory suffix="Test.php">tests/</directory>
</testsuite>
</testsuites>
</phpunit>
Bootstrap file is included before any tests run
#dc4d - Automated Testing in WordPress with @ptahdunbar
43. PHPUnit
function testThatItsTestingTime()
{
1. Arrange (the context/dependencies)
2. Act (call the method/trigger the action)
3. Assert (check for the expected behavior)
}
#dc4d - Automated Testing in WordPress with @ptahdunbar
67. PHPUnit
FAIL
There was 1 failure:
1) Tests_Basic::test_readme
readme.html's version needs to be updated to 3.9.
Failed asserting that '3.8' matches expected '3.9'.
/private/tmp/wordpress-tests/tests/phpunit/tests/basic.php:29
#dc4d - Automated Testing in WordPress with @ptahdunbar
89. WordPress Testsuite
●
Navigate to site URL (Updates globals)
$this->get_url($url);
●
Test WP_Query for Conditionals (is_page, is_single, is_404)
$this->assertQueryTrue($arg1, $arg2, ...);
●
Test for Errors
$this->assertWPError($thing);
●
Genereate WordPress data fixtures
$this->factory->post->create_and_get();
$this->factory->comment->create_post_comments($pid, 100);
$this->factory->user->create_many(5);
$this->factory->blog->create();
and more…
#dc4d - Automated Testing in WordPress with @ptahdunbar
90. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
// Act
// Assert
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
91. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
// Act
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
92. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
// Act
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
93. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
// Act
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
94. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
// Act
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
95. ./vendor/bin/phpunit
WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
// Act
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
96. ./vendor/bin/phpunit
WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
// test cases
function testRedirectForDateBasedPermalinks()
{
// Arrange
$customWP = new WPCustomization;
$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);
Time: 148ms, Memory: 2.75Mb
OK: (1// Act 1 assertions)
test,
$customWP->deprecate_unused_pages();
$this->go_to('/2007/');
// Assert
$this->assertQueryTrue( 'is_404' );
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
97. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getRequiredPlugins
*/
function testAllRequiredPluginsAreActive($plugin)
{
// Assert
$this->assertTrue( is_plugin_active($plugin),
sprintf('%s is not activated.', $plugin) );
}
function getRequiredPlugins()
{
return [
[‘hello.php’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
98. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getRequiredPlugins
*/
function testAllRequiredPluginsAreActive($plugin)
{
// Assert
$this->assertTrue( is_plugin_active($plugin),
sprintf('%s is not activated.', $plugin) );
}
function getRequiredPlugins()
{
return [
[‘hello.php’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
99. ./vendor/bin/phpunit
WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getRequiredPlugins
*/
function testAllRequiredPluginsAreActive($plugin)
{
// Assert
$this->assertTrue( is_plugin_active($plugin),
sprintf('%s is not activated.', $plugin) );
}
function getRequiredPlugins()
{
return [
[‘hello.php’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
100. ./vendor/bin/phpunit
WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getRequiredPlugins
*/
function testAllRequiredPluginsAreActive($plugin)
{
// Assert
$this->assertTrue( is_plugin_active($plugin),
sprintf('%s is not activated.', $plugin) );
}
Time: 148ms, Memory: 2.75Mb
OK: (1 test, 1 assertions)
function getRequiredPlugins()
{
return [
[‘hello.php’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
101. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getWPOptions
*/
function testWPOptionSettingsAreConfigured($option_name, $option_value)
{
// Assert
$this->assertSame($option_value, get_option($option_name));
}
function getWPOptions()
{
return [
[‘home’, ‘http://example.org/wp/’],
[‘siteurl’, ‘http://example.org/’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
102. WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getWPOptions
*/
function testWPOptionSettingsAreConfigured($option_name, $option_value)
{
// Assert
$this->assertSame($option_value, get_option($option_name));
}
function getWPOptions()
{
return [
[‘home’, ‘http://example.org/wp/’],
[‘siteurl’, ‘http://example.org/’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
103. ./vendor/bin/phpunit
WordPress Testsuite
<?php
class WPCustomizationTest extends WP_UnitTestCase
{
/**
* @dataProvider getWPOptions
*/
function testWPOptionSettingsAreConfigured($option_name, $option_value)
{
// Assert
$this->assertSame($option_value, get_option($option_name));
}
function getWPOptions()
{
return [
[‘home’, ‘http://example.org/wp/’],
[‘siteurl’, ‘http://example.org/’],
];
}
}
plugin/tests/integration/WPCustomizationTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
106. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Framework_TestCase
{
protected function setUp()
{
}
public function testUserCanLogInViaTwitter()
{
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
107. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
}
public function testUserCanLogInViaTwitter()
{
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
108. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
public function testUserCanLogInViaTwitter()
{
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
109. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
public function testUserCanLogInViaTwitter()
{
$this->open("/");
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
110. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
public function testUserCanLogInViaTwitter()
{
$this->open("/");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
111. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
public function testUserCanLogInViaTwitter()
{
$this->open("/");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");
$this->click("css=img[alt="Twitter"]");
$this->waitForPageToLoad("30000");
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
112. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
public function testUserCanLogInViaTwitter()
{
$this->open("/");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");
$this->click("css=img[alt="Twitter"]");
$this->waitForPageToLoad("30000");
$this->assertContains( ‘dashboard’, $this->title() );
}
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
113. Acceptance Testing
<?php
class ConnectTest extends PHPUnit_Extensions_Selenium2TestCase
{
protected function setUp()
{
$this->setBrowser("*chrome");
$this->setBrowserUrl("https://wpss.dev/");
}
Time: 148ms, Memory: 2.75Mb
public function testUserCanLogInViaTwitter()
{
$this->open("/");
$this->click("link=Log in");
$this->waitForPageToLoad("30000");
$this->click("css=img[alt="Twitter"]");
$this->waitForPageToLoad("30000");
$this->assertContains( ‘dashboard’, $this->title() );
}
OK: (1 test, 1 assertions)
}
plugin/tests/acceptance/ConnectTest.php
#dc4d - Automated Testing in WordPress with @ptahdunbar
114. Acceptance
Selenium IDE Plugin
●
Visually navigate throughout your
site and generate a PHPUnit
test case.
●
Download Extension
○ http://www.seleniumhq.
org/projects/ide/
●
Download PHPUnit Formatter
○ https://addons.mozilla.org/enUS/firefox/addon/seleniumide-php-formatters/
#dc4d - Automated Testing in WordPress with @ptahdunbar
115. Acceptance
Selenium IDE Plugin
●
Visually navigate throughout your
site and generate a PHPUnit
test case.
●
Download Extension
○ http://www.seleniumhq.
org/projects/ide/
●
Download PHPUnit Formatter
○ https://addons.mozilla.org/enUS/firefox/addon/seleniumide-php-formatters/
#dc4d - Automated Testing in WordPress with @ptahdunbar
116. How can we be
confident that our tests
cover everything?
#dc4d - Automated Testing in WordPress with @ptahdunbar
118. Testing boundaries
●
(User) Acceptance Testing
○
Verify that all features are done done.
○
Black-box testing, no knowledge of internals.
#dc4d - Automated Testing in WordPress with @ptahdunbar
119. Testing boundaries
●
(User) Acceptance Testing
○
○
●
Verify that all features are done done.
Black-box testing, no knowledge of internals.
Integration Testing
○
Test WordPress settings/configuration;
○
Compatibility between plugins and themes.
#dc4d - Automated Testing in WordPress with @ptahdunbar
120. Testing boundaries
●
(User) Acceptance Testing
○
○
●
Verify that all features are done done.
Black-box testing, no knowledge of internals.
Integration Testing
○
○
●
Test WordPress settings/configuration,
Compatibility between plugins and themes
Unit Testing
○
Test class methods and functions in isolation, zero dependencies
○
Does one “behavoir”
#dc4d - Automated Testing in WordPress with @ptahdunbar
121. Testing boundaries
●
(User) Acceptance Testing
Verify that all features are done done,
black-box testing, no knowledge of
Acceptance
Testing
internals
●
Integration Testing
Test WordPress settings/configuration,
compatibility between plugins and
Integration Testing
themes
●
Unit Testing
Unit Testing
Test class methods and functions in
isolation, zero dependencies,
does one “behavoir”.
#dc4d - Automated Testing in WordPress with @ptahdunbar
122. Testing boundaries
●
(User) Acceptance Testing
Verify that all features are done done,
black-box testing, no knowledge of
Acceptance
Testing
internals
●
Integration Testing
Test WordPress settings/configuration,
compatibility between plugins and
Integration Testing
themes
●
Unit Testing
Unit Testing
Test class methods and functions in
isolation, zero dependencies,
does one “behavoir”.
#dc4d - Automated Testing in WordPress with @ptahdunbar
123. Testing boundaries
●
(User) Acceptance Testing
Verify that all features are done done,
black-box testing, no knowledge of
Acceptance
Testing
internals
●
Integration Testing
Test WordPress settings/configuration,
compatibility between plugins and
Integration Testing
themes
●
Unit Testing
Unit Testing
Test class methods and functions in
isolation, zero dependencies,
does one “behavoir”.
#dc4d - Automated Testing in WordPress with @ptahdunbar
124. What to tests?
● Test plugin works in various WordPress setups
○ Does it work under multisite?
○ What about a custom content directory?
● Test all code paths in functions and methods
● Test compatiblity between most popular plugins
● Test that default pages exists
#ATWP // Automated Testing in WordPress // @ptahdunbar
125. What to tests?
● Test for theme support
● Test that post formats contain property elements
● Test any required assets that need to be loaded in
templates
● Test for required elements on a page
● Verify search results template displays search term
● Verify SEO meta tags
#ATWP // Automated Testing in WordPress // @ptahdunbar
126. What to not tests?
1. WordPress APIs
#ATWP // Automated Testing in WordPress // @ptahdunbar
127. What to not tests?
1. WordPress APIs
2. PHP language features
#ATWP // Automated Testing in WordPress // @ptahdunbar
128. What to not tests?
1. WordPress APIs
2. PHP language features
3. Third party vendor code
#ATWP // Automated Testing in WordPress // @ptahdunbar
129. Getting into the groove
#ATWP // Automated Testing in WordPress // @ptahdunbar
130. Getting into the groove
● Build out templates
#ATWP // Automated Testing in WordPress // @ptahdunbar
131. Getting into the groove
● Build out templates
○ Create HTML/CSS
#ATWP // Automated Testing in WordPress // @ptahdunbar
132. Getting into the groove
● Build out templates
○ Create HTML/CSS
○ Identify dynamic elements and their data
structure
#ATWP // Automated Testing in WordPress // @ptahdunbar
133. Getting into the groove
● Build out templates
○ Create HTML/CSS
○ Identify dynamic elements and their data
structure
○ Label them and fill them with dummy data
#ATWP // Automated Testing in WordPress // @ptahdunbar
134. Getting into the groove
○ Verbally state your trying to do
#ATWP // Automated Testing in WordPress // @ptahdunbar
135. Getting into the groove
○ Verbally state your trying to do
○ Verbally explain what the code does
#ATWP // Automated Testing in WordPress // @ptahdunbar
136. Getting into the groove
○ Verbally state your trying to do
○ Verbally explain what the code does
○ Do this alone or with a fellow dev :)
#ATWP // Automated Testing in WordPress // @ptahdunbar
138. Get started
“A Walking Skeleton is a tiny implementation of the thinnest
possible slice of real functionality that we can automatically
build, deploy and test end-to-end.”
●
Download WP Skeleton Family
○
https://github.com/ptahdunbar/wp-skeleton-site
○
https://github.com/ptahdunbar/wp-skeleton-plugin
○
https://github.com/ptahdunbar/wp-skeleton-theme
#dc4d - Automated Testing in WordPress with @ptahdunbar
139. Resources
● Art of Unit Testing (.NET)
○ https://leanpub.com/u/royosherove
○ Udemy Five day course
● #GOOS Book (Java)
● XUnit Test Patterns (Java)
● Grumpy Books (PHP)
○ https://leanpub.com/u/chartjes
● Misko Hevery
#dc4d - Automated Testing in WordPress with @ptahdunbar
140. Homework!
TODO
● Learn moar PHPUnit features
○ data providers,
○ mocks and stubs
○ wordpress testsuite
● Goal: Write at least 100 assertions!
#dc4d - Automated Testing in WordPress with @ptahdunbar
145. Automated Testing
is your professional duty
as a developer
#dc4d - Automated Testing in WordPress with @ptahdunbar
146. Thank you
Automated Testing in WordPress
Pirate Dunbar
@ptahdunbar
yarr@piratedunbar.com
Rate this talk:
https://joind.in/10115
#dc4d - Automated Testing in WordPress with @ptahdunbar