Opinions differ on test driven development, and some very smart developers are debating TDD and test-first vs system tests at a very high level. However, down in the trenches when building and supporting complex eCommerce websites built around Magento we live in a very different world with a number of constraints about what we can, can’t and absolutely must test. Magento itself, clients and budgets put constraints on what we are able to test, but testing is a critical part of keeping a site stable. I will briefly discuss the options available for testing Magento sites and how you could be testing your work with Magento.

8. What you can, can’t and must test in Magento
!
presented by joshua warren
!
@joshuaSWarren

14. Don't worry about finding the perfect tools
to test Magento. Just start testing.
#MM14NY

15. creatuity.com/mmny
!
@joshuaswarren

17. 75% of Magento developers surveyed are not
conducting any automated testing of their
Magento projects
#MM14NY

18. Less than 20% of developers not testing
Magento aren't testing because they think
Magento is too complex to test
#MM14NY

19. lack of budget is the #1 reason developers
aren’t testing their magento projects
#MM14NY

20. Lack of knowledge is the 2nd most common
reason for not performing automated testing
of magento
#MM14NY

21. Lack of time is the 3rd most common reason
for not performing automated testing of
magento
#MM14NY

22. testing is important

24. automated testing allows magento
developers to focus on building new features
#MM14NY

25. with automated testing, bugs are found and
fixed as part of development.
#MM14NY

26. one interruption to fix a bug that made it past
QA costs the best developers 15 minutes of
lost productivity
#MM14NY

27. SOURCE: ‘PROGRAMMER INTERRUPTED’ - http://blog.ninlabs.com/2013/01/programmer-interrupted/

28. a well designed automated testing system
reduces interruptions and increases
productivity and happiness.
#MM14NY

29. types of tests used to test magento

30. unit testing
logical
analytical
great for ideal projects

32. unit testing
magento’s flexibility greatly
complicates unit testing

33. unit testing
great for extensions
github.com/EcomDev/EcomDev_PHPUnit
@IvanChepurnyi

34. it’s not that you can’t test magento - you
can’t unit test magento, but there’s more to
testing than just unit tests
#MM14NY

35. 100% of developers surveyed that are testing
magento are using functional tests
#MM14NY

36. functional testing
focuses on the entire site as a system
doesn’t care what function is being called
tests that the site works the way client wants

38. with full test coverage, functional tests
confirm that your magento site still works
as you add new features
#MM14NY

39. one more thing…
only 50% of developers surveyed are
conducting performance testing
!
performance testing is a must

40. performance testing
not just for sysadmins
not just for front-end developers
bad performance = bad sales

41. performance testing
backend developers don’t always remember
that extensions, integrations and new
features can slow the site down

42. when testing your magento projects, include
performance testing - poor performance is a
bug
#MM14NY

43. slow code should be optimized or removed
just like any other bug would be
#MM14NY

44. how to test magento

45. testing servers
bamboo - commercial, general purpose
!
jenkins - open source, general purpose
!
phpci - open source, PHP-focused

46. only 1 developer reported using phpci for
testing magento; 50% are using bamboo, 50%
are using jenkins
#MM14NY

47. jenkins vs bamboo
jenkins is free
Bamboo $10 starter license
!
Try both, pick whichever you like better

48. which testing server you use isn’t important -
the fact that you have a testing server and
are testing is.
#MM14NY

49. 100% of developers who said they are
performing functional tests are using
selenium
#MM14NY

51. selenium test cases
selenese vs phpunit
test standard magento features
test your custom features

53. gatling test cases
scripted in scala
normal visitor workflow
randomized delays

54. gatling test cases
specify number of simultaneous visitors and
overall test length
!
for example…

57. gatling test cases should be set for the
maximum simultaneous users you expect and a
timeframe of 60-120 seconds
#MM14NY

58. automated tests of magento should be fast -
test each commit and receive pass/fail
before you move to your next task
#MM14NY

59. gatling test cases
develop baseline at start of project
!
if tests are 20% slower than baseline,
consider the test failed

60. what’s Next?

62. testing magento 1 sites
can’t - unit testing
can - functional testing
must - performance testing

63. testing magento
automated testing saves time, money & sanity

64. let’s keep talking
what did I get wrong?
TDD vs BDD?
BeHat?
@joshuaswarren / joshuawarren.com

65. behat, bdd vs tdd, unit testing in magento 1 -
tweet at @Joshuaswarren with what you
disagree with
#MM14NY

66. but wait, there’s more!
in-depth talk at phpworld
november, washington dc
world.phparch.com

67. spread the word
educate our peers, managers and clients
!
summer 2015 - automated testing is the norm

68. spread the word
Creatuity will be open sourcing our testing
tools and test cases
!
foundation for fast, flexible, reusable set of
functional and performance tests

69. coming soon to github - @joshuaswarren and
@creatuity are releasing a foundation for
performance and functional testing of
magento
#MM14NY

70. thank you!
creatuity.com/mmny
!
@joshuaswarren